Omkar2118/datasets huggingface dataset:
RepoQA

RepoQA is a benchmark that aims to exercise the long-context code understanding ability of Language Learning Models (LLMs)². It supports repositories from 5 programming languages: Python, C++, TypeScript, Rust, and Java². 

The goal of RepoQA is to evaluate LLMs on long-context tasks that can reflect real-life uses². For example, it includes a function called `repoqa.search_needle_function` that evaluates model outputs and computes scores after text generation³. 

(1) Releases · evalplus/repoqa · GitHub. https://github.com/evalplus/repoqa/releases.
(2) RepoQA: Evaluating Long-Context Code Understanding - GitHub. https://github.com/evalplus/repoqa.
(3) BrowserBench.org — Browser Benchmarks. https://browserbench.org/.
EvoEval

EvoEval is a holistic benchmark suite created by evolving HumanEval problems¹. It contains 828 new problems across 5 semantic-altering and 2 semantic-preserving benchmarks¹. EvoEval allows evaluation and comparison across different dimensions and problem types, such as Difficult, Creative, or Tool Use problems¹.

The goal of EvoEval is to provide a comprehensive evaluation of Language Learning Models' (LLMs) coding abilities². It was introduced to address the limitations of existing benchmarks, which contain only a very limited set of problems, both in quantity and variety². 

EvoEval can be used to further evolve arbitrary problems to keep up with advances and the ever-changing landscape of LLMs for code². It comes complete with a leaderboard, ground truth solutions, robust test cases, and evaluation scripts to easily fit into your evaluation pipeline¹.

(1) evo-eval/evoeval: EvoEval: Evolving Coding Benchmarks via LLM - GitHub. https://github.com/evo-eval/evoeval.
(2) Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval .... https://arxiv.org/abs/2403.19114.
(3) evoeval (EvoEval) - Hugging Face. https://huggingface.co/evoeval.
(4) EvoEval: Evolving Coding Benchmarks via LLM. https://evo-eval.github.io/.
(5) Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval .... https://paperswithcode.com/paper/top-leaderboard-ranking-top-coding.
(6) undefined. https://doi.org/10.48550/arXiv.2403.19114.

Introducing paper: Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM