Omkar2118/datasets huggingface dataset:
SAFIM
Full name: Syntax-Aware Fill-In-the-Middle

Syntax-Aware Fill-in-the-Middle (SAFIM) is a benchmark for evaluating Large Language Models (LLMs) on the code Fill-in-the-Middle (FIM) task. SAFIM has three subtasks: Algorithmic Block Completion, Control-Flow Expression Completion, and API Function Call Completion. SAFIM is sourced from code submitted from April 2022 to January 2023 to minimize the impact of data contamination on evaluation results.

- Authors: [Linyuan Gong](https://gonglinyuan.com), [Sida Wang](https://www.sidaw.xyz/), [Mostafa Elhoushi](https://www.linkedin.com/in/mostafaelhoushi), [Alvin Cheung](https://people.eecs.berkeley.edu/~akcheung/)
- Paper: [https://arxiv.org/abs/2403.04814](https://arxiv.org/abs/2403.04814)
- Huggingface Dataset: [https://huggingface.co/datasets/gonglinyuan/safim](https://huggingface.co/datasets/gonglinyuan/safim)
- Leaderboard: [https://safimbenchmark.com](https://safimbenchmark.com)
- Code & Submission Instructions: [https://github.com/gonglinyuan/safim](https://github.com/gonglinyuan/safim
)

The SAFIM benchmark is partially derived from problem descriptions and code solutions from https://codeforces.com. According to the license of CodeForces, you may publish the texts of Codeforces problems in any open sources, but you must preserve a direct link to the site.

Introducing paper: Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks