Omkar2118/datasets huggingface dataset:
SWE-bench-lite

SWE-bench is a dataset that tests systems’ ability to solve GitHub issues automatically. The dataset collects 2,294 Issue-Pull Request pairs from 12 popular Python repositories. Evaluation is performed by unit test verification using post-PR behavior as the reference solution.

SWE-bench lite is a subset of SWE-bench, which is curated to make evaluation less costly and more accessible. SWE-bench lite  comprises 300 instances that have been sampled to be more self-contained, with a focus on evaluating functional bug fixes.

Introducing paper: SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
T$^3$Bench

T$^3$Bench is the first comprehensive text-to-3D benchmark containing diverse text prompts of three increasing complexity levels that are specially designed for 3D generation (300 prompts in total). To assess both the subjective quality and the text alignment, we propose two automatic metrics based on multi-view images produced by the 3D contents. The quality metric combines multi-view text-image scores and regional convolution to detect quality and view inconsistency. The alignment metric uses multi-view captioning and Large Language Model (LLM) evaluation to measure text-3D consistency.

Introducing paper: T$^3$Bench: Benchmarking Current Progress in Text-to-3D Generation