Omkar2118/datasets huggingface dataset:
destruction
Full name: desctruction detection dataset

This dataset contains pre and post destruction images and also segmentation labels for test images.

Introducing paper: Destruction from sky: Weakly supervised approach for destruction detection in satellite imagery
AlpacaEval

The AlpacaEval set contains 805 instructions form self-instruct, open-assistant, vicuna, koala, hh-rlhf. Those were selected so that the AlpacaEval ranking of models on the AlpacaEval set would be similar to the ranking on the Alpaca demo data.

Introducing paper: AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback
Dronescapes

a large video dataset captured with UAVs in different complex real-world scenes, with multiple representations, suitable for multi-task learning.
BlINK

BLINK is a new benchmark for multimodal language models (LLMs) that focuses on core visual perception abilities not found in other evaluations¹². 

Most of the BLINK tasks can be solved by humans “within a blink” (e.g., relative depth estimation, visual correspondence, forensics detection, and multi-view reasoning)¹². However, these perception-demanding tasks cast significant challenges for current multimodal LLMs because they resist mediation through natural language¹².

BLINK reformats 14 classic computer vision tasks into 3,807 multiple-choice questions, paired with single or multiple images and visual prompting¹². While humans get 95.70% accuracy on average, BLINK is surprisingly challenging for existing multimodal LLMs: even the best-performing GPT-4V and Gemini achieve accuracies of 51.26% and 45.72%, only 13.17% and 7.63% higher than random guessing, indicating that such perception abilities have not “emerged” yet in recent multimodal LLMs¹².

The BLINK benchmark is designed to stimulate the community to help multimodal LLMs catch up with human-level visual perception¹². It includes diverse visual prompting, beyond recognition perception, and visual commonsense¹. The benchmark is available on GitHub¹.

(1) zeyofu/BLINK_Benchmark - GitHub. https://github.com/zeyofu/BLINK_Benchmark.
(2) BLINK: Multimodal Large Language Models Can See but Not Perceive. https://arxiv.org/abs/2404.12390.
(3) Releases · zeyofu/BLINK_Benchmark · GitHub. https://github.com/zeyofu/BLINK_Benchmark/releases.
(4) undefined. https://doi.org/10.48550/arXiv.2404.12390.

Introducing paper: BLINK: Multimodal Large Language Models Can See but Not Perceive