Omkar2118/datasets huggingface dataset:
MULTI

MULTI-Benchmark is a cutting-edge benchmark for evaluating Multimodal Large Language Models (MLLMs). It is designed to test the understanding of complex tables and images, and reasoning with long context¹. Here are some key features of MULTI-Benchmark:

- **Multimodal Inputs**: MULTI-Benchmark provides multimodal inputs and requires responses that are either precise or open-ended, reflecting real-life examination styles¹.
- **Variety of Tasks**: It includes over 18,000 questions and challenges MLLMs with a variety of tasks, ranging from formula derivation to image detail analysis and cross-modality reasoning¹.
- **MULTI-Elite and MULTI-Extend**: It introduces MULTI-Elite, a 500-question selected hard subset, and MULTI-Extend, with more than 4,500 external knowledge context pieces¹.
- **Evaluation**: The evaluation indicates significant potential for MLLM advancement, with GPT-4V achieving a 63.7% accuracy rate on MULTI, in contrast to other MLLMs scoring between 28.5% and 55.3%¹.

(1) ️ MULTI-Benchmark: Multimodal Understanding Leaderboard ... - GitHub. https://github.com/OpenDFM/MULTI-Benchmark.
(2) ️ MULTI-Benchmark: Multimodal Understanding Leaderboard ... - GitHub. https://github.com/OpenDFM/MULTI-Benchmark.
(3) ️ MULTI-Benchmark: Multimodal Understanding Leaderboard .... https://github.com/OpenDFM/MULTI-Benchmark/blob/main/README_zh.md.
(4) MultiBench Dataset | Papers With Code. https://paperswithcode.com/dataset/multibench.
(5) [2107.07502] MultiBench: Multiscale Benchmarks for Multimodal .... https://arxiv.org/abs/2107.07502.
(6) MultiBench: Multiscale Benchmarks for Multimodal .... https://www.x-mol.com/paper/1416118156688826368?adv.
(7) undefined. https://avatars.githubusercontent.com/u/139950066?v=4.
(8) undefined. https://github.com/OpenDFM/MULTI-Benchmark/blob/main/README_zh.md?raw=true.
(9) undefined. https://desktop.github.com.
(10) undefined. https://docs.github.com/articles/about-issue-and-pull-request-templates.
(11) undefined. https://github.com/OpenDFM/MULTI-Benchmark/raw/main/README_zh.md.
(12) undefined. https://OpenDFM.github.io/MULTI-Benchmark/.
(13) undefined. https://arxiv.org/abs/2402.03173/.
(14) undefined. https://huggingface.co/datasets/OpenDFM/MULTI-Benchmark.

Introducing paper: MULTI: Multimodal Understanding Leaderboard with Text and Images
ENTIRe-ID
Full name: ENTIRe-ID

The growing importance of person re-identification in computer vision has highlighted the need for more extensive and diverse datasets. In response, we introduce the ENTIRe-ID dataset, an extensive collection comprising over 4.45 million images from 37 different cameras in varied environments. This dataset is uniquely designed to tackle the challenges of domain variability and model generalization, areas where existing datasets for person re-identification have fallen short. The ENTIRe-ID dataset stands out for its coverage of a wide array of real-world scenarios, encompassing various lighting conditions, angles of view, and diverse human activities. This design ensures a realistic and robust training platform for ReID models.

Introducing paper: ENTIRe-ID: An Extensive and Diverse Dataset for Person Re-Identification
TMD
Full name: Text-Music-Dance

The Text-Music-Dance (TMD) dataset establishes a pioneering benchmark comprising 2,153 text-music-motion pairs. Dance motions and corresponding text annotations are sourced from Motion-X, incorporating AIST++ and other datasets. For motion-text pairs lacking music, corresponding music is generated using Stable Audio Open with beat adjustment and validated through expert assessments, ensuring inter-rater reliability.

Introducing paper: Motion Anything: Any to Motion Generation
mango

Large language models such as ChatGPT and GPT-4 have recently achieved astonishing performance on a variety of natural language processing tasks. 
In this paper, we propose MANGO, a benchmark to evaluate their ability to perform text-based mapping and navigation.
Our benchmark includes $53$ mazes taken from a suite of textgames: each maze is paired with a walkthrough that visits every location but does **not** cover all possible paths. 
The task is question-answering: for each maze, a large language model reads the walkthrough and answers hundreds of mapping and navigation questions such as "How should you go to `Attic` from `West of House`?" and "Where are we if we go `north` and `east` from `Cellar`?".
Although these questions are easy for humans, it turns out that even GPT-4, the best-to-date language model, performs poorly when answering them. 
Further, our experiments suggest that a strong mapping and navigation ability would benefit the performance of large language models on relevant downstream tasks, such as playing textgames.
Our MANGO benchmark will facilitate future research on methods that improve the mapping and navigation capabilities of LLMs. 
We host our leaderboard, data, code, and evaluation program at [https://mango.ttic.edu](https://mango.ttic.edu) and [https://github.com/Oaklight/mango](https://github.com/Oaklight/mango).

Introducing paper: MANGO: A Benchmark for Evaluating Mapping and Navigation Abilities of Large Language Models
Flickr30K-Noisy
Full name: Flickr-30K with 20% of Noisy Correspondence

This dataset, based on Flickr30K, is introduced in *Learning with Noisy Correspondence for Cross-modal Matching*. Noisy correspondence is simulated by randomly shuffling the captions of training images for a specific percentage, denoted by noise ratio

Introducing paper: Learning with Noisy Correspondence for Cross-modal Matching
CC152K
Full name: Conceptual Captions 152K

CC152K is a subset of Conceptual Captions. It contains 150,000 randomly selected samples from the training split for training, 1,000 samples from the validation split for validation, and 1,000 samples from the validation split for testing.

Introducing paper: Cross-modal Retrieval with Noisy Correspondence via Consistency Refining and Mining
SimpleFood45

A collection of images of simple food items with various ground-truths

Introducing paper: Food Portion Estimation via 3D Object Scaling