Omkar2118/datasets huggingface dataset:
MusicQA

We propose the MusicQA dataset to train Music-enabled question-answering models and is used for training and evaluating our MU-LLaMA model. This dataset is generated using the MusicCaps and MagnaTagATune datasets. We utilize the descriptions/tags from existing datasets to prompt the MPT-7B Chat model to generate question-answer pairs through inference, reasoning, and paraphrasing. The dataset contains 12,542 music files for training making up 76.15 hours of music with 112,878 question-answer pairs.

Introducing paper: Music Understanding LLaMA: Advancing Text-to-Music Generation with Question Answering and Captioning
DAIR-V2X-Seq

An extension of DAIR-V2X with addition temporal information

Introducing paper: V2X-Seq: A Large-Scale Sequential Dataset for Vehicle-Infrastructure Cooperative Perception and Forecasting
MMVP

The **MMVP (Multimodal Visual Patterns) Benchmark** focuses on identifying "CLIP-blind pairs" – images that appear similar to the CLIP model despite having clear visual differences. These patterns highlight the challenges these systems face in answering straightforward questions, often leading to incorrect responses and hallucinated explanations.

Introducing paper: Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs
NERDS 360
Full name: NeRF for Reconstruction, Decomposition and Scene Synthesis of 360° outdoor scenes

We present a large-scale dataset for 3D urban scene understanding. Compared to existing datasets, our dataset consists of 75 outdoor urban scenes with diverse backgrounds, encompassing over 15,000 images. These scenes offer 360◦ hemispherical views, capturing diverse foreground objects illuminated under various lighting conditions. Additionally, our dataset encompasses scenes that are not limited to forward-driving views, addressing the limitations of previous datasets such as limited overlap and coverage between camera
views. The closest pre-existing dataset for generalizable evaluation is DTU [2] (80 scenes) which comprises mostly indoor objects and does not provide multiple foreground objects or background scenes.

We use the Parallel Domain synthetic data generation to render high-fidelity 360◦ scenes. We select 3 different maps i.e. SF 6thAndMission, SF GrantAndCalifornia and SF VanNessAveAndTurkSt and sample 75 different scenes across all 3 maps as our backgrounds (All 75 scenes across 3 maps are significantly different road scenes from each other, captured at different viewpoints in the city). We select 20 different cars in 50 different textures for training and randomly sample from 1 to 4 cars to render in a scene. We refer to this dataset as NeRDS 360: NeRF for Reconstruction, Decomposition and Scene Synthesis of 360◦ outdoor scenes. In total, we generate 15k renderings by sampling 200 cameras in a 
 hemispherical dome at a fixed distance from the center of cars. We held out 5 scenes with 4 different cars and different backgrounds for testing, comprising 100 cameras distributed uniformly sampled in the upper hemisphere, different from the camera distributions used for training. We use the diverse validation camera distribution to test our approach’s ability to generalize to unseen viewpoints as well as unseen scenes during training. Our dataset and the corresponding task is extremely challenging due to occlusions, diversity of backgrounds, and rendered objects with various lightning and shadows. 

Our task entails reconstructing 360◦ hemispherical views of complete scenes using a handful of observations i.e. 1 to 5 as shown by red cameras whereas evaluating using all 100 hemispherical views, hence our task requires strong priors for novel view synthesis of outdoor scenes. 

Tasks our dataset support:

- Generaliazable Novel view synthesis (Few shot evaluation)
- Novel view synthesis (Overfitting evaluation)
- 6D pose estimation
- Object editing 
- Depth estimation
- Semantic Segmentation
 - Instance Segmentation

Introducing paper: NeO 360: Neural Fields for Sparse View Synthesis of Outdoor Scenes