Omkar2118/datasets huggingface dataset:
M2KR
Full name: Multi-task Multi-modal Knowledge Retrieval

The **M2KR** is a collection of datasets designed for training and evaluating general-purpose vision-language retrievers. These datasets are released in **Huggingface Dataset format** and cover various retrieval tasks. Let's delve into the details:

1. **Image to Text (I2T) retrieval**: This task involves retrieving relevant textual descriptions given an input image.
2. **Question to Text (Q2T) retrieval**: Here, the goal is to retrieve relevant text passages based on a given question.
3. **Image & Question to Text (IQ2T) retrieval**: This task combines both image and question inputs to retrieve relevant textual information.

The M2KR benchmark comprises nine datasets, each tailored for specific tasks. Some of these datasets include:

- **WIT (Web Image Text)**: A dataset for I2T retrieval.
- **IGLUE (Image-Grounded Language Understanding Evaluation)**: Used for Q2T retrieval.
- **KVQA (Knowledge Visual Question Answering)**: Relevant for IQ2T retrieval.
- **CC3M (Common Crawl 3 Million)**: Another dataset for IQ2T retrieval.
- **OVEN (Open Vision and Language Evaluation)**: Used in IQ2T retrieval.
- **LLaVA (Large-scale Language-Visual Association)**: Relevant for I2T retrieval.
- **OKVQA (Open Knowledge Visual Question Answering)**: Used in IQ2T retrieval.
- **Infoseek**: A dataset for I2T retrieval.
- **E-VQA (English Visual Question Answering)**: Relevant for IQ2T retrieval.

These datasets enable researchers to develop and evaluate vision-language models, and they play a crucial role in advancing the field of multimodal understanding and retrieval¹².

(1) M2KR Benchmark Datasets - GitHub. https://github.com/LinWeizheDragon/FLMR/blob/main/docs/Datasets.md.
(2) arXiv:2402.08327v1 [cs.CL] 13 Feb 2024. https://arxiv.org/pdf/2402.08327.pdf.
(3) Scaling Up Fine-Grained Late-Interaction Multi-modal Retrievers. https://preflmr.github.io/.
(4) undefined. https://avatars.githubusercontent.com/u/33350454?v=4.
(5) undefined. https://github.com/LinWeizheDragon/FLMR/blob/main/docs/Datasets.md?raw=true.
(6) undefined. https://desktop.github.com.
(7) undefined. https://github.com/LinWeizheDragon/FLMR/raw/main/docs/Datasets.md.

Introducing paper: PreFLMR: Scaling Up Fine-Grained Late-Interaction Multi-modal Retrievers
RGBT234

The RGBT234 dataset is a comprehensive video dataset specifically designed for RGB-T (Red-Green-Blue and Thermal) tracking purposes. This dataset addresses the limitations of existing datasets like OSU-CT, LITIV, and GTOT in terms of size. RGBT234 consists of 234 RGB-T videos, each containing both an RGB video and a thermal video. The total number of frames in the dataset is approximately 234,000, with the largest video pair containing up to 8,000 frames.Each frame in the RGBT234 dataset is annotated with a minimum bounding box that covers the target for both the RGB and thermal modalities. The dataset also includes various environmental challenges such as rainy conditions, nighttime scenes, cold and hot weather scenarios. To analyze the performance of different tracking algorithms based on specific attributes, the RGBT234 dataset annotates 12 attributes and provides baseline trackers, including both deep learning and non-deep learning methods like structured SVM, sparse representation, and correlation filter-based trackers. Additionally, the dataset employs 5 metrics to evaluate the performance of RGB-T trackers effectively.

Introducing paper: RGB-T Object Tracking:Benchmark and Baseline
PRONTO
Full name: PRONTO heterogeneous benchmark dataset

The PRONTO heterogeneous benchmark dataset is based on an industrial-scale multiphase flow facility. It includes data from heterogeneous sources, including process measurements, alarm records, high frequency ultrasonic flow and pressure measurements, an operation log and video recordings. The study collected data from various operational conditions with and without induced faults to generate a multi-rate, multi-modal dataset. The dataset is suitable for developing and validating algorithms for fault detection and diagnosis (FDD) and data fusion.