Omkar2118/datasets huggingface dataset:
BBH
Full name: BIG-Bench Hard

BIG-Bench Hard (BBH) is a subset of the BIG-Bench, a diverse evaluation suite for language models. BBH focuses on a suite of 23 challenging tasks from BIG-Bench that were found to be beyond the capabilities of current language models. These tasks are ones where prior language model evaluations did not outperform the average human-rater.

The BBH tasks require multi-step reasoning, and it was found that few-shot prompting without Chain-of-Thought (CoT), as done in the BIG-Bench evaluations, substantially underestimates the best performance and capabilities of language models. When CoT prompting was applied to BBH tasks, it enabled PaLM to surpass the average human-rater performance on 10 of the 23 tasks, and Codex to surpass the average human-rater performance on 17 of the 23 tasks.

Introducing paper: Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them
Shot2Story20K

A short clip of video may contain progression of multiple events and an interesting story line. A human needs to capture both the event in every shot and associate them together to understand the story behind it.

In this work, we present a new multi-shot video understanding benchmark Shot2Story with detailed shot-level captions and comprehensive video summaries. To facilitate better semantic understanding of videos, we provide captions for both visual signals and human narrations. We design several distinct tasks including single-shot video and narration captioning, multi-shot video summarization, and video retrieval with shot descriptions.

Preliminary experiments show some challenges to generate a long and comprehensive video summary. Nevertheless, the generated imperfect summaries can already significantly boost the performance of existing video understanding tasks such as video question-answering, promoting an underexplored setting of video understanding with detailed summaries.

Introducing paper: Shot2Story20K: A New Benchmark for Comprehensive Understanding of Multi-shot Videos
SQL injection dataset

This is a cleaned version of the dataset introduced in Kaggle by user SAJID576. The original dataset's link is https://www.kaggle.com/datasets/sajid576/sql-injection-dataset
Q-Bench

The Q-Bench includes three realms for low-level vision: perception (A1), description (A2), and assessment (A3). 
- For perception (A1) /description (A2), we collect two benchmark datasets LLVisionQA/LLDescribe.
- For assessment (A3), as we use public datasets, we provide an abstract evaluation code for arbitrary MLLMs for anyone to test.

Introducing paper: Q-Bench: A Benchmark for General-Purpose Foundation Models on Low-level Vision
SPAQ
Full name: Smartphone Photography Attribute and Quality

The Smartphone Photography Attribute and Quality (SPAQ) dataset is a comprehensive database for the perceptual quality assessment of smartphone photography. It was introduced in a paper titled "Perceptual Quality Assessment of Smartphone Photography" presented at the IEEE Conference on Computer Vision and Pattern Recognition in 2020.

The SPAQ dataset consists of **11,125 pictures** taken by **66 smartphones**. Each image in the dataset is attached with rich annotations, including:
- Image quality
- Image attributes (brightness, colorfulness, contrast, noisiness, and sharpness)
- Scene category labels (animal, cityscape, human, indoor scene, landscape, night scene, plant, still life, and others)
- Exchangeable image file format (EXIF) data

These annotations were collected in a well-controlled laboratory environment. The dataset has been used to train blind image quality assessment (BIQA) models, providing insights into how EXIF data, image attributes, and high-level semantics interact with image quality. The SPAQ dataset and the proposed BIQA models are available on GitHub.

Introducing paper: Perceptual Quality Assessment of Smartphone Photography
LIVE-itw
Full name: LIVE In the Wild Image Quality Challenge Database

Image quality assessment (IQA) databases enable researchers to evaluate the performance of IQA algorithms and contribute towards attaining the ultimate goal of objective quality assessment research - matching human perception. Most publicly available image quality databases have been created under highly controlled conditions by introducing graded simulated distortions onto high-quality photographs. However, images captured using typical real-world mobile camera devices are usually afflicted by complex mixtures of multiple distortions, which are not necessarily well-modeled by the synthetic distortions found in existing databases. Our newly designed and created LIVE In the Wild Image Quality Challenge Database, contains widely diverse authentic image distortions on a large number of images captured using a representative variety of modern mobile devices. We also designed and implemented a new online crowdsourcing system, which we have used to conduct a very large-scale, multi-month image quality assessment subjective study. The LIVE In the Wild Image Quality Database has over 350,000 opinion scores on 1,162 images evaluated by over 8100 unique human observers.

Introducing paper: Massive Online Crowdsourced Study of Subjective and Objective Picture Quality
CGIQA-6K
Full name: Computer Graphics Image Quality Assessment

CGIQA-6k database is a large-scale, in-the-wild CGIQA database consisting of 6,000 CGIs. The CGIQA-6k dataset is a large-scale, in-the-wild database for Computer Graphics Image Quality Assessment (CGIQA). It consists of 6,000 Computer Graphics Images (CGIs). These CGIs are artificially generated visuals created using computer programs and are prevalent across various platforms, from video games to streaming media.

Introducing paper: Subjective and Objective Quality Assessment for in-the-Wild Computer Graphics Images
AGIQA-3K

The AGIQA-3K is a fine-grained AI-generated image (AGI) subjective quality assessment database. It was created to address the need for quality models that are consistent with human subjective ratings, considering the large quality variance among different AGIs. The database extensively considers various popular AGI models, generates AGI through different prompts and model parameters, and collects subjective scores at the perceptual quality and text-to-image alignment level.

Introducing paper: AGIQA-3K: An Open Database for AI-Generated Image Quality Assessment
GOD
Full name: Generic Object Decoding

The Generic Object Decoding (GOD) Dataset is a specialized resource developed for fMRI-based decoding. It aggregates fMRI data gathered through the presentation of images from 200 representative object categories, originating from the 2011 fall release of ImageNet. The training session incorporated 1,200 images (8 per category from 150 distinct object categories). In contrast, the test session included 50 images (one from each of the 50 object categories). It is noteworthy that the categories in the test session were unique from those in the training session and were introduced in a randomized sequence across runs. On five subjects the fMRI scanning was conducted.
IMPACT

The IMPACT dataset contains 50 human created prompts for each category, 200 in total, to test LLMs general writing ability. Instructed LLMs demonstrate promising ability in writing-based tasks, such as composing letters or ethical debates. This dataset consists prompts across 4 diverse usage scenarios:
- Informative Writing: User queries such as self-help advice or explanations for various concept
- Professional Writing: Format involves suggestions presentations or emails in a business setting
- Argumentative Writing: Debate positions on ethical and societal question
- Creative Writing: Diverse writing formats such as stories, poems, and songs.

Introducing paper: INSTRUCTEVAL: Towards Holistic Evaluation of Instruction-Tuned Large Language Models
HHH
Full name: Helpful, Honest, & Harmless

The HHH dataset, also known as the Helpful, Honest, & Harmless (HHH) Alignment dataset, is a dataset used for evaluating language models. It is pragmatically broken down into the categories of helpfulness, honesty/accuracy, and harmlessness. The dataset is formatted in terms of binary comparisons, often broken down from a ranked ordering of three or four possible responses to a given query or context. The goal of these evaluations is that on careful reflection, the vast majority of people would agree that the chosen response is better (more helpful, honest, and harmless) than the alternative offered for comparison.

Introducing paper: A General Language Assistant as a Laboratory for Alignment
CCGR
Full name: Cross-Covariate Gait Recognition

CCGR (Cross-Covariate Gait Recognition), the first gait dataset for studying cross-covariate challenges, contains 970 subjects, approximately 1.6 million sequences, 53 covariates, and 33 views.
CRACK500

For the details of the work, the readers are refer to the paper "Feature Pyramid and Hierarchical Boosting Network for Pavement Crack Detection" (FPHB), T-ITS 2019. You can find the paper in https://www.researchgate.net/publication/330244656_Feature_Pyramid_and_Hierarchical_Boosting_Network_for_Pavement_Crack_Detection or https://arxiv.org/abs/1901.06340.

The pavement crack datasets used in paper, crack detection results on each datasets, trained model, and crack annotation tool are stored in Google Drive, One Drive, and Daidu Yunpan extract code: jviq.

Introducing paper: Feature Pyramid and Hierarchical Boosting Network for Pavement Crack Detection
Casia V1+

Casia V1 is a dataset for forgery classification. Casia V1+ is a modification of the Casia V1 dataset proposed by Chen et al. that replaces authentic images that also exist in Casiav2 with images from the COREL dataset to avoid data contamination.

Introducing paper: Image Manipulation Detection by Multi-View Multi-Scale Supervision