Omkar2118/datasets huggingface dataset:
PhysioNet Challenge 2016

Introduction
The 2016 PhysioNet/CinC Challenge aims to encourage the development of algorithms to classify heart sound recordings collected from a variety of clinical or nonclinical (such as in-home visits) environments. The aim is to identify, from a single short recording (10-60s) from a single precordial location, whether the subject of the recording should be referred on for an expert diagnosis.

During the cardiac cycle, the heart firstly generates the electrical activity and then the electrical activity causes atrial and ventricular contractions. This in turn forces blood between the chambers of the heart and around the body. The opening and closure of the heart valves is associated with accelerations-decelerations of blood, giving rise to vibrations of the entire cardiac structure (the heart sounds and murmurs) [1]. These vibrations are audible at the chest wall, and listening for specific heart sounds can give an indication of the health of the heart. The phonocardiogram (PCG) is the graphical representation of a heart sound recording. Figure 1 illustrates a short section of a PCG recording.
IXI
Full name: IXI Brain Development Dataset

**IXI Dataset** is a collection of 600 MR brain images from normal, healthy subjects. The MR image acquisition protocol for each subject includes:

 * T1, T2 and PD-weighted images
 * MRA images
 * Diffusion-weighted images (15 directions)

The data has been collected at three different hospitals in London:

 * Hammersmith Hospital using a Philips 3T system (details of scanner parameters)
 * Guy’s Hospital using a Philips 1.5T system (details of scanner parameters)
 * Institute of Psychiatry using a GE 1.5T system (details of the scan parameters not available at the moment)

The data has been collected as part of the project:

 * IXI – Information eXtraction from Images (EPSRC GR/S21533/02)

The images in NIFTI format can be downloaded from [here](https://brain-development.org/ixi-dataset/):

This data is made available under the Creative Commons CC BY-SA 3.0 license. If you use the IXI data please acknowledge the source of the IXI data.
LIFULL HOME'S

The National Institute of Informatics provides LIFULL HOME'S Dataset to researchers, which was offered by [LIFULL Co., Ltd.](https://lifull.com/en/) for promoting research in informatics and the related fields.

The dataset contains the data of [LIFULL HOME'S](https://www.homes.co.jp/), a Real Estate Information Service in Japan.

1. Snapshot Data of Rentals (snapshot of 2015-09)
Rental data (5.33 million all over Japan): rental fee, area, location, age, floor plan, structure, facilities, etc.; approx. 1.6GB .tsv format files.
Image data (83 million files): floor plan image, room view, etc. of all the above items; approx. 210GB .jpg format files, max size: 120x120.

2. High Resolution Floor Plan Image Data
High resolution version data of floor plan image (5.31 million files) included in Snapshot Data of Rentals; approx. 140GB .jpg format files. Additional application required to use this data (see Application section below).

3. Monthly Data of Rentals and Sales (2015-07 - 2017-06, 24 months)
Property data of rentals and sales (5.33 million all over Japan): rental fee/price, area, location, age, floor plan, structure, facilities, etc.; approx. 1.7 - 4.5GB .tsv format files for respective months.
In addition, LIFULL Co., Ltd. provides a sample script for classifying image types via Github:
https://github.com/Littel-Laboratory/homes-dataset-tools
Sketch2aia (Mobile User Interface Sketches)

Dataset of 374 photos of hand-drawn sketches of App Inventor apps used for development of the Sketch2aia model for automatic generation of App Inventor wireframes from hand-drawn sketches.

Data format
Training:2 37 images in JPG (.jpg) format with 720×1280 pixels, each accompanied by a JSON (.json) file with manually attributed bounding box annotation for 10 different classes of UI elements (Screen, Label, Button, Switch, Slider, TextBox, CheckBox, ListPicker, Image and Map), used to train the Sketch2aia model.

Validation: 42 images in JPG (.jpg) format with 720×1280 pixels, each accompanied by a JSON (.json) file with manually attributed bounding box annotation for 10 different classes of UI elements (Screen, Label, Button, Switch, Slider, TextBox, CheckBox, ListPicker, Image and Map), used to test the Sketch2aia model.

Additional Images: 95 images in JPG (.jpg) format with 720×1280 pixels. Some images are accompanied by a JSON (.json) file with manually attributed bounding box annotation for 10 different classes of UI elements (Screen, Label, Button, Switch, Slider, TextBox, CheckBox, ListPicker, Image and Map), while others have not yet been labeled. This portion of the dataset was collected during user evaluation of the Sketch2aia model, and have not been directly used to train or test the object detection model.

Introducing paper: Automatic code generation from sketches of mobile applications in end-user development using Deep Learning
An Amharic News Text classification Dataset

In NLP, text classification is one of the primary problems we try to solve and its uses in language analyses are indisputable. The lack of labeled training data made it harder to do these tasks in low resource languages like Amharic. The task of collecting, labeling, annotating, and making valuable this kind of data will encourage junior researchers, schools, and machine learning practitioners to implement existing classification models in their language. In this short paper, we aim to introduce the Amharic text classification dataset that consists of more than 50k news articles that were categorized into 6 classes. This dataset is made available with easy baseline performances to encourage studies and better performance experiments.

Introducing paper: An Amharic News Text classification Dataset
PHOENIX14T
Full name: RWTH-PHOENIX-Weather-2014T

Over a period of three years (2009 - 2011) the daily news and weather forecast airings of the German public tv-station PHOENIX featuring sign language interpretation have been recorded and the weather forecasts of a subset of 386 editions have been transcribed using gloss notation. Furthermore, we used automatic speech recognition with manual cleaning to transcribe the original German speech. As such, this corpus allows to train end-to-end sign language translation systems from sign language video input to spoken language.

The signing is recorded by a stationary color camera placed in front of the sign language interpreters. Interpreters wear dark clothes in front of an artificial grey background with color transition. All recorded videos are at 25 frames per second and the size of the frames is 210 by 260 pixels. Each frame shows the interpreter box only.

Introducing paper: Neural Sign Language Translation
CUAD
Full name: Contract Understanding Atticus Dataset

**Contract Understanding Atticus Dataset** (**CUAD**) is a dataset for legal contract review. CUAD was created with dozens of legal experts from The Atticus Project
and consists of over 13,000 annotations. The task is to highlight salient portions of a contract that are important for a human to review.

Introducing paper: CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review
BIKED
Full name: BIKED

**BIKED** is a dataset comprised of 4500 individually designed bicycle models sourced from hundreds of designers. BIKED enables a variety of data-driven design applications for bicycles and generally supports the development of data-driven design methods. The dataset is comprised of a variety of design information including assembly images, component images, numerical design parameters, and class labels.

Introducing paper: BIKED: A Dataset for Computational Bicycle Design with Machine Learning Benchmarks
PCD
Full name: Poem Comprehensive Dataset

The Arabic dataset is scraped mainly from الموسوعة الشعرية and الديوان. After merging both, the total number of verses is 1,831,770 poetic verses. Each verse is labeled by its meter, the poet who wrote it, and the age which it was written in. There are 22 meters, 3701 poets and 11 ages: Pre-Islamic, Islamic, Umayyad, Mamluk, Abbasid, Ayyubid, Ottoman, Andalusian, era between Umayyad and Abbasid, Fatimid, and finally the modern age. We are only interested in the 16 classic meters which are attributed to Al-Farahidi, and they comprise the majority of the dataset with a total number around 1.7M verses. It is important to note that the verses diacritic states are not consistent. This means that a verse can carry full, semi diacritics, or it can carry nothing.

Introducing paper: Learning meters of Arabic and English poems with Recurrent Neural Networks: a step forward for language understanding and synthesis
ARCH

**ARCH** is a computational pathology (CP) multiple instance captioning dataset to facilitate dense supervision of CP tasks. Existing CP datasets focus on narrow tasks; ARCH on the other hand contains dense diagnostic and morphological descriptions for a range of stains, tissue types and pathologies. 

Source: [Gamper et al.](https://arxiv.org/pdf/2103.05121v1.pdf)

Image source: [Gamper et al.](https://arxiv.org/pdf/2103.05121v1.pdf)

Introducing paper: Multiple Instance Captioning: Learning Representations from Histopathology Textbooks and Articles
UASOL
Full name: A large-scale high-resolution outdoor stereo dataset

The UASOL an RGB-D stereo dataset, that contains 160902 frames, filmed at 33 different scenes, each with between 2 k and 10 k frames. The frames show different paths from the perspective of a pedestrian, including sidewalks, trails, roads, etc. The images were extracted from video files with 15 fps at HD2K resolution with a size of 2280 × 1282 pixels. The dataset also provides a GPS geolocalization tag for each second of the sequences and reflects different climatological conditions. It also involved up to 4 different persons filming the dataset at different moments of the day.

We propose a [train, validation and test split](https://www.nature.com/articles/s41597-019-0168-5/tables/4) to train the network. 
Additionally, we introduce a subset of [676 pairs of RGB Stereo images and their respective depth](https://osf.io/64532/files/), which we extracted randomly from the entire dataset. This given test set is introduced to make comparability possible between the different methods trained with the dataset.

Introducing paper: UASOL, a large-scale high-resolution outdoor stereo dataset
SUM

SUM is a new benchmark dataset of semantic urban meshes which covers about 4 km2 in Helsinki (Finland), with six classes: Ground, Vegetation, Building, Water, Vehicle, and Boat.

The authors used Helsinki 3D textured meshes as input and annotated them as a benchmark dataset of semantic urban meshes. The Helsinki's raw dataset covers about 12 km2 and was generated in 2017 from oblique aerial images that have about a 7.5 cm ground sampling distance (GSD) using an off-the-shelf commercial software namely ContextCapture.

The entire region of Helsinki is split into tiles, and each of them covers about 250 m2. 

Image source: [Gao et al.](https://arxiv.org/pdf/2103.00355v1.pdf)

Introducing paper: SUM: A Benchmark Dataset of Semantic Urban Meshes
BLURB
Full name: Biomedical Language Understanding and Reasoning Benchmark

**BLURB** is a collection of resources for biomedical natural language processing. In general domains such as newswire and the Web, comprehensive benchmarks and leaderboards such as GLUE have greatly accelerated progress in open-domain NLP. In biomedicine, however, such resources are ostensibly scarce. In the past, there have been a plethora of shared tasks in biomedical NLP, such as BioCreative, BioNLP Shared Tasks, SemEval, and BioASQ, to name just a few. These efforts have played a significant role in fueling interest and progress by the research community, but they typically focus on individual tasks. The advent of neural language models such as BERTs provides a unifying foundation to leverage transfer learning from unlabeled text to support a wide range of NLP applications. To accelerate progress in biomedical pretraining strategies and task-specific methods, it is thus imperative to create a broad-coverage benchmark encompassing diverse biomedical tasks.

Inspired by prior efforts toward this direction (e.g., BLUE), BLURB (short for Biomedical Language Understanding and Reasoning Benchmark) was created. BLURB comprises of a comprehensive benchmark for PubMed-based biomedical NLP applications, as well as a leaderboard for tracking progress by the community. BLURB includes thirteen publicly available datasets in six diverse tasks. To avoid placing undue emphasis on tasks with many available datasets, such as named entity recognition (NER), BLURB reports the macro average across all tasks as the main score. The BLURB leaderboard is model-agnostic. Any system capable of producing the test predictions using the same training and development data can participate. The main goal of BLURB is to lower the entry barrier in biomedical NLP and help accelerate progress in this vitally important field for positive societal and human impact.

Source: [BLURB](https://microsoft.github.io/BLURB/index.html)

Image source: [BLURB](https://microsoft.github.io/BLURB/index.html)

Introducing paper: Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing
GAD
Full name: Gene Associations Database

**GAD**, or **Gene Associations Database**, is a corpus of gene-disease associations curated from genetic association studies.
BC2GM
Full name: BC2GM

Created by Smith et al. at 2008, the BioCreative II Gene Mention Recognition (BC2GM) Dataset contains data where participants are asked to identify a gene mention in a sentence by giving its start and end characters. The training set consists of a set of sentences, and for each sentence a set of gene mentions (GENE annotations). [registration required for access], in English language. Containing 20 in n/a file format.
Kaggle EyePACS
Full name: Kaggle EyePACS. Diabetic Retinopathy Detection Identify signs of diabetic retinopathy in eye images

Diabetic retinopathy is the leading cause of blindness in the working-age population of the developed world. It is estimated to affect over 93 million people.

retina

The US Center for Disease Control and Prevention estimates that 29.1 million people in the US have diabetes and the World Health Organization estimates that 347 million people have the disease worldwide. Diabetic Retinopathy (DR) is an eye disease associated with long-standing diabetes. Around 40% to 45% of Americans with diabetes have some stage of the disease. Progression to vision impairment can be slowed or averted if DR is detected in time, however this can be difficult as the disease often shows few symptoms until it is too late to provide effective treatment.

Currently, detecting DR is a time-consuming and manual process that requires a trained clinician to examine and evaluate digital color fundus photographs of the retina. By the time human readers submit their reviews, often a day or two later, the delayed results lead to lost follow up, miscommunication, and delayed treatment.

Clinicians can identify DR by the presence of lesions associated with the vascular abnormalities caused by the disease. While this approach is effective, its resource demands are high. The expertise and equipment required are often lacking in areas where the rate of diabetes in local populations is high and DR detection is most needed. As the number of individuals with diabetes continues to grow, the infrastructure needed to prevent blindness due to DR will become even more insufficient.

The need for a comprehensive and automated method of DR screening has long been recognized, and previous efforts have made good progress using image classification, pattern recognition, and machine learning. With color fundus photography as input, the goal of this competition is to push an automated detection system to the limit of what is possible – ideally resulting in models with realistic clinical potential. The winning models will be open sourced to maximize the impact such a model can have on improving DR detection.

Acknowledgements
This competition is sponsored by the California Healthcare Foundation.

![img](https://storage.googleapis.com/kaggle-competitions/kaggle/4104/media/chcf.gif)

Retinal images were provided by EyePACS, a free platform for retinopathy screening.

![img](https://storage.googleapis.com/kaggle-competitions/kaggle/4104/media/eyepacs.png)
THFOOD-50
Full name: Thai Food 50 Image Classification

Fine-Grained Thai Food Image Classification Datasets

THFOOD-50 containing 15,770 images of 50 famous Thai dishes.

Introducing paper: NU-InNet: Thai Food Image Recognition Using Convolutional Neural Networks on Smartphone
SRD
Full name: Shadow Removal Dataset

SRD is a dataset for shadow removal that contains 3088 shadow and shadow-free image pairs.

Introducing paper: DeshadowNet: A Multi-Context Embedding Deep Network for Shadow Removal
BL30K

BL30K is a synthetic dataset rendered using Blender with ShapeNet's data. We break the dataset into six segments, each with approximately 5K videos. The videos are organized in a similar format as DAVIS and YouTubeVOS, so dataloaders for those datasets can be used directly. Each video is 160 frames long, and each frame has a resolution of 768*512. There are 3-5 objects per video, and each object has a random smooth trajectory -- we tried to optimize the trajectories in a greedy fashion to minimize object intersection (not guaranteed), with occlusions still possible (happen a lot in reality). See [MiVOS](https://github.com/hkchengrex/MiVOS) for details.

Introducing paper: Modular Interactive Video Object Segmentation: Interaction-to-Mask, Propagation and Difference-Aware Fusion
BIG

A high-resolution semantic segmentation dataset with 50 validation and 100 test objects. Image resolution in BIG ranges from 2048×1600 to 5000×3600. Every image in the dataset has been carefully labeled by a professional while keeping the same guidelines as PASCAL VOC 2012 without the void region.

Introducing paper: CascadePSP: Toward Class-Agnostic and Very High-Resolution Segmentation via Global and Local Refinement
COCO Object Detection VIPriors subset

The training and validation data are subsets of the training split of the MS COCO dataset (2017 release, bounding boxes only). The test set is taken from the validation split of the MS COCO dataset.

Introducing paper: VIPriors 1: Visual Inductive Priors for Data-Efficient Deep Learning Challenges
Cityscapes VIPriors subset

The training and validation data are subsets of the training split of the Cityscapes dataset. The test set is taken from the validation split of the Cityscapes dataset.

Introducing paper: VIPriors 1: Visual Inductive Priors for Data-Efficient Deep Learning Challenges
UCF-101 VIPriors subset

The VIriors Action Recognition Challenge uses a subset of the UCF101 action recognition dataset:

Train set: ~4.8K clips.
Validation set: ~4.7K clips.
Test set: ~3.8K clips.

Introducing paper: VIPriors 1: Visual Inductive Priors for Data-Efficient Deep Learning Challenges
CLEVR-Hans

The CLEVR-Hans data set is a novel confounded visual scene data set, which captures complex compositions of different objects. This data set consists of [CLEVR](clevr) images divided into several classes. 

The membership of a class is based on combinations of objects’ attributes and relations. Additionally, certain classes within the data set are confounded. Thus, within the data set, consisting of train, validation, and test splits, all train, and validation images of confounded classes will be confounded with a specific attribute or combination of attributes.

Each class is represented by 3000 training images, 750 validation images, and 750 test images. The training, validation, and test set splits contain 9000, 2250, and 2250 samples, respectively, for CLEVR-Hans3 and 21000, 5250, and 5250 samples for CLEVR-Hans7. The class distribution is balanced for all data splits.

For CLEVR-Hans classes for which class rules contain more than three objects, the number of objects to be placed per scene was randomly chosen between the minimal required number of objects for that class and ten, rather than between three and ten, as in the original CLEVR data set.

Finally, the images were created such that the exact combinations of the class rules did not occur in images of other classes. It is possible that a subset of objects from one class rule occur in an image of another class. However, it is not possible that more than one complete class rule is contained in an image.

Source: [CLEVR-Hans](https://github.com/ml-research/CLEVR-Hans)
Image Source: [CLEVR-Hans](https://github.com/ml-research/CLEVR-Hans)

Introducing paper: Right for the Right Concept: Revising Neuro-Symbolic Concepts by Interacting with their Explanations
Tsinghua Dogs

Tsinghua Dogs is a fine-grained classification dataset for dogs, over 65% of whose images are collected from people's real life. Each dog breed in the dataset contains at least 200 images and a maximum of 7,449 images, basically in proportion to their frequency of occurrence in China, so it significantly increases the diversity for each breed over existing dataset. Furthermore, Tsinghua Dogs annotated bounding boxes of the dog’s whole body and head in each image, which can be used for supervising the training of learning algorithms as well as testing them.

Introducing paper: A new dataset of dog breed images and a benchmark for fine-grained classification
L1000

The **L1000** dataset consists of ~1,400,000 gene-expression profiles on the responses of ~50 human cell lines to one of ~20,000 compounds across a range of concentrations. The L1000 dataset and its normalization versions10 were recently widely used in drug repurposing and discovery.

Description from: [A deep learning framework for high-throughput mechanism-driven phenotype compound screening and its application to COVID-19 drug repurposing](https://www.nature.com/articles/s42256-020-00285-9)

Publication introducing the dataset: [A Next Generation Connectivity Map: L1000 Platform and the First 1,000,000 Profiles](https://pubmed.ncbi.nlm.nih.gov/29195078/)
DSBEC
Full name: Dark solitons in BECs dataset

The data set consists of 6257 labeled images of Bose-Einstein condensates (BECs) with and without solitonic excitations, including kink solitons and solitonic vortices. Each element of the data set contains a masked image (132x164 pixels) of 2D atomic density used to train the machine learning model used in the paper "Machine-learning enhanced dark soliton detection in Bose-Einstein condensates," (https://arxiv.org/abs/2101.05404), and a label indicating the class a given image belongs to (0 indicates no solitons, 1 indicates a single soliton, and 2 indicates other excitations). The data structure file and project description are included with the data.
This data set was used to train a deep convolutional neural network to automatically recognize whether or not a lone dark soliton has been created in BECs that was then implemented within an automated soliton detection and positioning system (see https://arxiv.org/abs/2101.05404 for details).

Introducing paper: Machine-learning enhanced dark soliton detection in Bose-Einstein condensates
Gowalla
Full name: Gowalla

Gowalla is a location-based social networking website where users share their locations by checking-in. The friendship network is undirected and was collected using their public API, and consists of 196,591 nodes and 950,327 edges. We have collected a total of 6,442,890 check-ins of these users over the period of Feb. 2009 - Oct. 2010.

Introducing paper: Friendship and Mobility: User Movement In Location-Based Social Networks