Omkar2118/datasets huggingface dataset:
SceneNet RGB-D
Full name: SceneNet RGB-D

SceneNet-RGBD is a synthetic dataset containing large-scale photorealistic renderings of indoor scene trajectories with pixel-level annotations. Random sampling permits virtually unlimited scene configurations, and the dataset creators provide a set of 5M rendered RGB-D images from over 15K trajectories in synthetic layouts with random but physically simulated object poses. Each layout also has random lighting, camera trajectories, and textures. The scale of this dataset is well suited for pre-training data-driven computer vision techniques from scratch with RGB-D inputs, which previously has been limited by relatively small labelled datasets in NYUv2 and SUN RGB-D. It also provides a basis for investigating 3D scene labelling tasks by providing perfect camera poses and depth data as proxy for a SLAM system.

Source: [ViewAL: Active Learning with Viewpoint Entropy for Semantic Segmentation](https://arxiv.org/abs/1911.11789)
Image Source: [https://robotvault.bitbucket.io/scenenet-rgbd.html](https://robotvault.bitbucket.io/scenenet-rgbd.html)
BMS-26
Full name: Berkeley Motion Segmentation

The  **Berkeley Motion Segmentation** Dataset (**BMS-26**) is a dataset for motion segmentation, which consists of 26 video sequences with pixel-accurate segmentation annotation of moving objects. A total of 189 frames is annotated. 12 of the sequences are taken from the Hopkins 155 dataset and new annotation is added.

Source: [https://lmb.informatik.uni-freiburg.de/resources/datasets/moseg.en.html](https://lmb.informatik.uni-freiburg.de/resources/datasets/moseg.en.html)
Image Source: [https://lmb.informatik.uni-freiburg.de/resources/datasets/moseg.en.html](https://lmb.informatik.uni-freiburg.de/resources/datasets/moseg.en.html)
Freiburg Groceries
Full name: Freiburg Groceries

**Freiburg Groceries** is a groceries classification dataset consisting of 5000 images of size 256x256, divided into 25 categories. It has imbalanced class sizes ranging from 97 to 370 images per class. Images were taken in various aspect ratios and padded to squares.

Source: [XNAS: Neural Architecture Search with Expert Advice](https://arxiv.org/abs/1906.08031)
Image Source: [http://aisdatasets.informatik.uni-freiburg.de/freiburg_groceries_dataset/](http://aisdatasets.informatik.uni-freiburg.de/freiburg_groceries_dataset/)

Introducing paper: The Freiburg Groceries Dataset
Freiburg Spatial Relations
Full name: Freiburg Spatial Relations

The **Freiburg Spatial Relations** dataset features 546 scenes each containing two out of 25 household objects. The depicted spatial relations can roughly be described as on top, on top on the corner, inside, inside and inclined, next to, and inclined. The dataset contains the 25 object models as textured .obj and .dae files, a low resolution .dae version for visualization in rviz, a scene description file containing the translation and rotation of the objects for each scene, a file with labels for each scene, the 15 splits used for cross validation, and a bash script to convert the models to pointclouds.

Source: [http://spatialrelations.cs.uni-freiburg.de/](http://spatialrelations.cs.uni-freiburg.de/)
Image Source: [http://spatialrelations.cs.uni-freiburg.de/](http://spatialrelations.cs.uni-freiburg.de/)

Introducing paper: Metric Learning for Generalizing Spatial Relations to New Objects
Freiburg Street Crossing
Full name: Freiburg Street Crossing

The **Freiburg Street Crossing** dataset consists of data collected from three different street crossings in Freiburg, Germany; ; two of which were traffic light regulated intersections and one a zebra crossing without traffic lights. The data can be used to train agents to cross roads autonomously.

Source: [http://aisdatasets.informatik.uni-freiburg.de/streetcrossing/](http://aisdatasets.informatik.uni-freiburg.de/streetcrossing/)
Image Source: [http://aisdatasets.informatik.uni-freiburg.de/streetcrossing/](http://aisdatasets.informatik.uni-freiburg.de/streetcrossing/)

Introducing paper: Multimodal Interaction-aware Motion Prediction for Autonomous Street Crossing
Freiburg Campus 3D Scan
Full name: Freiburg Campus 3D Scan

The **Freiburg Campus 3D Scan** dataset consists of 3D area maps from the Freiburg campus that were scanned with 3D lasers. Areas include corridors, the outdoor campus, and some of the colleges and buildings.

Source: [http://aisdatasets.informatik.uni-freiburg.de/streetcrossing/](http://aisdatasets.informatik.uni-freiburg.de/streetcrossing/)
Image Source: [http://ais.informatik.uni-freiburg.de/projects/datasets/octomap/](http://ais.informatik.uni-freiburg.de/projects/datasets/octomap/)
Plant Centroids
Full name: Plant Centroids

**Plant Centroids** is a dataset for stem emerging points (SEP) detection in RGB and NIR image data. The dataset is meant to aid the construction of agricultural robots, where detecting SEPs is an important perception task (to position weeding or fertilizing tools at the plant’s center and finding natural landmarks in the field environment). The dataset contains annotations for ~2000 image sets with a broad variance of plant species and growth stages.

Source: [http://plantcentroids.cs.uni-freiburg.de/](http://plantcentroids.cs.uni-freiburg.de/)
Image Source: [http://plantcentroids.cs.uni-freiburg.de/](http://plantcentroids.cs.uni-freiburg.de/)
Freiburg Across Seasons
Full name: Freiburg Across Seasons

**Freiburg Across Seasons** captures long-term perceptual changes across a span of 3 years. Image sequences were recorded with a forward facing bumblebee stereo camera mounted on a car. During summer, the camera was mounted outside the car where as during winters the camera was inside the car. The image sequences are recorded at relatively low frame rates of 1Hz and 4Hz. All the images have a resolution of 1024 × 768 (width×height) and are JPEG compressed. In total, there are ground truth matchings for 8,133 images for localization based on GPS position.

Source: [http://aisdatasets.informatik.uni-freiburg.de/freiburg_across_seasons/](http://aisdatasets.informatik.uni-freiburg.de/freiburg_across_seasons/)
Image Source: [http://aisdatasets.informatik.uni-freiburg.de/freiburg_across_seasons/](http://aisdatasets.informatik.uni-freiburg.de/freiburg_across_seasons/)
Freiburg Terrains
Full name: Freiburg Terrains

**Freiburg Terrains** consist of three parts: 3.7 hours of audio recordings of the microphone pointed at the robot wheels. It also contains 24K RGB images from the camera mounted on top of the robot. The dataset creators also provide the SLAM poses for each data collection run. The dataset can be used for terrain classification which is useful for agent navigation tasks.

Source: [http://deepterrain.cs.uni-freiburg.de/](http://deepterrain.cs.uni-freiburg.de/)
Image Source: [http://deepterrain.cs.uni-freiburg.de/](http://deepterrain.cs.uni-freiburg.de/)

Introducing paper: Self-Supervised Visual Terrain Classification from Unsupervised Acoustic Feature Learning
Freiburg Block Tasks
Full name: Freiburg Block Tasks

**Freiburg Block Tasks** is a dataset for robot skill learning. It consists of two datasets.
The first data set consisted of three simulated robot tasks: stacking (A), color pushing (B) and color stacking (C). The data set contains 300 multi-view demonstration videos per task. The tasks are simulated with PyBullet. Of these 300 demonstrations, 150 represent unsuccessful executions of the different tasks. The authors found it helpful to add unsuccessful demonstrations in the training of the embedding to enable training RL agents on it. Without fake examples, the distances in the embedding space for states not seen during training might be noisy. The test set contains the manipulation of blocks. Within the validation set, the blocks are replaced by cylinders of different colors.
The second data set includes real-world human executions of the simulated robot tasks (A, B and C), as well as demonstrations for a task where one has to first separate blocks in order to stack them (D). For each task, there are 60 multi-view demonstration videos, corresponding to 24 minutes of interaction. In contrast to the simulated data set, the real demonstrations contain no unsuccessful executions and are of varying length. The test set contains blocks of unseen sizes and textures, as well as unknown backgrounds.

Source: [http://robotskills.cs.uni-freiburg.de/](http://robotskills.cs.uni-freiburg.de/)
Image Source: [http://robotskills.cs.uni-freiburg.de/](http://robotskills.cs.uni-freiburg.de/)
Cityscapes-Motion
Full name: Cityscapes-Motion

The **Cityscapes-Motion** dataset is a supplement to the semantic annotations provided by the Cityscapes dataset, containing 2975 training images and 500 validation images. The dataset creators provide manually annotated motion labels for the category of cars. The images are of resolution 2048×1024 pixels. The task to learn is not just semantic segmentation but also the motion status of the objects.

Source: [http://deepmotion.cs.uni-freiburg.de/](http://deepmotion.cs.uni-freiburg.de/)
Image Source: [http://deepmotion.cs.uni-freiburg.de/](http://deepmotion.cs.uni-freiburg.de/)
KITTI-Motion
Full name: KITTI-Motion

The **KITTI-Motion** dataset contains pixel-wise semantic class labels and moving object annotations for 255 images taken from the KITTI Raw dataset. The images are of resolution 1280×384 pixels and contain scenes of freeways, residential areas and inner-cities. The task is not just to semantically segment objects but also to identify their motion status.

Source: [http://deepmotion.cs.uni-freiburg.de/](http://deepmotion.cs.uni-freiburg.de/)
Image Source: [http://deepmotion.cs.uni-freiburg.de/](http://deepmotion.cs.uni-freiburg.de/)
MobilityAids
Full name: MobilityAids

**MobilityAids** is a dataset for perception of people and their mobility aids. The annotated dataset contains five classes: pedestrian, person in wheelchair, pedestrian pushing a person in a wheelchair, person using crutches and person using a walking frame. In total the hospital dataset has over 17, 000 annotated RGB-D images, containing people categorized according to the mobility aids they use. The images were collected in the facilities of the Faculty of Engineering of the University of Freiburg and in a hospital in Frankfurt.

Source: [http://mobility-aids.informatik.uni-freiburg.de/](http://mobility-aids.informatik.uni-freiburg.de/)
Image Source: [http://mobility-aids.informatik.uni-freiburg.de/](http://mobility-aids.informatik.uni-freiburg.de/)

Introducing paper: Deep Detection of People and their Mobility Aids for a Hospital Robot
RobotPush
Full name: RobotPush

**RobotPush** is a dataset for object singulation – the task of separating cluttered objects through physical interaction. The dataset contains 3456 training images with labels and 1024 validation images with labels. It consists of simulated and real-world data collected from a PR2 robot that equipped with a Kinect 2 camera. The dataset also contains ground truth instance segmentation masks for 110 images in the test set.

Source: [http://robotpush.cs.uni-freiburg.de/](http://robotpush.cs.uni-freiburg.de/)
Image Source: [http://robotpush.cs.uni-freiburg.de/](http://robotpush.cs.uni-freiburg.de/)

Introducing paper: Learning to Singulate Objects using a Push Proposal Network
DeepLocCross
Full name: DeepLocCross

**DeepLocCross** is a localization dataset that contains RGB-D stereo images captured at 1280 x 720 pixels at a rate of 20 Hz. The ground-truth pose labels are generated using a LiDAR-based SLAM system. In addition to the 6-DoF localization poses of the robot, the dataset additionally contains tracked detections of the observable dynamic objects. Each tracked object is identified using a unique track ID, spatial coordinates, velocity and orientation angle. Furthermore, as the dataset contains multiple pedestrian crossings, labels at each intersection indicating its safety for crossing are provided.
This dataset consists of seven training sequences with a total of 2264 images, and three testing sequences with a total of 930 images. The dynamic nature of the surrounding environment at which the dataset was captured renders the tasks of localization and visual odometry estimation extremely challenging due to the varying weather conditions, presence of shadows and motion blur caused by the movement of the robot platform. Furthermore, the presence of multiple dynamic objects often results in partial and full occlusions to the informative regions of the image. Moreover, the presence of repeated structures render the pose estimation task more challenging. Overall this dataset covers a wide range of perception related tasks such as loop closure detection, semantic segmentation, visual odometry estimation, global localization, scene flow estimation and behavior prediction.

Source: [http://deeploc.cs.uni-freiburg.de/](http://deeploc.cs.uni-freiburg.de/)
Image Source: [http://deeploc.cs.uni-freiburg.de/](http://deeploc.cs.uni-freiburg.de/)

Introducing paper: VLocNet++: Deep Multitask Learning for Semantic Visual Localization and Odometry
DeepLoc
Full name: DeepLoc

**DeepLoc** is a large-scale urban outdoor localization dataset. The dataset is currently comprised of one scene spanning an area of 110 x 130 m, that a robot traverses multiple times with different driving patterns. The dataset creators use a LiDAR-based SLAM system with sub-centimeter and sub-degree accuracy to compute the pose labels that provided as groundtruth. Poses in the dataset are approximately spaced by 0.5 m which is twice as dense as other relocalization datasets.

Furthermore, for each image the dataset creators provide pixel-wise semantic segmentation annotations for ten categories: Background, Sky, Road, Sidewalk, Grass, Vegetation, Building, Poles & Fences, Dynamic and Void. The dataset is divided into a train and test splits such that the train set comprises seven loops with alternating driving styles amounting to 2737 images, while the test set comprises three loops with a total of 1173 images. The dataset also contains global GPS/INS data and LiDAR measurements.

This dataset can be very challenging for vision based applications such as global localization, camera relocalization, semantic segmentation, visual odometry and loop closure detection, as it contains substantial lighting, weather changes, repeating structures, reflective and transparent glass buildings.

Source: [http://deeploc.cs.uni-freiburg.de/](http://deeploc.cs.uni-freiburg.de/)
Image Source: [http://deeploc.cs.uni-freiburg.de/](http://deeploc.cs.uni-freiburg.de/)

Introducing paper: VLocNet++: Deep Multitask Learning for Semantic Visual Localization and Odometry
Freiburg Lighting Adaptable Map Tracking
Full name: Freiburg Lighting Adaptable Map Tracking

**Freiburg Lighting Adaptable Map Tracking** is a dataset for camera trajectory estimation. The dataset consists of two subdatasets, each consisting of a Lighting Adaptable Map and three camera trajectories recorded under varying lighting conditions. The map meshes are stored in PLY format with custom properties and elements. The trajectories contain synchronized RGB-D images, exposure times and gains, ground-truth light settings and camera poses, as well as the camera tracking results presented in the paper.

Source: [http://tracklam.informatik.uni-freiburg.de/](http://tracklam.informatik.uni-freiburg.de/)
Image Source: [http://tracklam.informatik.uni-freiburg.de/](http://tracklam.informatik.uni-freiburg.de/)
Freiburg Poking
Full name: Freiburg Poking

The **Freiburg Poking** dataset is a dataset for learning intuitive physics from physical interaction. It consists of 40K of interaction data with a KUKA LBR iiwa manipulator and a fixed Azure Kinect RGB-D camera. The dataset creators built an arena of styrofoam with walls for preventing objects from falling down. At any given time there were 3-7 objects randomly chosen from a set of 34 distinct objects present on the arena. The objects differed from each other in shape, appearance, material, mass and friction.

Source: [http://hind4sight.cs.uni-freiburg.de/](http://hind4sight.cs.uni-freiburg.de/)
Image Source: [http://hind4sight.cs.uni-freiburg.de/](http://hind4sight.cs.uni-freiburg.de/)
7-Scenes
Full name: 7-Scenes

The **7-Scenes** dataset is a collection of tracked RGB-D camera frames. The dataset may be used for evaluation of methods for different applications such as dense tracking and mapping and relocalization techniques.
All scenes were recorded from a handheld Kinect RGB-D camera at 640×480 resolution. The dataset creators use an implementation of the KinectFusion system to obtain the ‘ground truth’ camera tracks, and a dense 3D model. Several sequences were recorded per scene by different users, and split into distinct training and testing sequence sets.

Source: [https://www.microsoft.com/en-us/research/project/rgb-d-dataset-7-scenes/](https://www.microsoft.com/en-us/research/project/rgb-d-dataset-7-scenes/)
Image Source: [https://www.microsoft.com/en-us/research/project/rgb-d-dataset-7-scenes/](https://www.microsoft.com/en-us/research/project/rgb-d-dataset-7-scenes/)
Cross-Dataset Testbed
Full name: Cross-Dataset Testbed

The Cross-dataset Testbed is a Decaf7 based cross-dataset image classification dataset, which contains 40 categories of images from 3 domains: 3,847 images in Caltech256, 4,000 images in ImageNet, and 2,626 images for SUN. In total there are 10,473 images of 40 categories from these three domains.

Source: [Probability Weighted Compact Feature for Domain Adaptive Retrieval](https://arxiv.org/abs/2003.03293)
Image Source: [https://sites.google.com/site/crossdataset/](https://sites.google.com/site/crossdataset/)

Introducing paper: A Testbed for Cross-Dataset Analysis
TUM Kitchen
Full name: TUM Kitchen

The **TUM Kitchen** dataset is an action recognition dataset that contains 20 video sequences captured by 4 cameras with overlapping views. The camera network captures the scene from four viewpoints with 25 fps, and every RGB frame is of the resolution 384×288 by pixels. The action labels are frame-wise, and provided for the left arm, the right arm and the torso separately.

Source: [Temporal Human Action Segmentation via Dynamic Clustering](https://arxiv.org/abs/1803.05790)
Image Source: [https://ias.in.tum.de/dokuwiki/software/kitchen-activity-data](https://ias.in.tum.de/dokuwiki/software/kitchen-activity-data)

Introducing paper: The TUM Kitchen Data Set of everyday manipulation activities for motion tracking and action recognition
HIC
Full name: Hands in Action

The Hands in action dataset (**HIC**) dataset has RGB-D sequences of hands interacting with objects.

Source: [Learning joint reconstruction of hands and manipulated objects](https://arxiv.org/abs/1904.05767)
Image Source: [http://files.is.tue.mpg.de/dtzionas/Hand-Object-Capture/](http://files.is.tue.mpg.de/dtzionas/Hand-Object-Capture/)

Introducing paper: Capturing Hands in Action using Discriminative Salient Points and Physics Simulation
Parzival
Full name: Parzival

The **Parzival** dataset consists of 47 pages by three writers. These pages were taken from a medieval German manuscript from the 13th century that contains the epic poem Parzival by Wolfram von Eschenbach. The image size is 2000 x 3000 pixels. 24 pages are selected as training set; 14 pages are selected as test set; 2 pages are selected as validation set.

Source: [https://diuf.unifr.ch/main/hisdoc/divadia](https://diuf.unifr.ch/main/hisdoc/divadia)
Image Source: [https://diuf.unifr.ch/main/hisdoc/divadia](https://diuf.unifr.ch/main/hisdoc/divadia)

Introducing paper: Language Model Integration for the Recognition of Handwritten Medieval Documents
CDTB
Full name: Color-and-Depth Tracking



Source: [https://www.vicos.si/Projects/CDTB 4.2 State-of-the-art Comparison A TH CTB (color-and-depth visual object tracking) dataset is recorded by several passive and active RGB-D setups and contains indoor as well as outdoor sequences acquired in direct sunlight. The sequences were recorded to contain significant object pose change, clutter, occlusion, and periods of long-term target absence to enable tracker evaluation under realistic conditions. Sequences are per-frame annotated with 13 visual attributes for detailed analysis. It contains around 100,000 samples.](https://www.vicos.si/Projects/CDTB 4.2 State-of-the-art Comparison A TH CTB (color-and-depth visual object tracking) dataset is recorded by several passive and active RGB-D setups and contains indoor as well as outdoor sequences acquired in direct sunlight. The sequences were recorded to contain significant object pose change, clutter, occlusion, and periods of long-term target absence to enable tracker evaluation under realistic conditions. Sequences are per-frame annotated with 13 visual attributes for detailed analysis. It contains around 100,000 samples.)
Image Source: [https://www.vicos.si/Projects/CDTB](https://www.vicos.si/Projects/CDTB)

Introducing paper: CDTB: A Color and Depth Visual Object Tracking Dataset and Benchmark
EgoDexter
Full name: EgoDexter

The **EgoDexter** dataset provides both 2D and 3D pose annotations for 4 testing video sequences with 3190 frames. The videos are recorded with body-mounted camera from egocentric viewpoints and contain cluttered backgrounds, fast camera motion, and complex interactions with various objects. Fingertip positions were manually annotated for 1485 out of 3190 frames.

Source: [Hand Pose Estimation via Latent 2.5D Heatmap Regression](https://arxiv.org/abs/1804.09534)
Image Source: [https://handtracker.mpi-inf.mpg.de/projects/OccludedHands/EgoDexter.htm](https://handtracker.mpi-inf.mpg.de/projects/OccludedHands/EgoDexter.htm)

Introducing paper: Real-time Hand Tracking under Occlusion from an Egocentric RGB-D Sensor
SynthHands
Full name: SynthHands

The **SynthHands** dataset is a dataset for hand pose estimation which consists of real captured hand motion retargeted to a virtual hand with natural backgrounds and interactions with different objects. The dataset contains data for male and female hands, both with and without interaction with objects. While the hand and foreground object are synthtically generated using Unity, the motion was obtained from real performances as described in the accompanying paper. In addition, real object textures and background images (depth and color) were used. Ground truth 3D positions are provided for 21 keypoints of the hand.

Source: [Egocentric 6-DoF Tracking of Small Handheld Objects](https://arxiv.org/abs/1804.05870)
Image Source: [https://handtracker.mpi-inf.mpg.de/projects/OccludedHands/SynthHands.htm](https://handtracker.mpi-inf.mpg.de/projects/OccludedHands/SynthHands.htm)

Introducing paper: Real-time Hand Tracking under Occlusion from an Egocentric RGB-D Sensor
Washington RGB-D Scenes v2
Full name: Washington RGB-D Scenes v2

The RGB-D Scenes Dataset v2 consists of 14 scenes containing furniture (chair, coffee table, sofa, table) and a subset of the objects in the RGB-D Object Dataset (bowls, caps, cereal boxes, coffee mugs, and soda cans). Each scene is a point cloud created by aligning a set of video frames using Patch Volumes Mapping.

Source: [https://rgbd-dataset.cs.washington.edu/dataset/rgbd-scenes-v2/](https://rgbd-dataset.cs.washington.edu/dataset/rgbd-scenes-v2/)
Image Source: [https://arxiv.org/abs/1904.02530](https://arxiv.org/abs/1904.02530)
Washington RGB-D Scenes
Full name: Washington RGB-D Scenes

The RGB-D Scenes Dataset contains 8 scenes annotated with objects that belong to the Washington RGB-D Object Dataset. Each scene is a single video sequence consisting of multiple RGB-D frames.

Source: [https://rgbd-dataset.cs.washington.edu/dataset/rgbd-scenes-v2/](https://rgbd-dataset.cs.washington.edu/dataset/rgbd-scenes-v2/)
Image Source: [https://arxiv.org/abs/1904.02530](https://arxiv.org/abs/1904.02530)