DOI: 10.5281/zenodo.15178094
title: Frame-Labeled 60 GHz FMCW Radar Gesture Dataset

description: As the field of human-computer interaction continues to evolve, there is a growing need for robust datasets that can enable the development of gesture recognition systems that operate reliably in diverse real-world scenarios. We present a radar-based gesture dataset, recorded using the BGT60TR13C XENSIV™ 60GHz Frequency Modulated Continuous Radar sensor to address this need. This dataset includes both nominal gestures and anomalous gestures, providing a diverse and challenging benchmark for understanding and improving gesture recognition systems.

The dataset contains a total of 49,000 gesture recordings, with 25,000 nominal gestures and 24,000 anomalous gestures. Each recording consists of 100 frames of raw radar data, accompanied by a label file that provides annotations for every individual frame in each gesture sequence. This frame-based annotation allows for high-resolution temporal analysis and evaluation.

Nominal Gesture Data

The nominal gestures represent standard, correctly performed gestures. These gestures were collected to serve as the baseline for gesture recognition tasks. The details of the nominal data are as follows:





Gesture Types: The dataset includes five nominal gesture types:



Swipe Left

Swipe Right

Swipe Up

Swipe Down

Push





Total Samples: 25,000 nominal gestures.




Participants: The nominal gestures were performed by 12 participants (p1 through p12).



Each nominal gesture has a corresponding label file that annotates every frame with the nominal gesture type, providing a detailed temporal profile for training and evaluation purposes.

Anomalous Gesture Data

The anomalous gestures represent deviations from the nominal gestures. These anomalies were designed to simulate real-world conditions in which gestures might be performed incorrectly, under varying speeds, or with modified execution patterns. The anomalous data introduces additional challenges for gesture recognition models, testing their ability to generalize and handle edge cases effectively.





Total Samples: 24,000 anomalous gestures.




Anomaly Types: The anomalous gestures include three distinct types of anomalies:



Fast Executions: Gestures performed at a rapid pace, lasting approximately 0.1 seconds (much faster than the nominal average of 0.5 seconds).

Slow Executions: Gestures performed at a significantly slower pace, lasting approximately 3 seconds (much slower than the nominal average).

Wrist Executions: Gestures performed using the wrist instead of a fully extended arm, significantly altering the execution pattern.





Participants: The anomalous gestures involved contributions from eight participants, including p1, p2, p6, p7, p9, p10, p11, and p12.




Locations: All anomalous gestures were collected in location e1 (a closed-space meeting room).



 

Radar Configuration Details

The radar system was configured with an operational frequency range spanning from 58.5 GHz to 62.5 GHz. This configuration provides a range resolution of 37.5 mm and the ability to resolve targets at a maximum range of 1.2 meters. For signal transmission, the radar employed a burst configuration comprising 32 chirps per burst with a frame rate of 33 Hz and a pulse repetition time of 300 µs.

 

Data Format

The data for each user, categorized by location and anomaly type, is saved in compressed .npz files. Each .npz file contains key-value pairs for the data and its corresponding labels. The file naming convention is as follows:UserLabel_EnvironmentLabel(_AnomalyLabel).npy. For nominal gestures, the anomaly label is omitted.

 

The .npz file contains two primary keys:



inputs: Represents the raw radar data.

targets: Refers to the corresponding label vector for the raw data.


The raw radar data inputsis stored as a NumPy array with 5 dimensions, structured as follows:n_recordings x n_frames x n_antennas x n_chirps x n_samples, where:



n_recordings: The number of gesture sequence instances (i.e., recordings).

n_frames: The frame length of each gesture (100 frames per gesture).

n_antennas: The number of virtual antennas (3 antennas).

n_chirps: The number of chirps per frame (32 chirps).

n_samples: The number of samples per chirp (64 samples).



The labels targetsare stored as a NumPy array with 2 dimensions, structured as follows:n_recordings x n_frames, where:



n_recordings: The number of gesture sequence instances (i.e., recordings).

n_frames: The frame length of each gesture (100 frames per gesture).


Each entry in the targets matrix corresponds to the frame-level label for the associated raw radar data in inputs.

The total size of the dataset is approximately 48.1 GB, provided as a compressed file named radar_dataset.zip.

 

Metadata

The user labels are defined as follows:



p1: Male

p2: Female

p3: Female

p4: Male

p5: Male

p6: Male

p7: Male

p8: Male

p9: Male

p10: Female

p11: Male

p12: Male


The environmental labels included in the dataset are defined as follows:



e1: Closed-space meeting room

e2: Open-space office room

e3: Library

e4: Kitchen

e5: Exercise room

e6: Bedroom

 


The anomaly labels included in the dataset are defined as follows:



fast: Fast gesture execution

slow: Slow gesture execution

wrist: Wrist gesture execution


This dataset represents a robust and diverse set of radar-based gesture data, enabling researchers and developers to explore novel models and evaluate their robustness in a variety of scenarios. The inclusion of frame-based labeling provides an additional level of detail to facilitate the design of advanced gesture recognition systems that can operate with high temporal resolution.

Disclaimer

This dataset builds upon the version previously published on IEEE DataExplorer (https://ieee-dataport.org/documents/60-ghz-fmcw-radar-gesture-dataset), which included only one label per recording. In contrast, this version includes frame-based labels, providing individual annotations for each frame of the recorded gestures. By offering more granular labeling, this dataset further supports the development and evaluation of gesture recognition models with enhanced temporal precision. However, the raw radar data remains unchanged compared to the dataset available on IEEE DataExplorer.