Package: sign-language-datasets
Sign Language DatasetsThis repository includes TFDS data loaders for sign language datasets.InstallationFrom Sourcepipinstallgit+https://github.com/sign-language-processing/datasets.gitPyPipipinstallsign-language-datasetsUsageWe demonstrate a loading script for every dataset inexamples/load.ipynbOur config includes the option to choose the resolution and fps, for example:importtensorflow_datasetsastfdsimportsign_language_datasets.datasetsfromsign_language_datasets.datasets.configimportSignDatasetConfig# Loading a dataset with default configurationaslg_pc12=tfds.load("aslg_pc12")# Loading a dataset with custom configurationconfig=SignDatasetConfig(name="videos_and_poses256x256:12",version="3.0.0",# Specific versioninclude_video=True,# Download and load dataset videosprocess_video=True,# Process videos to tensors, or only save path to videofps=12,# Load videos at constant, 12 fpsresolution=(256,256),# Convert videos to a constant resolution, 256x256include_pose="holistic")# Download and load Holistic pose estimationrwth_phoenix2014_t=tfds.load(name='rwth_phoenix2014_t',builder_kwargs=dict(config=config))DatasetsDatasetVideosPosesVersionsaslg_pc12N/AN/A0.0.1asl-lexNo2.0.0rwth_phoenix2014_tYesHolistic3.0.0autslYesOpenPose, Holistic1.0.0dgs_corpusYesOpenPose, Holistic3.0.0dgs_typesYes3.0.0how2signYesOpenPose1.0.0sign2mintYes1.0.0signtypLinks1.0.0swojs_glossarioYes1.0.0SignBankN/A1.0.0wlaslFailedOpenPoseNonewmtsltYesOpenPose, Holistic1.2.0signsuisseYes1.0.0msaslNoneVideo-Based CSLNoneRVL-SLLL ASLNonengt_corpusYes3.0.0bsl_corpusNoNo3.0.0Data InterfaceWe follow the following interface wherever possible to make it easy to swap datasets.{"id":tfds.features.Text(),"signer":tfds.features.Text()|tf.int32,"video":tfds.features.Video(shape=(None,HEIGHT,WIDTH,3)),"depth_video":tfds.features.Video(shape=(None,HEIGHT,WIDTH,1)),"fps":tf.int32,"pose":{"data":tfds.features.Tensor(shape=(None,1,POINTS,CHANNELS),dtype=tf.float32),"conf":tfds.features.Tensor(shape=(None,1,POINTS),dtype=tf.float32)},"gloss":tfds.features.Text(),"text":tfds.features.Text()}Adding a new datasetFor general instructions, see theTFDS guide to writing custom datasets.
Instructions below are specific to this repository.Make a new folder insidesign_language_datasets/datasetswith the same name as the dataset. As a convention, the name
of the dataset should be lowercase and words should be separated by an underscore. Example:cdsign_language_datasets/datasets
tfdsnewnew_datasetFor our purposes, creating a custom TFDS dataset means writing a new class which inherits
fromtfds.core.GeneratorBasedBuilder. If you usetfds newto create a new dataset then the dataset class is stored
in a file with the exact same name as the dataset, i.e.new_dataset.py.new_dataset.pymust contain a line similar
to:classNewDataset(tfds.core.GeneratorBasedBuilder):Registering a new datasetThe mechanism to add a custom dataset to TFDS' dataset registry is to import the classNewDataset. For this reason the
foldersign_language_datasets/datasets/new_datasetmust have an__init__.pyfile that imports the classNewDataset:from.new_datasetimportNewDatasetEven though the name of the class isNewDataset, it will be available for loading in lowercase and uppercase
characters are interpreted as the start of a new word that should be separated with an underscore. This means that the
class can be loaded as follows:ds=tfds.load('new_dataset')Generating checksumsThe folder for the new dataset should contain a filechecksums.tsvwith checksums for every file in the dataset. This
allows the TFDS download manager to check the integrity of the data it downloads. Use thetfds buildtool to generate
the checksum file:tfdsbuild--register_checksumsnew_dataset.pyUse a dataset configuration which includes all files (e.g. does include the video files if any) using the--configargument. The default behaviour is to build all configurations which might be redundant.Why not Huggingface Datasets?Huggingface datasets do not work well with videos. From the lack of native support of the video type, to lack of support
of arbitrary tensors. Furthermore, they currently have memory leaks that prevent from saving even the smallest of video
datasets.Cite@misc{moryossef2021datasets,title={Sign Language Datasets},author={Moryossef, Amit and M\"{u}ller, Mathias},howpublished={\url{https://github.com/sign-language-processing/datasets}},year={2021}}