Package: handshape-datasets
GoalThere arevarious handshape datasetsfor Sign Language. However:Each dataset has its own format and many are hard to find.Each dataset has its own mapping of handshapes to classes. While signs depend on the specific Sign Language for a country/region, handshapes are universal. Hence, they could be shared between datasets/tasks.This library aims to provide two main features:A simplified API to download and load handshape datasetsA mapping between datasets so that datasets can be merged for training/testing models.This library is awork in progress. Contributions are welcome. If you wish to add a dataset you can make a push request or open an issue.InstallationYou can installhandshape_datasetsvia pip with:pip install handshape_datasetsBasic usageSimply import the module and load a dataset. The following downloads, preprocesses and load to memory theLSA16 dataset:import handshape_datasets as hd
images,metadata = hd.load("lsa16")Afterwards you can, for example, plot the first images of the datasetimport matplotlib.pyplot as plt
plt.imshow(images[0,:,:,:]) # show the first sample of the datasetAdvanced usageimport handshape_datasets as hd
hd.list_datasets() # List available datasets
hd.load("lsa16",version="color",delete=True) # use the color version, delete temporary files
hd.delete_temporary_files("lsa16")# Delete temporary files  (if any)
hd.clear("lsa16") # Delete all the local files for dataset LSA16
hd.info("lsa16") # Shows detailed info of the dataset, including url, data format, fields, etc.Supported datasetsDataset idDownload sizeSize on diskSamplesClasseslsa16640.6 Kb1.2 Mb80016rwth44.8 Mb206.8 Mb335945Irish173.4 Mb515.0 Mb5811426Ciarp10.6 Mb18.6 Mb712710PugeaultASL_A2.1 Gb4.3 Gb6577424PugeaultASL_B317.4 Mb717.9 Mb7267626indianA1.7 Gb1.9 Gb5040140indianB320.5 Mb8.6 Gb5000140Nus12.8 Mb3.6 Mb47910Nus273.7 Mb106.1 Mb275010jsl4.5 Mb7.9 Mb805541psl285.2 Mb1.2 Gb96016You can find more information about the datasets in the followingsign language dataset surveyTraining a handshape classifier with KerasLoad the dataset:x,metadata = handshape_datasets.load("lsa16")
y = metadata["y"]Get the input_shape and number of classes:input_shape = x[0].shape
classes = y.max() + 1Define a model (using a pretrained MobileNet here):base_model = keras.applications.mobilenet.MobileNet(input_shape=(input_shape[0],input_shape[1],3), 
                                                            weights='imagenet', include_top=False)
output = keras.layers.GlobalAveragePooling2D()(base_model.output)
output = keras.layers.Dense(32, activation='relu')(output)
output = keras.layers.Dense(classes, activation='softmax')(output)
model = Model(inputs=base_model.input, outputs=output)
model.compile(optimizer='Adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])Split the dataset intro train/test sets:X_train, X_test, Y_train, Y_test = sklearn.model_selection.train_test_split(x,y,
                                                                                test_size=0.9,
                                                                                stratify=y)Fit the modelhistory = model.fit(X_train, Y_train, batch_size=self.batch_size, epochs=self.epochs, validation_data=(X_test, Y_test))Google Colab example:https://colab.research.google.com/drive/1kY-YrbegGFVT7NqVaeA4RjXYRVlZiISR?usp=sharing