Accurate and Fast Classification of Foot Gestures for Virtual Locomotion This work explores the use of foot gestures for locomotion in virtual environments. Foot gestures are represented as the distribution of plantar pressure and detected by three sparselylocated sensors on each insole. The Long ShortTerm Memory model is chosen as the classifier to recognize the performeru0027s foot gesture based on the captured signals of pressure information. The trained classifier directly takes the noisy and sparse input of sensor data, and handles seven categories of foot gestures (stand, walk forwardbackward, run, jump, slide left and right) without manual definition of signal features for classifying these gestures. This classifier is capable of recognizing the foot gestures, even with the existence of large sensorspecific, interperson and intraperson variations. Results show that an accuracy of 80% can be achieved across different users with different shoe sizes and 85% for users with the same shoe size. A novel method, DualCheck Till Consensus, is proposed to reduce the latency of gesture recognition from 2 seconds to 0.5 seconds and increase the accuracy to over 97%. This method offers a promising solution to achieve lower latency and higher accuracy at a minor cost of computation workload. The characteristics of high accuracy and fast classification of our method could lead to wider applications of using foot patterns for humancomputer interaction.

Gesture Correctness Estimation with Deep Neural Networks and Rough Path Descriptors One of the classical problems in computer vision is the automatic identification of different gestures or motions from a sequence of frames, captured via cameras or sensors, that usually contain angles or positions of many body joints. A large body of research addressing this problem exists in the literature. However, an interesting variant is not just to identify the gesture, but to decide whether a certain gesture is correctly or incorrectly executed. This can be in the context of physical exercises for rehabilitation in healthcare, robotics, sports, sign language and more. In this work we present our research on the automatic classification of gestures into correct or incorrect categories using the following three different but powerful techniques: convolutional neural networks (CNNs), long shortterm memory (LSTM) neural networks and a technique based on rough path theory to extract unique sequences (i.e. path signatures) from gesture data and to use them as inputs to classical classifiers such as RandomForest (RF) and kNearest Neighbor (kNN). We collected the data we used as part of a collaborative project from Perkeso Tun Razak Rehabilitation Centre, Malacca, Malaysia. We recorded 17 patients performing nine types of gestures. For controls, we recorded 14 healthy individuals performing the same gestures. Our cross subject evaluation yields high accuracy for several subjects and gestures. CNNs and LSTMs achieve accuracy values up to 100% for several subjects in many gestures with the highest average accuracy for some gestures reaching 92%. The best overall classifier when using the rough paths representation was kNN as its average accuracy can reach 96.74% (100% accuracy for a subset of subjects in some gestures). For the three methods, the average accuracy across all gestures is 85.24%, 85.63% and 94.67% respectively. Overall, our research shows a great potential in identifying the right learning algorithm in motion correctness identification for real data.