# Estimation of Hand Motion from Piezoelectric Soft Sensor Using Deep Recurrent Network
## 1 Introduction  
In recent years, human-machine interaction techniques have been spotlighted in robotics research. For human-machine interaction, accurate representation of the human body state is important. In studies on human motion recognition, various sensors are used to measure human body states [1; 2]. In particular, soft sensors have received significant attention with respect to human-machine interaction and are applied to many wearable applications due to their high flexibility and adaptability [3; 4; 5]. The raw data acquired from various sensors need to be appropriately processed to be used in a motion state estimation. However, if the sensor has a nonlinear characteristic, it is hard to analyze or utilize. To solve this problem, recent studies have proposed the use of networks using deep learning to predict the nonlinear correlation [6; 7; 8; 9]. Given that the soft sensor has higher nonlinear characteristics compared to traditional sensors [10], deep learning has been applied to characterize soft sensors and motion recognition in [11; 12; 13]. Among the various soft sensors, those made of piezoelectric materials are well suited for human-machine interactions in wearable devices [14; 15] due to their flexibility associated with micro-scale thickness, low power, and convenience in installation [16; 17]. In our previous studies, we demonstrated the performance of smart gloves using flexible piezoelectric sensors and demonstrated the interaction ability of these gloves in virtual reality applications [18; 19]. Apart from these advantages, piezoelectric sensors exhibit high nonlinearity and noise similar to other soft sensors. In this study, we adopted deep learning as a method to address this problem. Specifically, the offsets of piezoelectric sensors were randomly changed in addition to the nonlinearly in a real-time system. Since we followthe offset value only in the case that the sensor does not move, the accuracy of the sensor offset value is related to the estimation of the moving state. At the same time, the angle value is calculated using the sensor offset value. This means that a good estimation of the moving state leads to a good result for angle estimation. Thus, we propose a network to predict the moving state for stabilization of the offset value. Moreover, we incorporated the trained network into the interface system for the smart glove to recognize hand joint angles. Given that our goal is to apply the proposed network to a the real-time virtual reality interaction system, we designed it to be suitable for processing sequential data at high speed. In the proposed network, sequential sensor data, which are analog voltages, are used as the input data. We introduce the structure of the network using the Long-Short Term Memory (LSTM) units for predicting outputs and utilized the estimated result in our glove system. Several previous studies have already used the LSTM unit for processing of sequential sensor data [11; 12; 20; 21] and the performance of the network was demonstrated. Training of the network was conducted using a leap motion controller as the ground-truth data. The moving state, the target data for training, was defined with the measured angle value, which means that if the value is 1, the sensor is in moving state and static when value is 0. The performance of the network was demonstrated by two different experiments, with pre-acquired datasets and using a real-time interface system with a smart glove. We evaluated the accuracy of network predictions using our training and test dataset. In addition, the performance of the network for real-time application was evaluated using two criteria, an identical ratio of moving states and the similarity of the joint angles.