# Handwriting Recognition Based on 3D Accelerometer Data by Deep Learning
## 3 CNN and LSTM for Sequence Recognition
\begin{table}
\begin{tabular}{p{113.8 pt} p{113.8 pt} p{113.8 pt}} \hline \hline
**Algorithm** & **Methodology** & **Limitations** \\ \hline Digital Pen [12] & 3D accelerometer signals are converted to image which is recognized by a neural network & Ten digits written in a special single stroke font \\ \hline WIMU-Based Hand Motion Analysis [13] & Movement and attitude features are extracted from motion sensor and magnetometer signals and recognition is completed by DTW & English lowercase letters and digits written in a special single stroke font \\ \hline Accelerometer-Based Digital Pen [14] & 3D accelerometer signals are recognized by a PNN & Ten digits written in a special single stroke font \\ \hline Air writing [15] & 3D accelerometer and gyroscope signals are recognized by HMMs & English uppercase letters and words \\ \hline Gyroscope-equipped smartphone [4] & 3D gyroscope signals are recognized by stepwise lower-bounded dynamic time warping & English lowercase letters written with a smartphone grabbed as a pen \\ \hline Marker-based Air writing [17] & Handwriting is captured from motion of a marker in a video and recognition is completed by a CNN & Ten digits written in a single stroke \\ \hline PhonePoint Pen [18] & Basic strokes are detected from 3D accelerometer signals by correlation with templates and handwritten characters are recognized by juxtaposition of basic strokes & English letters and digits written using basic strokes, smartphone grabbed as a pen \\ \hline Deep Fisher Discriminant Learning [19] & 3D accelerometer and gyroscope signals are recognized as hand gestures by an F-BiGRU & Six uppercase English letters and six digits written in a predefined stroke ordering \\ \hline Motion data from smartwatch [20] & Features are extracted from accelerometer and gyroscope signals and letter recognition is done by DTW & English uppercase letters written on a whiteboard \\ \hline SHOW [22] & Features are extracted from accelerometer and gyroscope signals and recognition was tested with seven machine learning algorithms & English letters and digits written on a horizontal surface with the elbow as support point \\ \hline MotionHacker [23] & After preprocessing and segmentation, features are extracted from accelerometer and gyroscope signals and letter recognition is performed by random forest classifier & Demonstration of motion sensors-based eavesdropping on handwriting \\ \hline AirScript [7] & Recognition is completed by a fusion of a CNN, and two GRU networks using as input an image derived from 2-DifViz features, post-processed 2-DifViz features and standardized raw data, respectively & Ten digits written in the air \\ \hline Finger Writing with Smartwatch [25] & Energy, posture, motion shape and motion variation features are extracted from accelerometer and gyroscope signals, and three classifiers are tested for recognition (Naive Bayes, linear regression and decision trees) & English lowercase letters written on a surface \\ \hline Trajectory-Based Air Writing [32] & Trajectory of handwriting with fingertip using a video camera and recognition was completed by a CNN and an LSTM & Ten digits written with a predefined stroke ordering \\ \hline Air Writing with Interpolation [33] & Motion sensor data are interpolated and then recognized by a 2D-CNN & Uses datasets of others \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Summary of related approaches. the best architectures for sequence processing. This is because LSTM networks have the ability to extract long-range dependencies. In fact, these networks were proposed to solve the vanishing gradient problem faced by Recurrent Neural Networks (RNN) [35]. The combination of spatial and temporal feature learning is crucial to reliably performing motion sequence recognition. A hybrid CNN-LSTM network is used in this work to extract and exploit these two types of features. The convolutional part obtains spatial features, while the LSTM modules extract short-term temporal information.