# Walking Gait Phase Detection Based on Acceleration Signals Using LSTM-DNN Algorithm
## 3 Materials and Methods
### Proposed LSTM-DNN Algorithm
The training data set contains five features extracted from the subjects' gait events. The DNN is a feed-forward artificial neural network consisting of an input layer, an output layer, and at least two hidden layers [45]. Several important parameters need to be set, including the gradient descent optimizer, learning rate, activation function, and training steps. In most machine learning tasks, the objective is to minimize loss and, in the case of loss definitions, the subsequent task is to select and define the optimizer. Gradient optimization is a common method of deep learning, that is to say, the optimizer is actually the optimization of the gradient descent algorithm at the end. The optimizer chosen in this study is the AdaGrad algorithm. The original gradient descent optimization algorithm converges slowly near the bottom of a slope but this is avoided when the AdaGrad algorithm is used. Because of the steep directional gradient, the learning rate will decay faster, which is conducive to the parameter moving in the direction closer to the bottom of the slope, thus accelerating the convergence [46]. The learning rate has a crucial influence on the learning quality of the established learning model. The smaller the learning rate, the more sophisticated the learning is but at the same time the learning  
Figure 5: Long short-term memory (LSTM) cell structure. speed is also reduced, which will easily cause overmeshing and reduces the generalization ability of the training model. Similarly, if the learning rate is too high, the span of each step is too large and the model will miss a lot of important information in the training process, resulting in poor detection accuracy of the training model [47]. The learning rate is generally related to the selected optimizer, the neural network model, the number of training steps, and the size of the sample set. In the LSTM-DNN algorithm, the learning rate is 0.001, 0.003, 0.005, 0.008, 0.01, 0.05, 0.08, 0.1, 0.03, 0.5, 0.7, and 0.8, respectively. Finally, when the learning rate is 0.5, each evaluation index is relatively high. There are three multi-label hidden layers. In a multilayer neural network, there is a functional relationship between the output of the upper node and the input of the lower node, which is called the activation function. In this study, the ReLu function is used as the activation function of the neurons. The important feature of AdaGrad algorithm, which is used as the optimizer, is that it automatically adjusts the learning rate, thereby avoiding the occurrence of the dead ReLu problem caused by a very large learning rate. The softmax classifier is an extension of the logistic regression model for multi-classification problems. For multi-classification, the softmax function is generally used for the activation function of the output layer [42]. For multi-classification problems, it is evident from Equation (8) that the softmax function maps the output of multiple neurons to the interval of (0,1). When selecting the output node, we can select the node with the highest probability as the prediction target of the model, thus achieving multi-classification. \[S_{i}=\frac{e^{i}}{\sum\limits_{j=1}^{m}e^{j}} \tag{8}\]  
The acceleration signal obtained in this study is a time series signal; therefore, it can be input directly into the LSTM network for preprocessing. The LSTM network output layer signal represents the input into the DNN; the DNN improves the LSTM network output signal, resulting in greater accuracy. The structure of the LSTM-DNN is shown in Figure 6. The pre-processed data consisted of 18 parameters and a total of 198,930 learning data point were calculated with the LSTM-DNN algorithm. The LSTM network includes two important parameters, i.e., num_units and forget_bias. Num_units refers to the number of neurons in a cell and forget_bias refers to the bias in the forget gates. In this study, there are 36 num_units and the forget_bias is 0.7.