# Sequence-Information Recognition Method Based on Integrated mDTW
## 3 Methodology
### Joint Modeling with Sequence Processing Networks  
In the framework proposed herein, sequence data are initially processed using multivariate Dynamic Time Warping (mDTW), followed by clustering of the processed data to achieve a coarse categorization. Subsequently, the results of this coarse categorization are utilized as conditional inputs along with the sequence data (SEQ) for processing by the Conditional Gated Recurrent Unit (cGRU). The role of the cGRU is to enable the model to consider additional conditional or contextual information, therefore enhancing adaptability and performance for specific tasks. In this model, the additional condition provided is the coarse categorization result obtained from the initial clustering, while the sequence data itself offers contextual information. The processed data are then fed into a fully connected neural network, where fine categorization is performed using the SoftMax function, resulting in the final categorization outcomes after balancing the loss. As shown in Figure 4, the formulas for cGRU are as follows:  
\[d_{t}=\tanh(W_{d}d_{t}+b_{d}) \tag{11}\]  
\[s_{t}=\tanh(W_{s}s_{t}+b_{s}) \tag{12}\]  
\[r_{t}=\sigma(W_{t}h_{t-1}+U_{r}d_{t}+V_{s}s_{t}+M_{r}fc+b_{r}) \tag{13}\]  
\[z_{t}=\sigma(W_{z}h_{t-1}+U_{z}d_{t}+V_{z}s_{t}+M_{z}fc+b_{z}) \tag{14}\]  
\[\tilde{h}_{t}=\tanh(W(r_{t}\odot h_{t-1})+Ud_{t}+Vs_{t}+Mfc+b) \tag{15}\]  
\[h_{t}=z_{t}\odot h_{t-1}+(1-z_{t})\odot\tilde{h}_{t} \tag{16}\]  
\[o_{t}=\tanh(W_{o}h_{t}+U_{o}d_{t}+V_{o}s_{t}+M_{o}fc+b_{o}) \tag{17}\]The variables utilized in Equations (11)-(17) are defined as follows:\(W\), \(U\), \(V\), and \(M\) represent weight matrices, which are crucial parameters that govern the influence of various inputs on the neurons within a neural network layer. \(b\) denotes the biases, which are added to the weighted sum of inputs to adjust the output of the activation function. \(h_{t}\) signifies the hidden state at time \(t\), encapsulating the internal representation of the network based on the inputs received up until that point. \(o_{t}\) represents the output vector at time \(t\), which is generated by the network after processing the input and hidden state. The input vector\(d_{t}\) corresponds to the \(t\)-th state of the writing motion path signature, denoted as \(\{d_{1,i},d_{2,i}\}\). Here, \(i\) varies from 1 to \(n\), indicating that the signature is comprised of a sequence of \(n\) points. \(s_{t}\) represents the pen's up-down state at time \(t\), indicating whether the pen is touching the writing surface or not. The variable \(fc\) stands for the conditional input, which provides supplementary information to the network to influence its output. The output of the conditional Gated Recurrent Unit (cGRU), \(o_{t}\), is subsequently fed into two fully connected layers. These layers further process the output, ultimately yielding the system's output, which is a \(M\times 5+3\) vector denoted as \(g_{t+1},s_{t+1}\). Here, \(g_{t+1}\) is a vector of size \(M\times 5\), which serves as the input data for a Gaussian Mixture Model (GMM). The GMM is employed to predict pen movements, considering the probability distribution of potential movements. \(s_{t+1}\) is a \(3\times 1\) vector that is used for predicting the pen tip state, indicating its orientation or other pertinent properties. To enhance the performance of our networks, we utilize a Mixture Density Loss function (MDLoss). This loss function is specifically tailored to handle the output of mixture models, such as the GMM used in this context. By minimizing the MDLoss, we aim to refine the accuracy of the predicted pen movements and pen tip states, ultimately bolstering the overall performance of the system in generating realistic and precise handwriting simulations.