# A New Method of Time-Series Event Prediction Based on Sequence Labeling
## 2 Definitions and Methods
### CX-LC Framework
#### 2.2.3 Event Prediction  
Unlike most existing work on modeling individual subsequence sets, this paper introduced the idea of sequence annotation, which not only classifies subsequence sets according to their respective characteristics but also treats all subsequence sets as a whole, taking into account the interaction and coherence between events. RNN is a series of powerful correlation models, which capture the dynamics of time through the loops inthe graph, and is often used in the problem of sequence annotation. In practice, it is difficult for RNN to capture the long-term dependence of timing due to the problem of gradient disappearance or gradient explosion [24]. The long short term memory model (LSTM) [25] is a variant of RNN, which aims to solve these vanishing gradient problems. An LSTM unit consists of four multiplication gates, which control the proportion of the forgotten information and the information passed to the next time step. Figure 3 is a comparison diagram of the basic structure of RNN and LSTM units. In Section 3.2, the pattern features of the subsequence set \(TProb_{m}^{Q}\) (\(m=1,2,\ \ldots,\ K-\tau\ +\ 1\)) have been extracted and the pattern features \(TProb_{m}^{Q}\) in turn as the input of the LSTM model, the prediction result of the event \(\hat{Y}_{m\ +\ \tau}\) is obtained. The specific formula is as follows:  
\[ \begin{split}& F_{m}=\sigma\Big{(}W_{F}\Big{[}a_{m-1},T\,\text{ Prob}_{m}^{Q}\Big{]}+b_{F}\Big{)}\\ & I_{m}=\sigma\Big{(}W_{I}\Big{[}a_{m-1},T\,\text{Prob}_{m}^{Q} \Big{]}+b_{I}\Big{)}\\ & O_{m}=\sigma\Big{(}W_{o}\Big{[}a_{m-1},T\,\text{Prob}_{m}^{Q} \Big{]}+b_{o}\Big{)}\\ & C_{m}=F_{m}\circ C_{m-1}+I_{m}\circ\sigma\Big{(}W_{C}\Big{[}a_{ m-1},T\,\text{Prob}_{m}^{Q}\Big{]}+b_{C}\Big{)}\\ & a_{m}=O_{m}\circ\tanh C_{m}\\ &\hat{Y}_{m\ +\ \tau}=\tanh\big{(}W_{p}a_{m}+b_{p}\big{)};\end{split}  \tag{8}\]  
among them, F, I, O, and C are forget gate, input gate, output gate, and memory gate, respectively; a is the intermediate output of the hidden layer, used to store all useful information of time m (and before), and \(tanh\) is activation functions, \"\(\circ\)\" is the element-wise product symbol, \(W_{*}\) is the weight matrix, \(b_{*}\) is the bias matrix, \(W_{*}\) and \(b_{*}\) are both learnable parameters. Using the LSTM model to perform recursive operations, the preliminary predicted value \(\hat{Y}_{m\ +\ \tau}\) of the event is obtained. Because the Adam optimizer [26] has the advantages of simple implementation, efficient calculation, and fewer memory requirements, this article used the Adam optimizer to adjust the learnable parameters in LSTM. The true event \(Y_{m\ +\ \tau}\) has a value range of \(\{0,1\}\), so the following formula for minimizing the cross-entropy loss \(\mathcal{L}\) can be obtained:  
\[\min\mathcal{L}=-\sum Y_{m\ +\ \tau}\log\hat{Y}_{m\ +\ \tau}+(1-Y_{m\ +\ \tau})\log\big{(}1-\hat{Y}_{m\ +\ \tau}\big{)}. \tag{9}\]  
Because LSTM can construct high-dimensional and dense nonlinear features, it has very powerful feature extraction capabilities. However, if we only use LSTM to mark a sequence, the transfer relationship cannot be explicitly modeled. LSTM can only extract complex features by simulating the emission function, but cannot simulate the transfer function, so sometimes it is too tolerant to unreasonable parts. Take the labeling problem of sentence structure as an example; B stands for the beginning of the word, E stands for  
Figure 3: Comparison of the basic structure of RNN and LSTM units. (**a**) Basic structure of RNN unit and (**b**) basic structure of LSTM unit. the end of the word. Logically speaking, the marking result of a complete sentence has a structure similar to BB and EB, which is absolutely wrong, but LSTM pays more attention to the overall loss minimization and does not explicitly model the neighbor transition relationship, so such structural errors are always unavoidable. The linear chain conditional random field (linear-CRF) model brings a transition matrix parameter, which can very effectively solve this problem.