# A C-BiLSTM Approach to Classify Construction Accident Reports
## 3 C-BiLSTM-Based Classification Framework
### BiLSTM  
LSTM model can be used to solve the problem that traditional machine learning models are difficult to extract high-level semantics in texts when classifying texts. This model adopts a text sequence matrix composed of pre-trained distributed word vectors as input and then extracts the feature expressions containing context information by using its unique memory structure. The LSTM model structure is shown in Figure 4a. The standard LSTM network can only leverage the historical context. However, the lack of future context may lead to an incomplete understanding of the meaning of the text. BiLSTM is the combination of a forward LSTM layer and a backward LSTM layer. The information of context can be fully used by summarizing the information of two ideates before and after the word. The model structure is shown in Figure 4b. The key idea of RNN is to use sequential information [51]. Text classification can be treated as a sequential modeling task. Due to its sequential feature, RNN models have been widely used in text classification tasks [52; 53]. The LSTM model is a special type of RNN [54] that overcomes the issue of vanishing gradients during RNN model training. The key point is to find and establish long-term dependencies between input values by its specially designed memory unit so that it can understand more contextual information to extract high-level abstract features from texts. The memory unit structure of the BiLSTM model is shown in Figure 5.  
Figure 4: The architecture of the (**a**) LSTM model and (**b**) BiLSTM model. Figure 5: Memory unit structure of BiLSTM. The most important part of the memory unit is the memory state C which is transmitted directly over the entire structure chain and performs only a small amount of linear operation so that the information can be easily kept unchanged during transmission. At the same time, the memory unit has a smart \"gate\" structure to add or delete the information contained in the memory state. The so-called \"gate\" is a method of selecting information, which includes the pointwise multiplication operation of vectors and the sigmoid function. A complete memory unit mainly includes the following parts: memory \(C_{t-1}\) at time t\(-1\), output \(h_{t-1}\) at time t\(-1\), forget gate \(f_{t}\), gate \(i_{t}\), and output gate \(O_{t}\), among which the values of the three gates are all between 0 and 1, while the memory state \(C_{t-1}\) records the historical information of all previous time nodes, which is the long-term memory of the model, and the \(h_{t-1}\) records the information of the time node right before the current time, which is the short-term memory of the model. The memory state of the \(j^{th}\) memory unit at time \(t\), \(C_{t}^{j}\), is the result of the input gate \(i_{t}^{j}\), forget gate \(f_{t}^{j}\), and the previous memory state \(C_{t-1}^{j}\). Referring to the version of [PERSON] et al. [55], the calculation formula of the memory unit is as follows  
\[C_{t}^{j}=f_{t}^{j}\times C_{t-1}^{j}+i_{t}^{j}\times\overline{C}_{t}^{j} \tag{6}\]  
where  
\[f_{t}=\sigma\big{(}W_{f^{\prime}}[h_{t-1},\ x_{t}]+b_{f}\big{)}i_{t}=\sigma(W _{i^{\prime}}[h_{t-1},\ x_{t}]+b_{i})\overline{C}_{t}=tanh(W_{c^{\prime}}[h_{ t-1},\ x_{t}]+b_{C}) \tag{7}\]  
In Equation (7), W represents the weight matrix corresponding to each control gate, b denotes bias parameter, \(\sigma\) denotes sigmoid activation function, tanh represents the hyperbolic tangent function and \(x_{t}\) represents the input of the model at time \(t\). The input gate \(i_{t}\) and the forget gate \(f_{t}\) respectively control the addition of new information and the deletion of old information.