# Machine Learning Based Representative Spatio-Temporal Event Documents Classification
## 3 Materials and Methods
### Machine Learning Models
#### 3.2.2 Bidirectional Long Short-Term Memory  
The recurrent neural network (RNN) model is a model suitable for analyzing time-series data as it has a recursive structure in which the result of the hidden layer is input to the next hidden layer [27]. However, in the RNN model, as the sequence lengthens, past information is not transferred well to the next step, and as the step is repeated, a loss of gradient problem may occur in which learning weights disappear. LSTM is a model proposed to solve the vanishing gradient problem of RNNs. LSTM distinguishes important information from unnecessary information through the cell state, passes it on to the next state, and can preserve long-term memory even after repeated steps [28]. LSTM is used for text classification [29] and time-series data processing [30].  
Although LSTM provides excellent performance, it still has a limitation in that the result tends to converge based on the previous pattern because it still processes the input order in chronological order. In order to solve these disadvantages, BiLSTM (Bidirectional Long Short-Term Memory) has been proposed [31]. BiLSTM shows a structure in which a  
Figure 1: The proposed CNN layer and BiLSTM layer-based models.  
forward and a backward hidden layer are connected to the input and output layers, and learning is performed in both directions, and the output value from each hidden layer is calculated to derive the final output value. Since learning is performed in the forward and backward directions separately and then combined, better prediction performance can be expected compared to the existing unidirectional LSTM.  
BiLSTM (Bidirectional Long Short-Term Memory) processes input data in both directions, allowing it to utilize both past and future information. This allows for the transition from short-term memory to long-term memory, which is achieved as follows: first, the memory cell state in the forward LSTM is updated using input, forget, and output gates to adjust the memory cell state. Second, the memory cell state in the backward LSTM is updated using input, forget, and output gates to adjust the memory cell state. The output values from the forward and backward LSTMs are concatenated and used as the input for the next layer. This way, BiLSTM adjusts short-term and long-term memory while processing input data in both directions, allowing it to effectively comprehend and process information in input sequences.  
The layers of the proposed BiLSTM model are shown in Figure 1. The BiLSTM model consists of a word embedding layer, four Bidirectional LSTM layers, and two dense layers. FastText [32] is used for word embedding in this model. Among the parameters of the Bidirectional LSTM layer, units were set to 100. We implemented a Bidirectional LSTM layer using Keras. Among the parameters of one dense layer, unit is set to 100 and activation is set to relu. For binary classification, the unit of the last dense layer is set to 1, and the activation is set to sigmoid.  
To achieve optimal performance in a classification model, it is not enough to apply a BiLSTM layer to the model; one must also find the optimal hyperparameters. Additionally, the depth of the layer that delivers the best performance in terms of time must be identified. We have conducted experiments with various hyperparameters and selected the optimal hyperparameters that demonstrated the best performance.