## 2 Materials and Methods
### Modeling
#### 2.2.2 Hyperparameter Tuning for LSTM Model  
This study required tuning the hidden units and number of layers of hyperparameters separately for each model to determine the optimal network structure. First, we separated  
Figure 3: Architectures of the LSTM layer [22,23].  
our data into training and test sets. For the training set, we used statistics on marine accidents from 1981 to 2019, and statistics on marine accidents in 2020 were used for the test set. In order to observe how well the network structure is learning, the training set was further divided into 80% training data and 20% validation data. Validation of the prediction model was conducted using the test set. In addition to the use of the validation set, we also used an early stopping strategy that stops training when there was no improvement in the validation metric to avoid an overfitting issue [26]. Input variables that affect learning and accuracy of the LSTM models include the number of layers in the hidden layer, the number of nodes in each hidden layer, activation functions, optimizers, batch size and epoch size. Finding suitable hyperparameter values used in algorithms is of paramount importance in model design for performance optimization of the LSTM models. The hyperparameter tuning process has no fixed method, and the best hyperparameter can be found through an empirical study. Hyperparameters do not have the absolute best values, but they can find suitable values depending on the data and model used. We tuned hyperparameters, such as the number of layers, number of nodes (i.e., hidden units), epoch size, and batch size, using the automated tuning package in Keras [27]. The other input variables we used in this study were as follows:  
* Time steps: This is the number of observations required by the model as inputs to make a future prediction. We set the time steps as six for this study since we divided time into six time periods based on the navigators' watch duty time.
* Activation function: This determines the output of the model and serves to transfer the calculation result by the weight to the next layer. We used a hyperbolic tangent, which is commonly used as the activation function [28].
* Loss function: The mean squared error (MSE) method was used for loss function in this study. The error on the predicted value is expressed in numbers, and the larger the errors, the larger the values, and vice versa.
* Optimizer: This is used to update the network parameters to minimize loss functions. We chose the Adam optimizer which has the advantage of reducing the load of computational memory [29]. The Adam, which is short for adaptive moment estimation, is an algorithm for the optimization technique for gradient descent. The Adam optimization algorithms have been widely adopted in computer vision and natural language processing applications as an extension to stochastic gradient descent. In order to update network weights iteratively based on training data, the Adam can be used instead of conventional stochastic gradient descent [29].
* Batch size: This determines the size of the data to be learned at each training stage, and in this paper, it was empirically determined by examining the effect of batch size of 32, 64, 128, 256, and 512 on the prediction performance for all four LSTM networks.
* Epoch: This is the number of repetitive learnings for the entire training data by batch size. The best epoch we used in this study is 100, which was determined by applying different epoch sizes of 50, 100, 200, 300, 400, and 500.