## 4 Environment and Evaluation
### Evaluation of Multi-Task Learning Model  
Table 1 shows the accuracy of Task 1, and the classes are encoded in order as follows: collision, stranding, contact, fire, sinking, engine damage, human casualty, and marine pollution. As shown in Table 1, the number of collisions and human casualties (class 1 and class 7) account for about 65% of the total data. We use two accuracy metrics, micro-average and macro-average since the imbalance between these types can be caused. As a result, the micro-average and the macro-average of accuracy are 96.53 and 94.96, respectively. As you can see, the micro-average, which is calculated from the individual classes, is greater than the macro-average. Tables 4 and 5 show the prediction performance for Task 2 and Task 3. The classes are the same as that of Table 6. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Class** & **Correct** & **Error** & **Accuracy** \\ \hline
1 & 89 & 12 & 88.12 \\
2 & 73 & 10 & 87.95 \\
3 & 46 & 6 & 88.46 \\
4 & 28 & 5 & 84.85 \\
5 & 115 & 14 & 89.15 \\
11 & 191 & 19 & 90.95 \\
12 & 777 & 72 & 91.52 \\
13 & 123 & 14 & 89.78 \\
26 & 176 & 19 & 90.26 \\
27 & 691 & 59 & 92.13 \\
28 & 137 & 15 & 90.13 \\
36 & 105 & 14 & 88.24 \\
37 & 89 & 10 & 89.90 \\
38 & 111 & 15 & 88.10 \\
39 & 90 & 10 & 90.00 \\
40 & 33 & 5 & 86.84 \\ Total & 3413 & 467 & \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Performance for each label measured in Task 2 of applied article prediction. \begin{table}
\begin{tabular}{c c c} \hline \hline \multirow{2}{*}{**Number of Tasks**} & \multicolumn{2}{c}{**Attention Mechanism**} \\  & **Unused Without** & **With Used** \\ \hline Task 1 & 96.12 & 96.65 \\ Task 2 & 84.55 & 89.10 \\ Task 3 & 95.84 & 95.90 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Accuracy about the usage of attention mechanism for each task. A characteristic common to all prediction tasks is data imbalance by class. The difference between micro-accuracy and macro-accuracy is 1.67% in the first task, 1.14% in the second task, and 2.21% in the third task. This difference is caused by the difference in data distribution between the classes. This difference is even more pronounced when comparing accuracies by class. In the first task, class 1, with the most data, was predicted with an accuracy of 97.48%, while the accuracy of class 8, with the least data, was 89.32%, showing a difference of 8.16%. The second task and the third task also showed a difference of 7.27% and 8.69%, respectively. To solve this problem, it is not appropriate to additionally collect a relatively small number of class data. Owing to the fact that the data used for training and testing are the verdicts issued from 2010 to 2019, a small number of class data means that fewer accidents related to the class occur in reality. Thus, to overcome this issue, imbalance learning is necessary. Analogous to the evaluation in Section 4.2, the performance of the model improved when the attention mechanism was employed. However, to further improve the accuracy, other neural network models that can extract more appropriate features than LSTM from factual information, or a method of adding new features that can be used for prediction, should be applied. In addition, for in-depth analysis of the proposed model, a confusion matrix as shown in Table 7 was conducted using Task 1, which showed the best performance among the three tasks. \begin{table}
\begin{tabular}{c c c c} \hline
**Class** & **Correct** & **Error** & **Accuracy** \\ \hline
1 & 1699 & 44 & 97.48 \\
2 & 318 & 10 & 96.95 \\
3 & 171 & 13 & 92.93 \\
4 & 309 & 10 & 96.87 \\
5 & 153 & 13 & 92.17 \\
6 & 268 & 9 & 96.75 \\
7 & 740 & 20 & 97.37 \\
8 & 92 & 11 & 89.32 \\ Total & 3750 & 130 & \\ \hline \multirow{2}{*}{Accuracy} & Micro & Macro & \\  & 96.66 & 94.98 & \\ \hline \end{tabular}
\end{table} 
Table 6: Performance for each label measured in Task 1 of accident type prediction.