# Deep Learning Forecasting for Supporting Terminal Operators in Port Business Development
## 4 Results
The filter weights were multiplied by the input data matrix (lag values of the container traffic volume) and combined in a non-linear fashion (via an activation function) to extract the optimal features with respect to the quantity of interest. In addition, three different activation functions were considered to process the time-specific component:  
* The _linear_ activation \(\phi(x)=x\);
* The _tanh_\(\phi(x)=\frac{e^{x}-e^{-x}}{e^{x}+e^{-x}}\);
* The _relu_ function \(\phi(x)=max(0,x)\). The fitting of the neural networks was performed using the Adam optimiser [58], setting the parameters to the default values. The network weights were recursively adjusted, aiming to minimise the mean absolute error between the predicted and actual values. To \begin{table}
\begin{tabular}{l l c c c c c c c} \hline \hline  & & \multicolumn{3}{c}{**Non-Seasonal Orders**} & \multicolumn{3}{c}{**Seasonal Orders**} \\  & **Type** & **AR** & **I** & **MA** & **AR\({}_{12}\)** & **IA\({}_{22}\)** & **AIC** \\ \hline \multirow{6}{*}{2019 Forecasts} & full unloaded foreign & 0 & 1 & 2 & 0 & 1 & 1 & 477.7100 \\  & full unloaded domestic & 0 & 1 & 2 & 0 & 0 & 0 & \(-\)73.9716 \\  & full loaded foreign & 5 & 1 & 0 & 1 & 0 & 1 & 648.0511 \\  & full loaded domestic & 2 & 1 & 0 & 1 & 0 & 0 & 224.9120 \\  & full transit & 0 & 1 & 1 & 0 & 0 & 2 & 797.7719 \\  & empty & 1 & 1 & 1 & 2 & 0 & 0 & 713.5212 \\ \hline \multirow{6}{*}{2020 Forecasts} & full unloaded foreign & 0 & 1 & 2 & 1 & 1 & 2 & 540.6166 \\  & full unloaded domestic & 0 & 1 & 1 & 0 & 0 & 1 & \(-\)58.0310 \\ \cline{1-1}  & full loaded foreign & 0 & 1 & 2 & 0 & 0 & 2 & 717.8455 \\ \cline{1-1}  & full loaded domestic & 1 & 1 & 1 & 1 & 0 & 0 & 252.6635 \\ \cline{1-1}  & full transit & 0 & 1 & 1 & 1 & 0 & 0 & 882.7890 \\ \cline{1-1}  & empty & 1 & 1 & 1 & 2 & 0 & 0 & 794.4201 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Best SARIMA models selected. avoid overfitting, we used leave-out validation by using 10% of the training set as the validation set. The update of the weights was calculated using the remaining 90% of the data, and at each epoch, the loss function was evaluated on the training and validation sets. To avoid overfitting, we saved the weight configuration that produced the lowest value of the loss function on the validation set. This was a set of data that the neural network did not use during training. The performance obtained on this set could be a reliable estimate of the model's forecasting performance. A review of other possible validation techniques can be found in [59]. The results of training a neural network are sensitive to some components of the training process: the starting point of the optimisation algorithm, the random sampling of batches of training data employed to calculate the network weight updates, and so on. Then, analysing the results of a single run could be not sufficient to make a judgment. We fit each architecture 10 times and compared the performance on the testing set in terms of accuracy and stability. Figure 5 shows the box plots of the outsample MAPE for all three networks (LSTM, GRU, and CONV) for the different activation functions. This analysis shows the sensitivity of the different models concerning the starting point of the optimisation algorithm and the random sampling of batches. Some evidence may be noted. First, we observed that the tanh activation (blue boxes) function produced the best performance for all the network architectures investigated. The linear function (red boxes) was the runner-up, while the relu activation (green boxes) produced the lowest out-of-sample accuracy in our tests. Analysis of the variability of the results confirmed this ranking: the tanh activation produced less variable results except for the GRU model, while relu activation was associated with less stable results. According to these results, we can conclude that the tanh activation function appeared to be the best activation function for the investigated phenomenon.