# DERN: Deep Ensemble Learning Model for Short- and Long-Term Prediction of Baltic Dry Index
## 4 Experiments
### Short-Term Prediction  
Short-term prediction finds one-step-ahead BDI data; the predicting mechanism follows the model in Figure 3a, due to the fact that the data is weekly. Therefore, it predicts one week ahead of the BDI. Figure 4 depicts the prediction of BDI in the testing phase using ARIMA, MLP, LSTM, Deep RNN, GRU, and DERN. As we can see, all of them approximately predict the BDI correctly. In this case, the worst method is the ARIMA model. As shown in Table 4, the ARIMA error rate is three times higher than DERN in terms of RMSE. Notice that by changing the portion of training data, the error \begin{table}
\begin{tabular}{l l l l} \hline \hline
**Hyper-Parameter** & **RNN** & **LSTM** & **GRU** \\ \hline Recurrent layer & 2 RNN layer & 2 LSTM layer & 2 LSTM layer \\ Hidden units & 500 of each layer & 500 of each layer & 500 of each layer \\ Gate activation & tanh & tanh & tanh \\ Dropout & 20\% of each layer & 20\% of each layer & 20\% of each layer \\ \hline Wrapper layer & 1 TimeDistributed layer & 1 TimeDistributed layer & 1 TimeDistributed layer \\ Hidden units & \# of step prediction & \# of step prediction & \# of step prediction \\ Activation & Sigmoid & Sigmoid & Sigmoid \\ \hline Loss function & MSE & MSE & MSE \\ Optimizer & Adam & Adam & Adam \\ \hline Ensemble layer & \multicolumn{3}{c}{1 Dense layer} \\ Activation & \multicolumn{3}{c}{Sigmoid} \\ Loss function & \multicolumn{3}{c}{MSE} \\ Optimizer & \multicolumn{3}{c}{Adam} \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Model configuration. rate is uncertain since the BDI data is volatile, as indicated in Table 3: the swings the data takes relative to the variance are enormously increased in the interval from 25 to 75%. Table 4 shows that DEEP RNN, LSTM, and GRU roughly have a similar error rate. GRU slightly outperforms LSTM, while LSTM outperforms Deep RNN. This was due to GRU and LSTM having a cell memory gate to handle long-term dependency. It is known that GRU and LSTM have the same mechanism for effective tracking of long-term dependencies while mitigating the vanishing/exploding gradient problems. The LSTM uses more complex gates than GRU. Therefore, in this case, the LSTM model, relative to GRU, tended to over-fit more. The Figure 5 shows that DERN is the nearest result to the original data, and this is an average value among RNN, GRU, and LSTM, due to our having combined them into an ensemble using a weighted technique. Moreover, in its short-term prediction \begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Method**} & \multicolumn{3}{c}{**70–30\%**} & \multicolumn{3}{c}{**80–20\%**} & \multicolumn{3}{c}{**90–10\%**} \\  & **RMSE** & **MAE** & **MAPE** & **RMSE** & **MAE** & **MAPE** & **RMSE** & **MAE** & **MAPE** \\ \hline ARIMA & 47.46 & 33.34 & 3.29 & 37.69 & 28.99 & 3.24 & 39.75 & 31.84 & 3.19 \\ MLP & 5.99 & 4.56 & 0.47 & 18.84 & 16.11 & 2.58 & 27.22 & 27.20 & 2.94 \\ Deep RNN & 5.92 & 4.45 & 0.46 & 4.23 & 3.89 & 0.59 & **3.31** & 2.84 & 0.36 \\ LSTM & 5.55 & 4.43 & 0.55 & 5.36 & 4.31 & 0.70 & 10.27 & 8.84 & 0.99 \\ GRU & 4.75 & 3.74 & 0.47 & 3.62 & 3.39 & 0.50 & 3.95 & 3.57 & 0.44 \\ DERN & **2.53** & **2.13** & **0.44** & **1.98** & **1.58** & **0.45** & 3.52 & **2.99** & **0.31** \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Experimental results of one-step-ahead (short-term) prediction of BDI. Figure 4: Short-term or one-step-ahead prediction of BDI in testing phase. \begin{table}
\begin{tabular}{c c} \hline \hline
**Parameter** & **Value** \\ \hline Count & 4567.00 \\ Mean & 2417.19 \\ Standard Deviation & 2127.62 \\ Minimum & 290.00 \\
25\% & 993.00 \\
50\% & 1588.00 \\
75\% & 3020.00 \\ Maximum & 11,793.00 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Summary statistics of BDI data. of BDI, our approach outperforms the previous, Artificial Neural Network (ANN) model, for which the average of MAPE was never lower than five [5]. In the ensemble layer, we obtain the weight \(w_{P1}\), \(w_{P2}\), and \(w_{P3}\) of RNN, LSTM and GRU, respectively.