# Ensemble Deep Learning Models for Forecasting Cryptocurrency Time-Series
## 5 Numerical Experiments
\begin{table}
\begin{tabular}{c l} \hline \hline
**Model** & \multicolumn{2}{c}{**Hyper-Parameters**} \\ \hline DTR & Criterion = ’mse’, \\  & max depth = unlimited, \\  & min samples split = 2. \\ \hline LR & No parameters specified. \\ \hline SVR & Kernel = RBF, \\  & Regularization parameter \(C=1.0\), \\  & gamma = \(10^{-1}\). \\ \hline \(k\)NN & Number of neighbors = 10, \\  & Euclidean distance. \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Parameter specification of state-of-art algorithms used as meta-learners. Summarizing, we evaluate the performance of the following ensemble models:  
* \"Averagingy\" and \"Averaging\({}_{11}\)\" stand for ensemble-averaging model utilizing 7 and 11 component learners, respectively. * \"Bagging\" and \"Bagging\({}_{11}\)\" stand for bagging ensemble model utilizing 7 and 11 component learners, respectively. * \"Stacking\({}_{7}^{(\text{DTR}),}\)\" and \"Stacking\({}_{11}^{(\text{DTR}),}\)\" stand for stacking ensemble model utilizing 7 and 11 component learners, respectively and DTR model as meta-learner. * \"Stacking\({}_{7}^{(\text{LR}),}\)\" and \"Stacking\({}_{11}^{(\text{LR}),}\)\" stand for stacking ensemble model utilizing 7 and 11 component learners, respectively and LR model as meta-learner. * \"Stacking\({}_{7}^{(\text{SVR}),}\)\" and \"Stacking\({}_{11}^{(\text{SVR}),}\)\" stand for stacking ensemble model utilizing 7 and 11 component learners, respectively and SVR as meta-learner. * \"Stacking\({}_{7}^{(k\text{NN}),}\)\" and \"Stacking\({}_{11}^{(k\text{NN}),}\)\" stand for stacking ensemble model utilizing 7 and 11 component learners, respectively and \(k\text{NN}\) as meta-learner. Tables 3 and 4 summarize the performance of all ensemble models using CNN-LSTM as base learner for \(m=4\) and \(m=9\), respectively. Stacking using LR as meta-learner exhibited the best regression performance, reporting the lowest RMSE score, for all cryptocurrencies. Notice that stacking\({}_{7}^{(\text{LR})}\) and Stacking\({}_{11}^{(\text{LR})}\) reported the same performance, which implies that the increment of component learners from 7 to 11 did not affect the regression performance of this ensemble algorithm. In contrast, stacking\({}_{7}^{(\text{LR})}\) exhibited better classification performance than Stacking\({}_{11}^{(\text{LR})}\), reporting higher accuracy, AUC and \(F_{1}\)-score. Additionally, the stacking ensemble reported the worst performance utilizing DTR and SVR as meta-learners among all ensemble models, also reporting worst performance than CNN-LSTM model; while the best classification performance was reported using \(k\text{NN}\) as meta-learner in almost all cases. The average and bagging ensemble reported slightly better regression performance, compared to the single model CNN-LSTM. In contrast, both ensembles presented the best classification performance, considerably outperforming all other forecasting models, regarding all datasets. Moreover, the bagging ensemble reported the highest accuracy, AUC and \(F_{1}\) in most cases, slightly outperforming the average-ensemble model. Finally, it is worth noticing that both the bagging and average-ensemble did not improve their performance when the number of component classifiers increased. for \(m=4\) while for \(m=9\) a slightly improvement in their performance was noticed.