## 4 Experiments
### Low- and High-Variability Cells  
It may be arguably easy to perform a multi-step forecast on a time series which keeps repeating periodically through time, without changing trends and with almost no variability in the data. Logically, even a naive model which predicts the next sequence based only on the mean of past observations can perform fairly well. However, we know that this is not the case with mobile phone activity data where spikes and anomalies--mainly due to gatherings of people in a small space and frequent use of the mobile phone--are quite likely and in some cells even frequent. In this dynamic scenario, which is typical for a subset of cells in our dataset, the time series results difficult to predict. Therefore, we propose a further experiment for which we prepared two distinct datasets: a first one containing cells with low variability, as shown in Figure 10a, and a second one containing irregular, difficult to predict cells, with high variability, such as in Figure 10b. Hereafter we will refer to these two cases as low- and high-variability scenarios, respectively. To decide which cells belong to each category, we followed a method which is similar to the one used to describe the volatility of a financial asset return over time. In particular, for every cell we computed the percentage change \(\Delta\) of the observation at time \(t+1\) with respect to the one at time \(t\) as follows:  
\[\Delta_{y}=100\cdot\frac{y_{t+1}-y_{t}}{y_{t}} \tag{8}\]  
Thus, for each cell we compute the standard deviation of the percentage change \(\Delta_{y}\) and then show the distribution across the cells, as in Figure 18. From this analysis, we chose to consider as cells with high variability those having a standard deviation of their percentage change higher than 15 in value. In the same way we chose as cells with low variability the ones having a standard deviation smaller than 9. These two thresholds allow us to partition the dataset into two subsets, each containing almost the same number of cells. Then, we compute the three forecasting performance measures for every method, in a 24-steps forecasting use case for both low- and high-variability datasets. Figures 19 and 20 show the distribution of RMSE and MAPE respectively across all the cells in a low (left) and high (right) variability scenario. For these two performance measures, the top-performer methods are ARIMA and CNN-LSTM both in low and high variability context. As the graphics shows, they perform equally well though ARIMA is slightly better in particular regarding the RMSE performance measure. \begin{table}
\begin{tabular}{l c c c c c} \hline \hline \multirow{2}{*}{**Methods**} & \multicolumn{5}{c}{**MAXAE**} \\ \cline{2-6}  & **3 Steps** & **6 Steps** & **12 Steps** & **24 Steps** & **48 Steps** \\ \hline
**LSTM**. & 43.78 & 54.57 & 78.57 & 182.51 & 868.55 \\ \hline
**ETS** & 123.17 & 156.53 & 126.46 & 130.74 & 132.73 \\ \hline
**AR** & 40.59 & 65.84 & 109.73 & 116.31 & 124.63 \\ \hline
**CNN** & 52.56 & 47.09 & 57.48 & 68.84 & 78.74 \\ \hline
**MLP** & 40.24 & 49.23 & 61.18 & 72.53 & 84.87 \\ \hline
**Prophet** & 39.23 & 52.52 & 63.57 & 134.56 & 122.25 \\ \hline
**ARIMA** & 46.11 & 51.16 & 63.15 & 62.54 & 74.16 \\ \hline
**CNN-LSTM** & 15.74 & 22.49 & 24.36 & 32.61 & 65.96 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: The table above summarizes the increase for the MAXAE error for every forecasting step from 3 and 6 up to 24 and 48. For this kind of performance measure, deep learning methods and in particular the CNN-LSTM outperforms the ARIMA method. More in general deep learning methods show a better forecasting performance than statistical methods. The MAXAE, whose distribution is shown in Figure 21, is where we can find a substantial difference between ARIMA and CNN-LSTM and more in general between statistical and deep learning methods.