# Prediction Interval Adjustment for Load-Forecasting using Machine Learning
## 4 Experiments and Results
### Results and Discussion  
To compare the proposed approach, the Autoregressive Integrated Moving Average (ARIMA) model and a persistence model are also evaluated. The ARIMA model is a classical time series forecasting method. This method depends on three parameters: \(p\) which stands for the number of autoregressive variables; \(q\) which refers to the number of moving average variables; and \(d\) which indicates the number of times the data needs to be _differentiate_ such that the time series is stationary. For experimental purposes, we estimated the ARIMA model using the process one described in [33]. The persistence model is often used to know if a forecast model provides better results than any trivial reference model [34]. First, we present the point-forecast MAE and RMSE. Whereas point-forecast MAE gives us a general idea of the precision of the forecast model, point-forecast RMSE penalizes large errors, so if the ANN models tend to return large error values the RMSE will be greatly separated from point-forecast MAE. In Figure 12, the point-forecast MAE and RMSE for the Persistence, ARIMA and the proposed model are shown. As we can observe in Figure 12, the persistence and the ARIMA models work better, for point forecasts, in the first 5 horizons in comparison to the proposed model. However, the proposed model point-forecast RMSE follows the same tendency as point-forecast MAE, so we can say that the errors are consistent along the horizons for the three experiments, unlike the ARIMA and the persistence model for which the errors are larger along the horizons. Although we can observe that errors are consistent along the horizons for the proposed model, we can also observe that point-forecast MAE and RMSE tends to be larger in the Weekend experiment for the three models. This behavior is expected as we suppose that human activities in the Weekend are less constant than Weekdays. Then we present the DSS and the PICP along with the probabilistic MAE and RMSE. These results are presented per horizon and for all the three experiments and the three models evaluated. In Figure 13, the result of the DDS and the PICP per horizon is shown. As we can observe in Figure 13, the DSS for the ARIMA and the persistence model is larger than the measurement of the proposed model for both Adjusted and Normal prediction intervals (larger than \(\sim\) 1000). Figure 12: Point-forecast MAE and RMSE for (from **top** to **bottom**) the persistence, ARIMA and the proposed model. Figure 13: DSS and PIPC corresponding to (from **top** to **bottom**) the persistence model, ARIMA, and the proposed model. values of DSS indicates lower quality of prediction intervals). Also, we can observe that the PIPC for the ARIMA and the persistence model is lower than the measurement of the proposed model for both Adjusted and Normal prediction intervals. For ARIMA and the proposed model, DSS and PICP per horizon of Adjusted and Normal prediction intervals are really close along the horizons for the three experiments, which indicates that for those models, the Adjusted and Normal prediction intervals are quite similar. Probabilistic MAE and RMSE provide another perspective of this result. In Figure 14, probabilistic MAE and RMSE for the persistence, ARIMA, and the proposed model is shown. As we can observe in Figure 14, for ARIMA and the persistence model the probabilistic MAE and RMSE are larger than the proposed model, and also that is increasing along the horizons. Although probabilistic MAE and RMSE for ARIMA and the persistence model are quite similar between Adjusted and Normal prediction intervals, probabilistic RMSE value is much separated from probabilistic MAE, which indicates that errors are very large sometimes. For the proposed model, probabilistic MAE indicates that Adjusted prediction intervals fail by less than 2.5 MW along the horizons, which is not significant if we compare it to the 1.3 MW the Normal prediction interval failure.