# Benchmarking of Load Forecasting Methods Using Residential Smart Meter Data
## 4 Conclusions  
The developed research revealed many interesting and useful insights. On one hand, despite the ANN models being more accurate and more flexible to be adapted to different consumers' patterns, it is clear that the three different forecasting approaches do not differ significantly in accuracy of estimating half-hourly electricity consumption records for the day-ahead, applied to different consumers. This implies that the quality of smart meter data used, the feature selection phase and each model's parametrization may be quite more determining to enhance the forecasting action, rather than the chosen forecasting  
Figure 14: ANN Dimensions’ Distribution. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Model** & **Min** & **Average** & **Max** \\ \hline MARS & 0.95 s & 10.68 s & 79.01 s \\ RF & 40.44 s & 83.71 s & 1743.81 s \\ ANN 1 & 2.30 s & 5.45 s & 11.54 s \\ \hline \hline \end{tabular} \({}^{\dagger}\)Training times associated with ANN models are related to the most accurate model out of 80 different simulations. (10 trials followed to a variable number of neurons—from 3 to 10).  
\end{table} 
Table 3: Training Time Analysis (in seconds). model itself. Regarding this, for the electricity supplier/distributor, the trade-off between the accuracy and the interpretability of each individual model must be considered. This conclusion is strengthened when a scale effect is involved, as several individual forecasting models should be trained and used for different end-users. On the other hand, with a thorough analysis of the trained models, it was possible to identify some relevant features that are transversal to the different approaches (mainly the electricity consumption records measured at similar hourly periods 2 and 7 days before). The strategy of including previous electricity consumption records at hourly periods \"adjacent\" to the similar hourly periods seems to be more justified in the case of ANN and not so relevant to MARS or RF models. The effects of including cyclic variables to distinguish the consumption pattern specific to each day of the week or even to each hourly period tend to be notorious and should not be avoided. Some experiments were exploited without these cyclic variables, leading to an overall degradation in forecasting performances. Further research should address the inclusion of exogenous variables in the models (including weather variables on a half-hourly basis or, at least, considering daily meteorological values associated with extreme conditions--e.g., minimum, and maximum values of temperature on a daily basis). To overcome the risk of overfitting in the RF model, different hyperparameters should be explored by eventually using a metaheuristic. Finally, rather than simply looking at the weights vector in the first ANN layer, a sensitivity analysis based on partial derivatives must be followed up to better evaluate features' relevance in ANNs. Conceptualisation, J.C.S. and H.B. ; methodology, J.C.S. ; software, J.C.S. ; validation, J.C.S. and H.B. ; formal analysis, J.C.S. and H.B. ; investigation, J.C.S. and H.B. ; resources, J.C.S. ; data curation, J.C.S. ; writing--original draft preparation, J.C.S. and H.B. ; writing--review and editing, J.C.S. and H.B. ; visualization, J.C.S. and H.B. ; project administration, J.C.S. ; funding acquisition, J.C.S. and H.B. All authors have read and agreed to the published version of the manuscript. This research was partially supported by the FCT--Portuguese Foundation for Science and Technology (project grant UIDB/00308/2020). The dataset was obtained from UK Power Networks, Low Carbon London Project, and it is available online at: [[https://data.london.gov.uk/dataset/smartmeter-energy-use-data-in-london-households](https://data.london.gov.uk/dataset/smartmeter-energy-use-data-in-london-households)]([https://data.london.gov.uk/dataset/smartmeter-energy-use-data-in-london-households](https://data.london.gov.uk/dataset/smartmeter-energy-use-data-in-london-households)) (accessed on 27 July 2021).