# Benchmarking of Load Forecasting Methods Using Residential Smart Meter Data
## 3 Results
### Error Metrics Description  
For our study, initial analysis on each method were based on the Mean Absolute Error (_MAE_) and on the Mean Squared Error (_MSE_). This is given as the simple formula:  
\[MAE=\frac{\sum_{i=1}^{N}\lvert Measured\ Energy\ Value_{i}-Predicted\ Energy\ Value_{i}\rvert}{N} \tag{2}\]  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Type of Data** & **Input Description 1** \\ \hline \multirow{7}{*}{Historical Data} & Electricity consumption measured at the same hourly period—8 days before (x\({}_{0}\)) \\ \cline{2-3}  & Electricity consumption measured 2 hourly periods before—7 days before (x\({}_{1}\)) \\  & Electricity consumption measured 1 hourly period before—7 days before (x\({}_{2}\)) \\  & Electricity consumption measured at the same hourly period—7 days before (x\({}_{3}\)) \\  & Electricity consumption measured 1 hourly period after—7 days before (x\({}_{3}\)) \\  & Electricity consumption measured 2 hourly periods after—7 days before (x\({}_{3}\)) \\  & Electricity consumption measured 1 hourly period before—6 days before (x\({}_{3}\)) \\  & Electricity consumption measured at the same hourly period —6 days before (x\({}_{3}\)) \\  & Electricity consumption measured 1 hourly period after—6 days before (x\({}_{3}\)) \\  & Electricity consumption measured 2 hourly periods after—6 days before (x\({}_{10}\)) \\ \cline{2-3}  & Electricity consumption measured 2 hourly periods before—2 days before (x\({}_{11}\)) \\  & Electricity consumption measured 1 hourly period before—2 days before (x\({}_{12}\)) \\  & Electricity consumption measured at the same hourly period—2 days before (x\({}_{13}\)) \\  & Electricity consumption measured 1 hourly period after—2 days before (x\({}_{14}\)) \\  & Electricity consumption measured 2 hourly periods after—2 days before(x\({}_{15}\)) \\ \hline \multirow{7}{*}{Cyclic variables} & Cyclic variable 1 based on the weekday (x\({}_{16}\)) \\  & (from 1—”Sunday” to 7—”Saturday”) \\ \cline{2-3}  & Cyclic variable 2 based on the hourly period (x\({}_{17}\)) \\ \cline{1-1}  & (from 1—”00:00” to 48—”11 h 30 p.m.”) \\ \hline \hline \end{tabular} \({}^{1}\) One day before was not considered in this case, because when forecasting the day-ahead (Day D), the energy records of the prior day are not yet completely available (for the 48 hourly periods).  
\end{table}
Table 2: Selected features to be integrated to characterize the models.  
\[MSE=\frac{\sum_{i=1}^{N}(\textit{Measured Energy Value}_{i}-\textit{Predicted Energy Value}_{i})^{2}}{N} \tag{3}\]  
\(N\) being the number of forecasted values. Since each model was tested separately before the comparison with another one, it was required to have a benchmark method to help grasp the quality of the performance. This also allowed to establish a scaling for the computed errors, avoiding having an error metric expressed in absolute values [5]. The most common error measure in forecasting is the Mean Absolute Percentage Error (MAPE), where the error is expressed as a percentage of the observed value, but this measurement is known to have undesired effects, such as non-symmetry and being affected by null values presented in the data series. Thus, new metrics, called the Mean Absolute Scaled Error (_MASE_) and the Mean Squared Scaled Error (_MSSE_), are then introduced and computed as:  
\[MASE=\frac{MAE}{MAE_{\textit{Native}}} \tag{4}\]  
\[MSSE=\frac{MSE}{MSE_{\textit{Native}}} \tag{5}\]  
_MASE_ and _MSSE_ above 1 means that the model being considered performed worse than the _Native_ approach.