## 6 Experiment
### Result and Discussion
#### 6.2.2 Experience Different Models  
In the second part of the experiments, we compare the performance evaluation results of M-LSTM to those of other competitive models (LSTM, Bi-LSTM, LR, and SVM models) on three datasets in terms of three metrics. Two sets of performance metrics are presented; one set includes time label information (\(MAPE_{t}\), \(NRMSE_{t}\), \(R_{t}^{2}\)_score_), whereas the other set does not include time label information (\(MAPE\), \(NRMSE\), \(R^{2}\)_score_). Table 2 presents the performance evaluation results of M-LSTM and LSTM, Bi-LSTM, LR, and SVM models on three datasets in terms of three metrics. Figure 11 shows the \(MAPE\) error of models with and without labeled time. In general, the results indicate that models using labelled time information tend to perform better than those that do not use labelled time information in the same setting. For example, on _Dataset_S1, the four models get lower values, while the same models have higher error values. On the other hand, among the different models, the model M-LSTM using the labeled time information tends to perform the best overall, with the lowest \(MAPE_{t}\), \(NRMSE_{t}\), and the highest \(R_{t}^{2}\)_score_ in most cases. To summarize, using the proposed preprocessing method with time information tends to improve the performance \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline \multicolumn{2}{c}{} & \multicolumn{4}{c}{**With Labelled Time**} & \multicolumn{4}{c}{**Without Labelled Time**} \\ \hline
**Dataset** & **Model** & \(MAPE_{t}\) & \(NRMSE_{t}\) & \(R_{t}^{2}\)_Score_ & \(MAPE\) & \(NRMSE\) & \(R^{2}\)_Score_ \\ \hline \multirow{6}{*}{_DatasetS1_} & M-LSTM & **0.159** & **0.071** & **0.543** & 0.251 & 0.084 & 0.347 \\ \cline{2-8}  & LSTM & 0.260 & 0.090 & 0.252 & 0.270 & 0.097 & 0.135 \\ \cline{2-8}  & Bi-LSTM & 0.324 & 0.097 & 0.180 & 0.343 & 0.134 & 0.115 \\ \cline{2-8}  & Linear & \multirow{2}{*}{0.285} & \multirow{2}{*}{0.094} & \multirow{2}{*}{0.191} & \multirow{2}{*}{0.311} & \multirow{2}{*}{0.093} & \multirow{2}{*}{0.216} \\  & Regression & & & & & & \\ \cline{2-8}  & SVM & 0.239 & 0.074 & 0.490 & 0.258 & 0.081 & 0.310 \\ \hline \multirow{6}{*}{_DatasetS2_} & M-LSTM & **0.139** & **0.034** & **0.831** & 0.176 & 0.044 & 0.719 \\ \cline{2-8}  & LSTM & 0.385 & 0.075 & 0.156 & 0.495 & 0.091 & \(-\)0.230 \\ \cline{2-8}  & Bi-LSTM & 0.352 & 0.087 & 0.143 & 0.476 & 0.093 & \(-\)0.258 \\ \cline{2-8}  & Linear & \multirow{2}{*}{0.167} & \multirow{2}{*}{0.034} & \multirow{2}{*}{0.827} & \multirow{2}{*}{0.345} & \multirow{2}{*}{0.069} & \multirow{2}{*}{0.291} \\  & Regression & & & & & & \\ \cline{2-8} \cline{2-8}  & SVM & 0.208 & 0.058 & 0.248 & 0.388 & 0.078 & 0.122 \\ \hline \multirow{6}{*}{_DatasetS3_} & M-LSTM & **0.072** & **0.130** & **0.506** & 0.099 & 0.134 & 0.399 \\ \cline{2-8}  & LSTM & 0.184 & 0.146 & 0.378 & 0.431 & 0.144 & 0.395 \\ \cline{2-8}  & Bi-LSTM & 0.449 & 0.331 & \(-\)2.196 & 0.814 & 0.295 & \(-\)1.536 \\ \cline{2-8}  & Linear & \multirow{2}{*}{0.312} & \multirow{2}{*}{0.182} & \multirow{2}{*}{0.035} & \multirow{2}{*}{0.390} & \multirow{2}{*}{0.138} & \multirow{2}{*}{0.141} \\ Regression & & & & & & & \\ \cline{2-8} SVM & 0.192 & 0.136 & 0.404 & 0.397 & 0.147 & 0.341 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Comparison of M-LSTM with competitive models with and without labelled time. \(MAPE_{t}\), \(NRMSE_{t}\), \(R_{t}^{2}\)_score_. are denoted metrics for model trained with labelled time. \(MAPE\), \(NRMSE\), \(R^{2}\) score_ are denoted metrics for model trained without labelled time. \(MAPE\) and \(MAPE\) are rescale to the range from 0 to 1. Better values are marked in bold. Figure 10: Comparison of M-LSTM with labelled time (M-LSTM) with M-LSTM without labelled time, and other models in case of lack of training data (_DatasetS3_ in 2019). The time step is 7 days. of the models. The M-LSTM model using time label information performs the best in general. The labeled time field provides useful information for predicting energy consumption in peak and non-peak periods.