## 5 Modeling Details
### Advanced Preprocessing and Hybrid DI Approach  
Figure 2 shows the technical scheme of the proposed advanced preprocessing and hybrid deep learning approach, which includes collecting data from smart meters of a number of consumers, outliers detection, features clustering, normalization, supervised learning, and hybrid deep learning model. After careful decomposition, the data are prepared for supervised learning and split into different sub-datasets for training, validation, and testing. Then, the prediction model, which is hybrid deep learning CNN-BLSTM, learns and predicts load profiles of customers more accurately. If the prediction is not accurate enough, the model goes back to the outlier detection segment to remove further outliers; otherwise, the prediction is satisfying. In the following paragraphs, details about each segment are presented. The cost function of the preprocessing model is based on the internal clustering validation metric (RMSSTD index). At the stage of parameter optimization of the DBSCAN algorithm, the main objective is to find the minimum RMSSTD index values. When this condition is satisfied, optimal parameters of the DBSCAN algorithm and the optimal training data set of the forecasting model are obtained. Therefore, the prediction accuracy is enhanced by optimal feature clustering and optimal training datasets. Otherwise, further feature clustering is applied to the preprocessing stage. Considering the abovementioned electricity load data sets, household consumers lack consistent and cycled patterns due to the uncertainties of human behavior. Although commercial consumers tend to have more regular consumption patterns than residential con  
Figure 2: Proposed prediction model scheme. sumers, there are uncertainties resulting from the workload. By preprocessing modelings proposed in this article, the forecasting model and results can be improved. Another obstacle can be listed as the deficiencies in smart measurement infrastructure services and the communication disconnection between smart meters and systems. This phenomenon has been also observed in our dataset. The ideal data acquisition scenario and the actual situation for residential building smart meters are shown in green and orange colors in Figure 3, respectively. It can be concluded that almost half of the smart meters do not have continuous data over one year period. Moreover, the blue area is the only suitable range for comparison of different customers' consumption behavior. In this study, non-metric MDS was implemented to \(N\)-dimensional data for dimension reduction. The _metricsstress_ criterion is used to obtain 2-D subspace from \(N\)-dimensional data set. The _metricsstress_ criterion is the normalized squared stress obtained from the sum of the fourth powers of the distance matrix [21; 33]. A commercial building's MDS representation and daily load profiles are shown in Figure 4a,b respectively. Each point in Figure 4a represents a daily profile of a commercial building over a year period. After the dimension-reduction process, anomalous daily load profiles were detected using the BDE-DBSCAN algorithm and removed from the dataset. The RMSSTD index  
Figure 4: Non-classical multidimensional scaling representation (**a**) and daily consumption profiles (**b**) of a commercial building (customer ID: y3 and BDE-DBSCAN clustering result). Figure 3: The optimal range for comparison of different household’s daily electricity consumption. was chosen as the clustering validation metric, and the clustering process was repeated 30 times (epoch:30) for each individual. Due to the monotonicity issues of the RMSSTD index [34], two different strategies were implemented to overcome the issues; the first is using 2-D subspace for clustering and using a \(N\) dimension for calculating the RMSSTD validation index. The second strategy is keeping the minimum RMSSTD value, which consists of 100 iterations each, until the last iteration. Implementation of the second strategy revealed some interesting hidden patterns in our data set.