# Combining Advanced Feature-Selection Methods to Uncover Atypical Energy-Consumption Patterns
## 4 The Feature-Selection Process
### Chi-Square Test  
In the context of feature selection for a predictive model, the CHI2 test is used to evaluate the independence between each feature and the target variable [44]. Before analyzing the categorical variables, the five numerical features (BED, HAC, DHW, NPE, and U18) were categorized and the categories are presented in Table 4.  
The process involves calculating the \(p\)-value for each feature to determine its statistical significance in relation to the target variable. If the \(p\)-value for a feature is greater than 0.05 (at a 95% confidence level), we fail to reject the null hypothesis, indicating that the feature is independent of the target variable. Consequently, that feature is removed from further analysis. We then recalculate the \(p\)-values for the remaining features and continue this process until all features have \(p\)-values lower than 0.05. This iterative process ensures that only statistically significant features are retained. The implementation of the CHI2 test was done using the scikit-learn 1.3.0 library in Python 3.9.18 [21].  
During the initial test, the k-means and SOM algorithms identified the PET feature with the highest \(p\)-value. In k-means, the \(p\)-value was approximately 0.6 (Figure 2a), while in SOM, it was almost equal to 1 (Figure 2c). On the other hand, the AHC algorithm identified the ARE feature with the highest \(p\)-value, around 0.55 (Figure 2b). Therefore, these features were eliminated as their \(p\)-values exceeded the 0.05 threshold.  
\begin{table}
\begin{tabular}{l l l l l} \hline \hline
**Variable** & **Mean** & **Std Dev** & **Min** & **Median** & **Max** \\ \hline Number of bedrooms & 2.83 & 1.00 & 1.00 & 3.00 & 8.00 \\ Number of heating and air & 0.94 & 1.13 & 0.00 & 1.00 & 5.00 \\ conditioning units & 1.04 & 0.93 & 0.00 & 1.00 & 5.00 \\ Number of domestic hot water units & 3.16 & 1.23 & 1.00 & 3.00 & 7.00 \\ Number of people under 18 & 0.81 & 0.92 & 0.00 & 1.00 & 5.00 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Statistical analysis of numerical variables in the dataset.  
Figure 2: \(p\)-value of features in the CHI2 test. (**a**) K-means, (**b**) AHC, (**c**) SOM.  
In the second stage of analysis, the WPU feature was removed. The \(p\)-value was approximately 0.5, slightly less in k-means (Figure 3a) and a little more in SOM (Figure 3c). In the AHC analysis, the PET feature was removed due to its \(p\)-value being close to 0.5.  
After removing two features, the highest \(p\)-value continued to be above the 0.05 threshold (Figure 4). The value was closer to 0.25 in all cases: k-means and SOM with the ARE feature, and AHC with WPU.  
After removing eight features, all \(p\)-values were below the 0.05 threshold, signaling the end of the process, as shown in Figure 5.  
After conducting a recursive analysis, the features with a \(p\)-value higher than the threshold were eliminated until all remaining features had a \(p\)-value below the threshold. This confirmed that ARE, PET and WPU were independent. As a result, it was concluded that only the following features have a significant relationship with the target variables: HAC, INC, DHW, TYP, NPE, BED, COE and U18.  
Figure 4: \(p\)-value of features after removing two features in the CHI2 test. (**a**) K-means, (**b**) AHC, (**c**) SOM.  
Figure 5: \(p\)-value of features after removing eight features in the CHI2 test. (**a**) K-means, (**b**) AHC, (**c**) SOM.  
Figure 3: \(p\)-value of features after removing one feature in the CHI2 test. (**a**) K-means, (**b**) AHC, (**c**) SOM.