# Combining Advanced Feature-Selection Methods to Uncover Atypical Energy-Consumption Patterns
## 2 Methodology
### Recursive Feature Elimination with Multinomial Logistic Regression  
The RFE process aims to select the most important features by iteratively removing the least significant ones, thereby improving the efficiency of the feature-selection process. As explained in Algorithm 1, the process begins with constructing a model using all available features. Subsequently, each feature is assigned a weight based on its relevance in classifying the target variable. The feature with the lowest weight is then eliminated. The model is subsequently reconstructed, and the importance of each remaining feature is recalculated [23]. This iterative process continues until the remaining set of features meets a predefined performance threshold. \begin{table}
\begin{tabular}{c|c c c c|c} \hline \hline  & \multicolumn{4}{c}{Variable \(Y\)} \\  & \(Y_{1}\) & \(Y_{2}\) & \(\ldots\) & \(Y_{k}\) & \\ \hline \(X_{1}\) & \(n_{11}\) & \(n_{12}\) & \(\ldots\) & \(n_{1k}\) & \(x_{1}\) \\ \(X_{2}\) & \(n_{21}\) & \(n_{22}\) & \(\ldots\) & \(n_{2k}\) & \(x_{2}\) \\ \(\vdots\) & \(\vdots\) & \(\vdots\) & \(\ddots\) & \(\vdots\) & \(\vdots\) \\ \(X_{m}\) & \(n_{m1}\) & \(n_{m2}\) & \(\ldots\) & \(n_{mk}\) & \(x_{m}\) \\ \hline  & \(y_{1}\) & \(y_{2}\) & \(\ldots\) & \(y_{k}\) & \(n\) \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Contingency table for the CHI2 test. ```
0: Dataset \(X\) with feature dimension \(k\)
0:\(S^{*}\leftarrow\) Set of features providing the highest performance metric
1: Initialize: \(S\leftarrow\{1,\dots,k\}\)
2: Train a model on \(X\) using features in \(S\)
3: min_score \(\leftarrow\)\(S\) performance metric
4:for each feature \(f\) in \(S\)do
5: \(S^{\prime}\gets S-\{f\}\)
6: Train a model on \(X\) using features in \(S^{\prime}\)
7:if\(S^{\prime}\) performance metric \(<\) min_score then
8: \(f_{\text{min\_score}}\gets f\)
9: min_score \(\leftarrow\)\(S^{\prime}\) performance metric
10: end if
11:endfor
12: Remove \(f_{\text{min}}\) from \(S\)
```  
**Algorithm 1** Recursive Feature Elimination (RFE)  
Given that the response variable in this study, household energy usage, is categorized into three distinct classes (low-, medium- and high-load-consumption profiles), we employed a multinomial logistic regression (MLR) algorithm. This generalized linear model is particularly suitable for situations where the response variable encompasses more than two categories. The MLR algorithm utilizes a non-linear log transformation, which facilitates the calculation of the probability of occurrence for each class of the dependent variable [24]. Several assumptions were verified to ensure the applicability of the multinomial logistic regression:  
* Independence of irrelevant alternatives: The assumption that the odds of preferring one class over another are independent of the presence or absence of other alternatives. This was tested using the Hausman-McFadden test [25]. * No multicollinearity: Multicollinearity among predictors was checked using variance inflation factor (VIF) values, ensuring all were below the threshold of 5 [26]. * Linearity of logits: The relationship between continuous predictors and the logit of the outcome was confirmed to be linear [27]. * Large sample size: The sample size was sufficiently large to provide reliable estimates of the model parameters [28]. The probability that an observation \(Y_{i}\) belongs to a particular class \(c\in\{1,2,\dots,C\}\), given the predictor variables \(\mathbf{x}_{i}\), is denoted as \(P(Y_{i}=c|\mathbf{x}_{i})\) and is given by  
\[P(Y_{i}=c|\mathbf{x}_{i};\alpha_{1},\alpha_{2},\dots,\alpha_{C},\beta_{1}, \beta_{2},\dots,\beta_{C})=\frac{e^{\alpha_{c}+\beta_{c}^{T}\mathbf{x}_{i}}}{ \sum_{j=1}^{C}e^{\alpha_{j}+\beta_{j}^{T}\mathbf{x}_{i}}},\]  
where \(\alpha_{c}\) denotes the intercept for class \(c\), and \(\beta_{c}\) represents the vector of regression coefficients for class \(c\). To assess the performance of the classifier, a confusion matrix is employed to compare the predicted classes to the actual classes from the ground truth data.