# Comparing Approaches for Explaining DNN-Based Facial Expression Classifications
## 4 Experimental Setting
### DNN-Based System Development
#### 4.2.3 Feature Selection  
The complete geometric feature set consists of 40 features, but this set could contain redundant features. The goal of feature selection (FS) is to obtain a compact subset of features that describes the dataset, eliminating irrelevant or noisy features [44]. Redundant features are those that provide no extra information to the model (i.e. the feature is not needed for correct classification of the data points), but such features can cause noise and can thus introduce bias into the model. This affects how well the model generalizes and hence the performance on unseen data. On top of that, the smaller a feature set, the more efficient the computation time will be.  
In order to eliminate redundant features from the complete feature set, we used several feature selection algorithms to select a feature subset, trained and tuned models with that subset and compared validation set accuracy with the no-feature selection performance baseline.  
The first technique we tested is forward sequential feature selection (FSFS), implemented in Scikit-learn [45]. This is a wrapper method, which means it uses the model as a black box predictor and evaluates the performance on a certain feature subset. FSFS starts with an empty set of features and at each iteration it adds the feature that yields the highest performance gain. This continues until the specified amount of features is reached.  
Furthermore, we tested recursive feature elimination (RFE), proposed by [PERSON] et al. [44], also implemented in Scikit-learn. RFE is an iterative process consisting of three steps: train a model, compute the ranking of the features and finally eliminate the feature with the lowest ranking. This process continues until the set of features is reduced to a certain amount. In our case, we used a logistic regression model to estimate the feature ranking, since that can be taken directly from the model's coefficients.  
Another technique we tried was picking the \(n\) features with the highest global SHAP value (i.e., the most important features over the entire dataset) as the feature subset. For this, we first calculate the SHAP score for each feature by summing the absolute SHAP score for each feature for each data point across all seven classes. Then we rank the features according to this total score and take the top \(n\) features.  
The final technique consisted of hand-picking feature subsets based on domain knowledge. One can argue that features belonging to the left side of the face are more important to the half left model than for the half right model, and vice versa. Therefore, we constructed two feature subsets. Both contained the features that do not correspond to a particular side of the face (e.g., mouth width) and all features that correspond to either the left or the right side of the face. This last method is only tested on the half left model and the half right model, as the frontal model does not necessarily benefit from excluding features belonging to a particular side.  
For each feature selection method, we test subsets with 5 to 35 features in steps of 5. Ultimately, for each model we pick the feature subset that yields the highest validation set accuracy.