# Recursive Feature Elimination for Improving Learning Points on Hand-Sign Recognition
## 3 Methodology
### Feature Selection Method  
Recursive Feature Elimination (RFE) is implemented to reduce the number of features used to train the models. RFE is chosen since we can decide the number of features to be selected from the original features. RFE fits a learning model as a wrapper method and removes the less critical features [35]. Random Forest (RF) is the learning model chosen for this research. Random Forest uses Gini importance to calculate the feature importance. The feature importance of a binary tree decision tree can be denoted by Equation (3):  
\[ni_{j}=w_{j}C_{j}-w_{left(j)}C_{left(j)}-w_{right(j)}C_{right(j)} \tag{3}\]  
where \(ni_{j}\) is the importance of node \(j\); \(w_{j}\) is the weighted number of samples reaching node \(j\); \(C_{j}\) is the impurity value of node \(j\); _left (j)_ is the child node from left split on node \(j\); _right (j)_ is the child node from right split on node \(j\). Then the importance of each feature on a decision tree is based on Equation (4):  
\[fi_{i}=\frac{\sum_{j:node\ j\ splits\ on\ feature\ i}\pi i_{j}}{\sum_{k\in all\ nodes}\pi i_{j}} \tag{4}\]  
where \(fi_{i}\) is the importance of feature \(I\); \(ni_{j}\) is the importance of node \(j\). Figure 5: Hand landmarks detected by Mediapipe Hands. These values then can be normalized by dividing them by the sum of all feature importance values with Equation (5):  
\[norm\;fi_{i}=\frac{fi_{i}}{\sum_{j\in all\;features\;fi_{j}}} \tag{5}\]  
Finally, the feature importance of a feature in the random forest level is the average over all the trees as in Equation (6):  
\[RF\;fi_{i}=\frac{\sum_{j:all\;trees\;norm\;fi_{ij}}}{T} \tag{6}\]  
\(RF\;fi_{i}\) is the importance of a feature \(I\) in the random forest model; \(norm\;fi_{ij}\) is the normalized feature importance of feature \(I\) in tree \(j\); \(T\) is the total number of trees in the random forest. Before making the feature selection, the hand landmark data need to be converted from coordinates to distance between points because the feature selection methods currently available in Python cannot accept a 2D array as the input feature. We choose not to flatten the input into 1D because doing so will destroy the relationship between the \(x\) and \(y\) coordinate of each point and RFE will treat each \(x\) and \(y\) value of a point as a separate feature. In this research, we propose using a novel distance between the hand landmark to the hand palm centroid. The distance is calculated as a Euclidean distance from the hand landmarks to an anchor point \(Pa\) which is in the center of the palm. The point \(Pa\) is calculated as a centroid of a triangle formed by the coordinates \(P0\), \(P5\), and \(P17\). This coordinate can be represented by Equations (7) and (8):  
\[Pa_{x}=\frac{P0_{x}+P5_{x}+P17_{x}}{3} \tag{7}\]  
\[Pa_{y}=\frac{P0_{y}+P5_{y}+P17_{y}}{3} \tag{8}\]  
\(Pa_{x}\) is the \(x\) coordinate for \(Pa\); \(Pu_{x}\) is the \(x\) coordinate of the \(n\) landmark point; \(Pa_{y}\) is \(Pa\)'s \(y\) coordinate; and \(Pny\) is the \(y\) coordinate of the \(n\) landmark point. Euclidean distance between \(Pa\) and \(Pn\) is indicated in Equation (9):  
\[d=\sqrt{\left(x_{2}-x_{1}\right)^{2}+\left(y_{2}-y_{1}\right)^{2}} \tag{9}\]  
where (\(x_{1}\), \(y_{1}\)) is the coordinate of the first point, which is the anchor point; (\(x_{2}\), \(y_{2}\)) is the coordinate of the second point; and \(d\) is the Euclidean distance between the 2 points. Using RFE, the number of features is reduced from the original 21 features to 15 and 10 features. The algorithm of RFE can be described in Algorithm 1:  
RFE will take an input of the number of desired features and also the dataset containing the original number of features. Then it will build and fit a random forest model and calculate the feature importance. The next step is to remove the least important feature from the dataset. If there is still a feature in the dataset, it will recursively repeat this until there are no more features in the dataset.