# Predicting Employee Attrition Using Machine Learning Approaches
## 4 Proposed Machine Learning Approaches  
The four advanced machine learning-based techniques, Support Vector Machine (SVM), Logistic Regression (LR), Decision Tree Classifier (DTC), and Extra Trees Classifier (ETC), were applied in this research study. The ETC is our proposed approach for predicting employee attrition.  
The SVM technique [29] is a family of supervised learning models that are based on support vectors used for classification. The SVM model [30] creates a best-fit decision boundary that divides input n-dimensional feature space data into target classes. The decision boundary is called a hyperplane. A hyperplane is an n-dimensional Euclidean space that divides the space into two disconnected subsets. The iterative manner was used by SVM to create the best fit hyperplane to minimize the error. The SVM selects the extreme vectors that are useful for creating the hyperplane. These extreme vectors are called support vectors. The separating hyperplane is expressed in Equation (1). Where \(w\) is the weight matrix, \(x\) is the input feature, and b is the biased values.  
\[\overrightarrow{w}.\overrightarrow{x}+b=0, \tag{1}\]  
The LR is a statistical supervised machine learning method used for classification problems. The LR technique [31] was used to describe the relationship between dependent and independent variables. The LR uses the concept of the sigmoid function. The probabilistic values of the sigmoid function lie between zero and one. The S-shaped logistic function is fit for prediction. The LR model is expressed in Equation (2). where the predicted class output is \(y\), the bias term is \(b_{0}\), and the coefficient for input \(x\) is \(b_{1}\).  
\[y=\frac{e^{(b_{0}+b_{1}*x)}}{\left(1+e^{(b_{0}+b_{1}*x)}\right)}, \tag{2}\]  
The DTC is a supervised machine learning technique utilized for classification problems. The DTC [32] is a tree structure representation where the target class labels represent on the leaf node, decision rules represent by branches, and the attributes are represented on the internal nodes. The motive of DTC [33] is to predict the target class by learning decision rules inferred from training data. The DTC is good to utilize because the decision-making  
Figure 7: The dataset resampling using SMOTE technique was applied: (a) describe the unbalanced target category; (b) describe the target category after balancing the dataset.  
rule is mimicked by human thinking ability. The best attributes are selected by Information Gain and Gini Index in DTC. The Gini Index is calculated as expressed in Equation (3).  
\[Gini\ Index=1-\sum_{j}{P_{j}}^{2}, \tag{3}\]  
The ETC is an extension of the ensemble learning method based on the construction of bagged decision trees [34]. The ETC concept is similar to the Random Forest; however, the difference in forest construction. The ETC aggregates the outcome of multiple de-correlated decision trees to predict the target class for the classification task. The ETC technique works by generating a large number of bagged decision tree samples from the training data. The decision rule is selected randomly. The majority voting is used for predictions from decision trees. The majority voting predictions are aggregated to yield the final prediction. The entropy calculated for ETC is expressed in Equation (4).  
\[Entropy(S)=\sum_{i=1}^{c}-p_{i}log_{2}(p_{i}), \tag{4}\]  
_Hyperparameter Tuning_  
The hyperparameter tuning was applied to find the best-fit parameters of applied machine learning models [35]. The hyperparameter configuration parameters were examined in Table 2. The hyperparameters were achieved by checking the model's outcome accuracy results iteratively on the dataset. The parameters on which model performance score are efficiently selected as their hyperparameters. The tuning proved very fruitful in our research study. Our applied machine learning models achieved good accuracy scores.