## 4 Proposed Approach
### Algorithm Selection  
The discussion in Section 4.1 above highlights that the DT algorithm is a strong candidate for the underlying study. However, the question is which variant of DT is the most appropriate choice? The literature has identified several variants of the decision tree. Some well-known algorithms are ID 3 (Iterative Dichotomiser 3) ([PERSON], 1993), C4.5 (which is a successor of ID3), and CART (classification and regression tree) (K  
information gain, gain ratio, or Gini index, depending on the algorithm used. All of these feature selection algorithms fall under the category of filter models. Among ID3, C4.5, and CART, the advantage that CART has is that it is not biased towards multivalued variables like ID3 or towards imbalanced trees like C4.5. The ID3 algorithm does not work on continuous data. Therefore, a discretization layer has to be added to it. As such, the CART algorithm holds a higher potential of showing better performance than ID3 and C4.5, as discussed in detail by [PERSON] et al. [66]. MARS is generally a good DT algorithm that makes a continuous function, unlike CART, but one drawback is its slower throughput, which can be detrimental in high-speed networks [65]. Additionally, MARS has been shown to be less accurate than CART in some cases [67]. A limitation of CHAID is that the approach performs pre-pruning, which can lead to less accuracy than CART [68; 69]. CART uses the _Gini index_ to calculate the relevance of features. This feature selection model creates a binary tree using probabilities of the occurrence of a value of a feature against other features or combinations of other features, using the following formula:  
\[Gini\ Index(f)=1-\sum_{i=1}^{n}(p_{i}|f)^{2} \tag{5}\]  
where \(n\) corresponds to the number of classes and \(f\) is the feature in question belonging to the input. The resultant calculation gives a quantifiable value to judge how much a particular feature is related to the classification. In this study, the simple CART algorithm is evaluated first. Then, a small modification is done to the CART algorithm where features are added until error starts increasing. In this modified CART algorithm, firstly, a separate DT is made for each feature individually, and the error of each tree is stored. This step is performed more than once so that averages  
Figure 3: Example of decision tree construction process. can be considered instead of relying on the results of a single iteration over the features. Then, the average errors are sorted. This also sorts the features in order of the accuracy provided by each feature. Afterwards, features are iteratively added one by one, and a DT is made for each aggregate, stopping when the error of the DT starts to increase. This has the potential to result in a better DT, as now there is a rationality behind including a feature other than its order of appearance in the dataset. The index of features that gives the minimum error has a meaning now. Finally, another algorithm is added that checks the change in accuracy after adding a feature to be considered by the CART algorithm. As a result, features that increase performance or have no effect on it are retained, and features that result in a decrease in performance are dropped following a hill climbing approach. Hill climbing is a technique that generally starts with an arbitrary solution to a multidimensional problem; then, iteratively, incremental changes are made to the solution, until a better solution is achieved [70]. The problem of feature selection based on model performance is similar to a graph search [71] in that we are trying to find the sequence and relevance of features that lead to the highest accuracy. This poses two options: either go for a slow, optimal, deterministic graph search algorithm, such as A* search; or perform a faster, simpler, but somewhat less optimal greedy search. Eventually, greedy search was chosen because it is faster [72].