## 3 Research Methodology
### Classification Algorithms
#### 3.5.1 Decision Trees  
DTs are tools that can help analyze and interpret a user's decisions and provide recommendations on their possible consequences. Linear Regression is a widely used and basic method for calculating the relationship between a variable and one or more independent ones [28; 29; 30; 31; 32; 33].  
DTs are a widely used technique in machine learning for performing regression and classification tasks. They recursively divide a dataset into constituent parts according to the most salient attributes [28; 29; 30; 31; 32; 33].  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Algorithm** & **Strengths** & **Weaknesses** & **Best Use Cases** \\ \hline \multirow{4}{*}{Random Forest} & Reduces overfitting. & \multirow{2}{*}{Computationally strong. Less interpretable.} & \multirow{2}{*}{Complex datasets with noise.} \\  & Manages & & \\  & high-dimensional & & \\  & data well. & & \\  & Provides feature & & \\  & importance. & & \\ \hline \multirow{4}{*}{Decision Trees} & Simple to interpret. & \multirow{2}{*}{Prone to overfitting.} & \multirow{2}{*}{Classification tasks, especially where model interpretability is key.} \\  & Handles categorical & & \\  & and numerical data. & & \\ \hline \multirow{4}{*}{Linear Regression} & Easy to interpret. & \multirow{2}{*}{Assumes linearity.} & \multirow{2}{*}{Regression problems with a clear linear relationship.} \\  & Efficient for & & \\  & small datasets. & & \\ \hline \hline \end{tabular}
\end{table}
Table 3: Algorithm performance comparison.  
DTs' structure is composed of various internal nodes and branches that represent the various attributes of the data. These nodes are used to determine the best classification or regression model for the given dataset. The most common algorithms used in DT inductions are C4.5, ID3, and CART. These algorithms use different approaches to choose the best classification or regression model [28; 29; 30; 31; 32; 33].  
The splitting criterion for DTs is usually based on the information gained, which considers the amount of information that a feature provides about a class.  
The value of the information that is gained is expressed gradually. The Shannon function's mathematical expression is shown as follows:  
\[I(P(u_{1}),\ldots,\ P(u_{n}))=\sum_{i=1}^{n}-P(u_{i})\ log_{2}P(u_{i}) \tag{1}\]  
where the information gain that is obtained by splitting a dataset is referred to as \(I(P(u_{1})\),..., \(P(u_{n}))\). The value of \(P(u)\) is dependent on the attributes of the data that are split. While the \(P(u_{i})\) refers to the probability of the answer that is given by the split (\(u_{i}\)) [28; 29; 30; 31; 32; 33].  
\[Remainder\ (A)=\sum_{i=1}^{n}\ \frac{p_{i}+n_{i}}{p+n}\ \ I\bigg{(}\ \frac{p_{i}}{p_{i}+n_{i}},\ \frac{n_{i}}{p_{i}+n_{i}}\bigg{)} \tag{2}\]  
\[Gain\ (A)=I\bigg{(}\ \frac{p_{i}}{p_{i}+n_{i}},\ \frac{n_{i}}{p_{i}+n_{i}} \bigg{)}-Remainder\ (A) \tag{3}\]  
\[Gini\ (A)=1-\sum_{i=1}^{n}\ p_{i}^{2} \tag{4}\]  
One of the main disadvantages of DTs is that they are prone to overfitting, which can occur when a tree grows too big and collects noise in the collected data. To minimize this issue, pruning techniques are usually used. These can be performed either during the building phase or post-pruning. Although DTs are generally interpretable and simple, they can also be affected by slight variations in data [28; 29; 30; 31; 32; 33].