## 3 Implementation of the Decision Tree Algorithm
#### Measures Used in Decision Trees  
Algorithms to construct the decision tree, when selecting the division for each node, solve the problem of maximization. It is based on searching all possible attribute values in order to find the best division, i.e., the highest value of the diversity measure. Usually, the criteria for diversity measures are based on entropy and information gain or Gini coefficient, among others.  
In 1948, the American mathematician [PERSON] introduced the entropy concept related to thermodynamics' second principle into communication systems [85]. It determines the direction of spontaneous (self-responsive) processes in an isolated thermodynamic system. Entropy is a measure of the degree of system disorder and energy dispersion [86]. In terms of communication systems, it has been applied in the area of analysis of transmitted information. Thus, from this perspective, entropy means the average amount of information per single message transmitted from an information source. In order to measure the degree of randomness of a given set of events, the concept of statistical entropy was introduced. Assuming that \(Z\) is a set of separable events \(\left\{A_{1},\ldots,A_{n}\right\}\) comprehensively covering the elementary event space, where the probability \(P(A_{1},\ldots,A_{n})\) is 1, the equation can determine the entropy of the \(Z\) set:  
\[H(Z)\ =\ -\sum\
olimits_{i=1}^{n}P(A_{i})log_{Z}P(A_{i}). \tag{1}\]  
In general, entropy can also be used to measure the degree of randomness of the \(X\) variable. Then entropy of a random \(X\) variable of a discrete type is described by the formula:  
\[H(X)\ =\ -\sum\
olimits_{i=1}^{n}P_{i}log_{Z}P_{i}, \tag{2}\]  
where \(P_{i}=P(X=x_{i})\) denotes the probability that a random variable \(X\) takes the value \(x_{i}\), and \(n\) is the number of values of a random variable \(X\). [PERSON]'s entropy, therefore, determines the average amount of information related to the probability distribution of a random variable \(X\). The measure of entropy defined by [PERSON] has been applied in many fields of science, such as statistics and computer science.  
Classification trees use the measure of entropy as heuristics to select an attribute. Given the assumption that entropy determines uncertainty, even chaos, a higher value of entropy in a given set means a higher disturbance. Figure 2 shows that entropy (uncertainty) is greatest when the probability is 0.5. This means that entropy is maximum when there is the same number of objects from different class attributes. On the other hand, entropy is lowest when the probability is 0 or 1 (there is no uncertainty or high probability), i.e., entropy is minimal if the node is blank, i.e., it is homogeneous from the point of view of diversity for a given attribute. Ultimately, a minimum entropy for the tree is desirable, i.e., the nodes' classes are blank or uniform.  
Therefore, by dividing the tree, the attributes that achieve the most significant reduction in entropy are selected in the classification tree. This reduction (or change) is measured by the information increment [78]:  
\[IG(S,A)=H(S)-\sum\
olimits_{v}\frac{|S_{v}|}{|S|}H(S_{v}), \tag{3}\]  
where \(A\) determines the attribute with a given value, \(H\) is the entropy of the data set, \(S\) is the data set, and \(S_{v}\) is the data set reduced by vectors with a different value of the \(A\) attribute.