# EduChat: An AI-Based Chatbot for University-Related Information Using a Hybrid Approach
## 5 Intent Classifier Based on Improved Random Forest Algorithm (Machine Learning Module)
### The Improved Random Forest (IRF) Algorithm  
Random forest is a powerful machine learning algorithm introduced by [PERSON] and [PERSON] in 2001 (2001). It quickly became an important tool in building predictive models and analyzing data. random forest works by creating an ensemble of multiple random decision trees. Each decision tree is trained on a bootstrap sample, which is a subset of the original data. This helps reduce the risk of overfitting, a common issue in machine learning. When making predictions, random forest aggregates the results from all the decision trees to provide the final prediction. random forest has several advantages, including its ability to handle noisy and large datasets, its applicability to classification and prediction tasks, and its capability to estimate the importance of input variables. The equation for defining the mean square error (MSE) of a random forest is as follows. \[\text{MSE}=\frac{1}{N}{\sum_{k=0}^{n}}{n\choose k}(F_{i}-F_{i}).b^{2} \tag{1}\]In Equation (1), \(N\) represents the total count of unique data points, where \(F_{i}\) represents the predicted outcome provided by model \(Y_{i}\), and \(i\) corresponds to the specific data point in question. In the system, the IRF algorithm consists of 02 main phases: training and testing. **Training phase. ** During the training phase, the standard random forest algorithm is enhanced by introducing a cost function that relies on the misclassification rate of the samples. The misclassification rate of the samples is calculated by assessing the false negative (FN) and false positive (FP) values in the confusion matrix for each decision tree during the training process. \[cost(T)=(\text{FN}(T)+\text{FP}(T)/\mid size(T)\mid \tag{2}\]  
where FN(T) represents the count of false negatives in a tree T, FP(T) signifies the count of false positives in tree T, and \(\mid\) size(T)\(\mid\) denotes the number of samples in the training set of tree T.  
Subsequently, an error-based technique is devised to allocate weights to each tree. This error is computed in 02 components: the overall training error and the positive training error. The overall training error is computed by considering the total misclassification of samples within a tree. The overall positive error is computed based on the minority class samples in a tree. In detail, the error calculation is defined as:  
\[err\_ov(x_{i})= \begin{cases}1:if\ y(x_{i})\
eq class(x_{i})\\ 0:if\ y(x_{i})=class(x_{i})\end{cases}  \tag{3}\]  
\[err\_pos(x_{m})= \begin{cases}1:if\ y(x_{m})\
eq positive\\ 0:if\ y(x_{m})=positive\end{cases}  \tag{4}\]  
where \(\text{x}_{i}\) is a sample in a tree, \(\text{y(x}_{i})\) is the class of \(\text{x}_{i}\) assigned by the tree, and \(\text{x}_{m}\) is a minority class sample in the tree. If more than one tree exhibits the same overall training error, the tree with the lowest overall positive error is assigned a higher weight. The cost function reflects the predictive capability of the tree, while the weight formulation indicates the confidence level associated with a tree. The optimal tree is determined by selecting the one with both the highest predictive ability and the highest confidence value. The complete training process is outlined in the Algorithm 2. ```
Input:\(S\) is dataset, \(S_{min}\) is minority class set, \(S_{min}\) is majority class set, \(n\): number of trees. Output: Weighted trees' set, \(\{weight(T)\): \(T\) is a tree\(\}\) 1. Split the set \(S\) into _train_ and _test_. 2. By sampling with replacement, divide _train_ into \(n\) training sets, termed as \(T\). 3. For \(i=1\) to \(n\) do
4. (a) Split \(T\) into \(T_{min}\) and \(T_{min}\): \(T_{min}\subseteq S_{min}\), \(T_{min}\subseteq S_{min}\). (b) Train \(T\) with decision tree learning algorithm.