# Student Performance Prediction in Mathematics Course Based on the Random Forest and Simulated Annealing

Authors: Shaohai Huang; Junjie Wei
Journal: Scientific Programming
 (2022)

---

## Abstract

Educational data mining is becoming a more and more popular research field in recent years, mainly with the help of cross research conducted by various disciplines, so as to solve various difficult problems in the teaching and education process. In this paper, we proposed a hybrid approach for student performance prediction. We collected the dataset, including 15 characteristics of students from three categories (individual basic information, individual education information, and individual behavior information). Based on the random forest (RF) and simulated annealing (SA) algorithms, we binary encode the relevant parameters (number of features, tree size, and tree decision weights) as the target variables for algorithm optimization, use the out-of-bag error as the optimization objective function, and then propose the IRFC (improved random forest classifier) algorithm in this paper. Compared with other mainstream improved random forest algorithms, the research results demonstrate that the proposed algorithm in this paper has higher generalization ability and smaller OOB error. This study provides a methodological reference for the prediction of student achievement and also makes a marginal contribution to student management work.


---

## Body

## 1. Introduction

At present, universities have accumulated a large amount of obvious data in the education of students, such as the basic information of students, their family situation, and their grades and scores in various subjects, as well as the specific information of students’ speech rate and correct rate in answering questions in the classroom and their attendance in the classroom. Obviously, these data in the field of education are constantly changing and will grow explosively with the development of education informatization, so how to extract useful information from these complex and tedious data for data analysis will have a good research value.It is not an easy task to use modern technology to process the data resources with huge data size education data and discover the hidden knowledge accurately. The emergence of data mining technology in recent years has provided powerful technical support for solving such challenges. The use of data mining technology to discover valuable data or relevant knowledge law information from a large amount of data and the analysis and mining results applied to the teaching management will have far-reaching significance to improve the quality of school teaching and education management level.Data parallelization processing has become one of the important issues of the current era. How to combine data mining work with data parallel processing mechanism for processing is also one of the major challenges for data mining work. The combination of data mining algorithm and data parallelism mechanism can make full use of the parallelism mechanism to accelerate the fast processing of data and also to dig the hidden rules or principles behind the massive data according to a deeper level so as to understand the nature of things more deeply.Educational data mining is becoming a more and more popular research field in recent years, mainly with the help of cross research conducted by various disciplines (such as computer science, education, and statistics), so as to solve various difficult problems in the teaching and education process [1–3]. In the context of the big data era, educational data mining research will take a new watershed. Educational data mining involves more working levels, including student performance prediction, teaching deficiency analysis, student adaptive learning ability analysis, automatic student performance discrimination, and other subdomains [4–7], and the scope of this paper will be limited to student performance prediction.The educational data mining for the specific direction of student achievement prediction consists of two stages. The first period was the last century, when researchers used data mining techniques for education only in a simple way, and the scope of application was narrow due to the limitation of the technology level at that time. The second period is the rapid development of the last two decades, with the use of Internet technology to trigger a change in educational technology, using a variety of data mining techniques to promote the development of educational informatization [8]. EDUCAUSE, the US Association for Information Technology in Higher Education, has proposed a definition of educational data mining in the new era of educational challenges as the ability to use data-based methods to predict student learning progress and outcomes and to act on them.In recent years, many researchers at home and abroad have used data mining technology to study and analyze curriculum score data. Ei-Halees used a data mining method to study students’ learning habits and put forward specific suggestions for improving students’ performance. Ayesha used the k-means clustering method to predict students’ course understanding level on the basis of students’ test score set, thus providing a data basis for the construction of final exam questions. Overall, course data mining research has attracted a large number of scholars to explore [9, 10].In this research, the random forest algorithm based on the decision tree algorithm is mainly used. The decision tree is a very efficient supervised learning algorithm with easy to interpret characteristics. The key to decision tree construction is to choose which attribute as the basis of classification, and the typical construction algorithms are ID3 [11], C4.5 [12], and CART [13], respectively.The random forest algorithm uses the bootstrap sampling method for M rounds of training and constructs M decision trees by classifiers (decision trees, SVM, logistic regression, etc.) to build a random forest and determine the final attribution classification of the data by voting results [14]. The random forest performs well on the dataset and does not easily fall into overfitting than neural network [15] while having good noise resistance. Random forests can handle very high dimensional (large feature size) data and do not have to make feature selection; they can handle both discrete and continuous data. Random forests are also highly adaptable to the dataset; the input dataset does not need to be normalized. Many studies have shown that combined classifiers are better at classification, which makes the random forest algorithm extremely important in both regression and classification. 2014 E Cernadas compared the actual results of different classifiers on different datasets and showed that the random forest algorithm was on average the strongest, getting first place on 9.9% of the datasets. Fernandez-Delgdo M et al. compared various 179-class classification algorithms on 121 UCI datasets, including excellent classification algorithms such as the random forest algorithm, decision tree algorithm, and SVM algorithm, and the experimental results confirmed that the random forest algorithm was optimal among all the algorithms [16].Although the rapid rise of data parallel processing platforms has enabled machine learning algorithms to be widely used, many machine learning algorithms are black-box models that cannot be adapted to certain special prediction research fields. As an important mining algorithm in the aspect of ML, the RF algorithm can detect data characteristics such as interactions and nonlinear relationships between data without any preconceptions while having low generalization errors in many practical application scenarios and problem solutions. More significantly, the random forest algorithm is interpretable and can be explained as the effect of certain importance measures, binary measures or multivariate dependencies, and other relevant factors.This paper focuses on the practical problem of student performance prediction and academic performance prediction and tries to find out the influencing factors and prediction models about student performance from them through the analysis of educational big data.In this paper, we proposed a hybrid approach for student performance prediction. We collected the dataset, including the characteristics of students from three categories. Based on the random forest (RF) and simulated annealing (SA) algorithm, we binary encode the relevant parameters (number of features, tree size, and tree decision weights) as the target variables for algorithm optimization, use the out-of-bag error as the optimization objective function, and then propose the IRFC (improved random forest classifier) algorithm in this paper. The results of this study will be useful for educational management, teaching participants, and individual students and will provide methodological support for improving student performance. In addition, the results of our research can be further extended, for example, to build an early warning identification system and early prevention and control system based on our research results or to personalize the curriculum by predicting student performance and to intervene in the teaching process in order to obtain an overall improvement of student performance.


## 2. Method

### 2.1. Random Forest

Compared with ANN, regression tree, and SVM, the random forest algorithm has been proven to be more stable and robust while also maintaining a high level of classification in scenarios with missing data [17].


### 2.2. Standard Random Forest

The standard RF algorithm is an ensemble of algorithms such as Bagging and CART, which mainly performs diverse combinations by using decision trees as the basic unit; the final classification result is output by voting for most single classifiers.To summarize, there are two parts of the RF classification; one is the random generation of input data for each single classifier by using the Bagging method; the second is the random selection of feature subsets to obtain the best classification features through information gain.In the following, the Bagging method and CART model of the RF are explained in detail, followed by a detailed description of the entire workflow of the random forest algorithm. Subsequently, this paper enhanced the classification results on the above foundation.


### 2.3. Bagging Method

The key point of the Bagging method is that the process of calculating any classification does not rely on any previous classification calculation process. In each rating calculation process, a part of data shall be selected for independent operation based on the original dataset, and the results of each classification shall be effectively aggregated to enhance the classification accuracy of the algorithm and avoid aggregation of unstable learning algorithms [18, 19].


### 2.4. CART Algorithms

In this study, we first explain the CART model in detail. The CART algorithm is a decision tree algorithm proposed by Breiman and Friedman. CART usually generates a binary decision tree through the Gini index; it uses a binary recursive model for binary tree construction; each division will divide the entire dataset into two sets, resulting in two branching subtrees. The Gini index is mostly employed to indicate the impurity of the dataset. Sample set D’s Gini index is defined as(1)GiniD=1−∑i=1mpi2,where pi denotes the probability that the data in D belongs to category Cj. If the binary division of D based on attribute A will be divided into two subsets, D1 and D2, the Gini index based on this division can be calculated as a weighted sum of the impurity of each partition:(2)GiniAD=D1DGiniD1+D2DGiniD2.By the above calculation, it is known that the reduction of impurity due to the binary division based on attributeA is(3)ΔGiniA=GiniD−GiniAD.During the execution of the CART algorithm, the attribute maximizes the impurity reduction.Out-of-bag (OOB) error calculation is often used as a common algorithm evaluation metric for classification performance assessment of random forest algorithms because it is an unbiased estimate, an alternative to the dataset cross-validation method, and a practical experience which represents that its measured values are generally of great nicety.The smaller the value of OOB error, the better the classification result of the RF, which is usually expressed as(4)OBB error=∑iKOBB errorik.


### 2.5. Random Forest Algorithm Flow

Combining the above Bagging method and CART algorithm, we will explain the workflow of the RF in detail. The detailed execution process of the RF is shown in Figure1, and the specific execution process of the RF is shown as follows:Symbol description: the original training dataset is represented by D, which is composed of M base attributes and a classification outcome attribute Y.Input: original training set D and decision tree size K.Output: classifier model and classification results.Process:Step 1: generate training subsets. Using the Bagging algorithm for the original training set D,N times repeated random sampling is performed to obtain a training subset of size N (where there may be duplicate data). The above process is repeated K (such as 10, 15, and 20) times to obtain the training subset {D1, D2, D3, …, Dx}. The feature selection is also performed for M base attributes and O (0 ≤ M, such as 1, 2, 3, …, M) attributes for analysis.Step 2: for each training subset Di (1 <i < K), the CART is employed to produce an unpruned dichotomous decision tree recursion. At each intermediate node of the decision tree, it needs to follow the following rules: instead of selecting the optimal segmentation features from all the features, a subset of features is constructed from a random selection of m features (m ≤ M), and then the features corresponding to the best segmentation form (with the largest Gini measure) are selected from the subset of features. CART trees need to continue the above process for node splitting. The corresponding decision tree hi (Di) is finally produced for each Di.Step 3: combine the decision trees produced in the last step to form an RF {h1 (D1), h2 (D2), …, hi (Di)} and test it based on the measured set of samplesX to get the corresponding results.Step 4: final results for test dataX are determined using the majority voting method and the weighted voting method for the K-tree classification results, and the classification error OOB error needs to be calculated. The weighted voting method is mainly for the results of each decision tree weighted statistics, Sc represents the total number of votes, and it can be calculated as(5)Sc=∑t=1KTc,xXWt,whereTc, x (X) takes the value of 1 or 0. If sample X results in class c after classification by decision tree, it takes the value of 1; otherwise, it takes the value of 0. After counting by the weighted voting method, the class with the most votes is selected as the final output classification result of sample X.Figure 1
Basic workflow of random forest algorithm.The algorithm ends.A point worth stating is that, throughout the operation of the algorithm, the individual variables, such as the number of resamplingN, the decision tree size K, feature selection value O, and decision weight Wt, remain independent of each other and do not have any dependency.


### 2.6. Simulated Annealing Algorithm

Metropolis first proposed the simulated annealing (SA) algorithm, and then SA was applied to combinatorial optimization algorithms by Kirkpatrick, with the main goal of solving the problems of overcoming initial value dependence [20, 21]. The calculation steps of the simulated annealing are as follows:Phase 1: generate a random initial solutionx0 such that xbest=x0, calculate the objective function value E(x0), and set k=0 and initial temperature t0=tmax.Phase 2: if the temperature reaches the stop condition then skip to phase 3, or else choosexbest randomly from the neighborhood Nv(xbest) of the current optimal solution xnew and calculate the associated objective function value E (xnew), from which the associated incremental value ΔE=Exnew−Exbest is derived. Update xbest by the following equation:(6)xbest=xnew,ΔE≤0xnew,ΔE>0xbest,other situationsande−ΔEtk>random0,1,Phase 3: settk+1=dtk; k=k+1. If the output value meets eventually the stop condition, the computation is terminated, or else return to phase 2 to continue the execution. Finally, the optimal solution xbest is output.As a general optimization algorithm, the simulated annealing algorithm has already been widely employed in many optimization field problems, such as in industrial shop layout optimization problems, transportation network optimization, supply chain optimization problems, atmospheric radiation equation solution problems, loan problem optimization solutions, and other application levels, which have good application effect, and in many cases can find the global optimal point.


### 2.7. Improved Algorithm

#### 2.7.1. Improvement Ideas

Although the original random forest algorithm has been able to achieve better operational results compared with other classification algorithms, there are still several problems in itself. First, the default parameters are not optimal choices, especially the weighted weight parameters, while the traditional grid search method for optimal parameter decisions is too time-consuming for practical applications. Second, the algorithm OOB error will be reduced further and used as algorithm objective function definition; the number of algorithms for feature selection is usually predetermined, which may be far from the exact value. The IRF proposed in our chapter is mostly for the above problems to correct.In recent years, many scholars have also incorporated intelligent algorithms into random forest algorithms to achieve good research results; for example, some researchers combined genetic algorithm and random forest algorithm for fault detection to reduce OOB outside the bag error [22]; a combination of particle algorithm and random forest algorithm is proposed for feature selection research and has good experimental results [23]; researchers incorporated AFSA into random forest algorithm for feature selection as well as parameter optimization [24], and the simulation experimental results prove that the algorithm OOB error is generally small and the generalization ability is relatively strong, which provides an effective guidance method for random forest algorithm feature selection and parameter estimation.In our study, the IRFC is proposed for parameter optimization, weight correction, and feature selection to finally achieve two classifications or multiple classifications. O value, the more commonly used value, includesM, 2M, and log2M+1, and it is found in the experimental data that when the value of M is relatively small, the algorithm classification effect is the best when the hyperparameter O chooses log2M+1, but it cannot obtain the optimal effect at any time. Many studies have started to use KNN methods, minimization of OOB errors, random combination of input variables, and other methods to carry out the optimal value solution of hyperparameter X, with relatively good classification results and with short computation time and small storage space compared with traditional methods.For the output sample classification results, the random forest algorithm will take the output results of most decision trees as the basis. In our study, we will give different weights to the decision trees to further improve the shortcomings of the traditional algorithm so as to improve the classification accuracy.


### 2.8. Improved Algorithm Flow

For the above parameter optimization, our study proposes a new method for integrated optimization based on the previous study, mainly by incorporating the SA into the RF execution process, and the parameters are combined and optimized. In our proposed method, the objective function is introduced:(7)fK∗,O∗,Attributei|i=1,2,...,M,wj|j=1,2,...,K,=arg minavg OBB error.Optimization variables are as follows: K, O,Attributei|i=1,2,...,M, wj|j=1,2,...,K, where K, О are real numbers, K takes values in the range [0, 500], and wj is an integer taking values in the range [0, 15]. Attribute is a 01 variable. Figure 2 introduces the optimization variables, which are represented using a binary code and consist of four parts.Figure 2
Optimization variables of binary encoding.On the basis of the above binary encoding of variables, the flow of our planned improved random forest algorithm IRFC is shown in Figure3, and its corresponding flow is shown as follows:Phase 1: set the initialization temperaturet=tmax, k = 1, generating a random initial solution X, and let Xbest=X0.Phase 2: calculateF=max1/f with RF algorithm.Phase 3:(1)
If the temperature is achieved the inner loop stopping condition, then skip to (2), or else choosexnew randomly from the neighborhood N (xbest ) of the current optimal solution xbest and calculate the associated objective function value f (xnew), from which the associated incremental value ΔF=fXnew−fXbest is derived. Update X best by the following equation:(8)Xbest=Xnew,ΔF<0Xnew,ΔF>0ande−ΔFtkXbest,other situations>random0,1.(2)
Settk+1=dtk; k=k+1; if the eventually stopping condition is satisfied, the computation is terminated and the xbest is output, or else return to (1).Phase 4: ifgen>maxgen, output the best X∗=K,O,Attribute,W. Or else, return to phase 2.Figure 3
The overall flowchart of the algorithm of the improved random forest algorithm IRFC.


## 2.1. Random Forest

Compared with ANN, regression tree, and SVM, the random forest algorithm has been proven to be more stable and robust while also maintaining a high level of classification in scenarios with missing data [17].


## 2.2. Standard Random Forest

The standard RF algorithm is an ensemble of algorithms such as Bagging and CART, which mainly performs diverse combinations by using decision trees as the basic unit; the final classification result is output by voting for most single classifiers.To summarize, there are two parts of the RF classification; one is the random generation of input data for each single classifier by using the Bagging method; the second is the random selection of feature subsets to obtain the best classification features through information gain.In the following, the Bagging method and CART model of the RF are explained in detail, followed by a detailed description of the entire workflow of the random forest algorithm. Subsequently, this paper enhanced the classification results on the above foundation.


## 2.3. Bagging Method

The key point of the Bagging method is that the process of calculating any classification does not rely on any previous classification calculation process. In each rating calculation process, a part of data shall be selected for independent operation based on the original dataset, and the results of each classification shall be effectively aggregated to enhance the classification accuracy of the algorithm and avoid aggregation of unstable learning algorithms [18, 19].


## 2.4. CART Algorithms

In this study, we first explain the CART model in detail. The CART algorithm is a decision tree algorithm proposed by Breiman and Friedman. CART usually generates a binary decision tree through the Gini index; it uses a binary recursive model for binary tree construction; each division will divide the entire dataset into two sets, resulting in two branching subtrees. The Gini index is mostly employed to indicate the impurity of the dataset. Sample set D’s Gini index is defined as(1)GiniD=1−∑i=1mpi2,where pi denotes the probability that the data in D belongs to category Cj. If the binary division of D based on attribute A will be divided into two subsets, D1 and D2, the Gini index based on this division can be calculated as a weighted sum of the impurity of each partition:(2)GiniAD=D1DGiniD1+D2DGiniD2.By the above calculation, it is known that the reduction of impurity due to the binary division based on attributeA is(3)ΔGiniA=GiniD−GiniAD.During the execution of the CART algorithm, the attribute maximizes the impurity reduction.Out-of-bag (OOB) error calculation is often used as a common algorithm evaluation metric for classification performance assessment of random forest algorithms because it is an unbiased estimate, an alternative to the dataset cross-validation method, and a practical experience which represents that its measured values are generally of great nicety.The smaller the value of OOB error, the better the classification result of the RF, which is usually expressed as(4)OBB error=∑iKOBB errorik.


## 2.5. Random Forest Algorithm Flow

Combining the above Bagging method and CART algorithm, we will explain the workflow of the RF in detail. The detailed execution process of the RF is shown in Figure1, and the specific execution process of the RF is shown as follows:Symbol description: the original training dataset is represented by D, which is composed of M base attributes and a classification outcome attribute Y.Input: original training set D and decision tree size K.Output: classifier model and classification results.Process:Step 1: generate training subsets. Using the Bagging algorithm for the original training set D,N times repeated random sampling is performed to obtain a training subset of size N (where there may be duplicate data). The above process is repeated K (such as 10, 15, and 20) times to obtain the training subset {D1, D2, D3, …, Dx}. The feature selection is also performed for M base attributes and O (0 ≤ M, such as 1, 2, 3, …, M) attributes for analysis.Step 2: for each training subset Di (1 <i < K), the CART is employed to produce an unpruned dichotomous decision tree recursion. At each intermediate node of the decision tree, it needs to follow the following rules: instead of selecting the optimal segmentation features from all the features, a subset of features is constructed from a random selection of m features (m ≤ M), and then the features corresponding to the best segmentation form (with the largest Gini measure) are selected from the subset of features. CART trees need to continue the above process for node splitting. The corresponding decision tree hi (Di) is finally produced for each Di.Step 3: combine the decision trees produced in the last step to form an RF {h1 (D1), h2 (D2), …, hi (Di)} and test it based on the measured set of samplesX to get the corresponding results.Step 4: final results for test dataX are determined using the majority voting method and the weighted voting method for the K-tree classification results, and the classification error OOB error needs to be calculated. The weighted voting method is mainly for the results of each decision tree weighted statistics, Sc represents the total number of votes, and it can be calculated as(5)Sc=∑t=1KTc,xXWt,whereTc, x (X) takes the value of 1 or 0. If sample X results in class c after classification by decision tree, it takes the value of 1; otherwise, it takes the value of 0. After counting by the weighted voting method, the class with the most votes is selected as the final output classification result of sample X.Figure 1
Basic workflow of random forest algorithm.The algorithm ends.A point worth stating is that, throughout the operation of the algorithm, the individual variables, such as the number of resamplingN, the decision tree size K, feature selection value O, and decision weight Wt, remain independent of each other and do not have any dependency.


## 2.6. Simulated Annealing Algorithm

Metropolis first proposed the simulated annealing (SA) algorithm, and then SA was applied to combinatorial optimization algorithms by Kirkpatrick, with the main goal of solving the problems of overcoming initial value dependence [20, 21]. The calculation steps of the simulated annealing are as follows:Phase 1: generate a random initial solutionx0 such that xbest=x0, calculate the objective function value E(x0), and set k=0 and initial temperature t0=tmax.Phase 2: if the temperature reaches the stop condition then skip to phase 3, or else choosexbest randomly from the neighborhood Nv(xbest) of the current optimal solution xnew and calculate the associated objective function value E (xnew), from which the associated incremental value ΔE=Exnew−Exbest is derived. Update xbest by the following equation:(6)xbest=xnew,ΔE≤0xnew,ΔE>0xbest,other situationsande−ΔEtk>random0,1,Phase 3: settk+1=dtk; k=k+1. If the output value meets eventually the stop condition, the computation is terminated, or else return to phase 2 to continue the execution. Finally, the optimal solution xbest is output.As a general optimization algorithm, the simulated annealing algorithm has already been widely employed in many optimization field problems, such as in industrial shop layout optimization problems, transportation network optimization, supply chain optimization problems, atmospheric radiation equation solution problems, loan problem optimization solutions, and other application levels, which have good application effect, and in many cases can find the global optimal point.


## 2.7. Improved Algorithm

### 2.7.1. Improvement Ideas

Although the original random forest algorithm has been able to achieve better operational results compared with other classification algorithms, there are still several problems in itself. First, the default parameters are not optimal choices, especially the weighted weight parameters, while the traditional grid search method for optimal parameter decisions is too time-consuming for practical applications. Second, the algorithm OOB error will be reduced further and used as algorithm objective function definition; the number of algorithms for feature selection is usually predetermined, which may be far from the exact value. The IRF proposed in our chapter is mostly for the above problems to correct.In recent years, many scholars have also incorporated intelligent algorithms into random forest algorithms to achieve good research results; for example, some researchers combined genetic algorithm and random forest algorithm for fault detection to reduce OOB outside the bag error [22]; a combination of particle algorithm and random forest algorithm is proposed for feature selection research and has good experimental results [23]; researchers incorporated AFSA into random forest algorithm for feature selection as well as parameter optimization [24], and the simulation experimental results prove that the algorithm OOB error is generally small and the generalization ability is relatively strong, which provides an effective guidance method for random forest algorithm feature selection and parameter estimation.In our study, the IRFC is proposed for parameter optimization, weight correction, and feature selection to finally achieve two classifications or multiple classifications. O value, the more commonly used value, includesM, 2M, and log2M+1, and it is found in the experimental data that when the value of M is relatively small, the algorithm classification effect is the best when the hyperparameter O chooses log2M+1, but it cannot obtain the optimal effect at any time. Many studies have started to use KNN methods, minimization of OOB errors, random combination of input variables, and other methods to carry out the optimal value solution of hyperparameter X, with relatively good classification results and with short computation time and small storage space compared with traditional methods.For the output sample classification results, the random forest algorithm will take the output results of most decision trees as the basis. In our study, we will give different weights to the decision trees to further improve the shortcomings of the traditional algorithm so as to improve the classification accuracy.


## 2.7.1. Improvement Ideas

Although the original random forest algorithm has been able to achieve better operational results compared with other classification algorithms, there are still several problems in itself. First, the default parameters are not optimal choices, especially the weighted weight parameters, while the traditional grid search method for optimal parameter decisions is too time-consuming for practical applications. Second, the algorithm OOB error will be reduced further and used as algorithm objective function definition; the number of algorithms for feature selection is usually predetermined, which may be far from the exact value. The IRF proposed in our chapter is mostly for the above problems to correct.In recent years, many scholars have also incorporated intelligent algorithms into random forest algorithms to achieve good research results; for example, some researchers combined genetic algorithm and random forest algorithm for fault detection to reduce OOB outside the bag error [22]; a combination of particle algorithm and random forest algorithm is proposed for feature selection research and has good experimental results [23]; researchers incorporated AFSA into random forest algorithm for feature selection as well as parameter optimization [24], and the simulation experimental results prove that the algorithm OOB error is generally small and the generalization ability is relatively strong, which provides an effective guidance method for random forest algorithm feature selection and parameter estimation.In our study, the IRFC is proposed for parameter optimization, weight correction, and feature selection to finally achieve two classifications or multiple classifications. O value, the more commonly used value, includesM, 2M, and log2M+1, and it is found in the experimental data that when the value of M is relatively small, the algorithm classification effect is the best when the hyperparameter O chooses log2M+1, but it cannot obtain the optimal effect at any time. Many studies have started to use KNN methods, minimization of OOB errors, random combination of input variables, and other methods to carry out the optimal value solution of hyperparameter X, with relatively good classification results and with short computation time and small storage space compared with traditional methods.For the output sample classification results, the random forest algorithm will take the output results of most decision trees as the basis. In our study, we will give different weights to the decision trees to further improve the shortcomings of the traditional algorithm so as to improve the classification accuracy.


## 2.8. Improved Algorithm Flow

For the above parameter optimization, our study proposes a new method for integrated optimization based on the previous study, mainly by incorporating the SA into the RF execution process, and the parameters are combined and optimized. In our proposed method, the objective function is introduced:(7)fK∗,O∗,Attributei|i=1,2,...,M,wj|j=1,2,...,K,=arg minavg OBB error.Optimization variables are as follows: K, O,Attributei|i=1,2,...,M, wj|j=1,2,...,K, where K, О are real numbers, K takes values in the range [0, 500], and wj is an integer taking values in the range [0, 15]. Attribute is a 01 variable. Figure 2 introduces the optimization variables, which are represented using a binary code and consist of four parts.Figure 2
Optimization variables of binary encoding.On the basis of the above binary encoding of variables, the flow of our planned improved random forest algorithm IRFC is shown in Figure3, and its corresponding flow is shown as follows:Phase 1: set the initialization temperaturet=tmax, k = 1, generating a random initial solution X, and let Xbest=X0.Phase 2: calculateF=max1/f with RF algorithm.Phase 3:(1)
If the temperature is achieved the inner loop stopping condition, then skip to (2), or else choosexnew randomly from the neighborhood N (xbest ) of the current optimal solution xbest and calculate the associated objective function value f (xnew), from which the associated incremental value ΔF=fXnew−fXbest is derived. Update X best by the following equation:(8)Xbest=Xnew,ΔF<0Xnew,ΔF>0ande−ΔFtkXbest,other situations>random0,1.(2)
Settk+1=dtk; k=k+1; if the eventually stopping condition is satisfied, the computation is terminated and the xbest is output, or else return to (1).Phase 4: ifgen>maxgen, output the best X∗=K,O,Attribute,W. Or else, return to phase 2.Figure 3
The overall flowchart of the algorithm of the improved random forest algorithm IRFC.


## 3. Experiment Design

### 3.1. Data Introduction

Students’ course performance depends on many elements, including their own learning characteristics, their own professional orientation, the degree of study in the course, the degree of review after class, the teacher of the class, and other factors. This course will incorporate the improved random forest algorithm into the student course prediction data through the IRFC to enhance the predicted results for the students’ achievement in each course; based on this, managers can pay attention to the factors that affect student performance and pay attention to these factors in the course teaching process [25, 26]. By continuously cycling this predict-practice-improve approach, we can enhance the teaching and learning reform efforts to make it more scientific and efficient and at the same time make students’ performance better and enhance their competitiveness in job opportunities after graduation.The dataset for conducting experiments in this section is the student performance dataset, which is student characteristics collected from our school’s advanced mathematics course. The dataset includes 15 characteristics of students, which consists of three parts: individual basic information, individual education information, and individual behavior information. The classification of student statistical features includes features such as birthplace, gender (male or female), place of entrance examination, and direct contact (father or mother) [27–30]. The classification of individual education information includes characteristics such as student grade level, teaching semester, teaching classroom, student major, faculty members, and student course subtest scores (in-class exam scores, midterm exam scores, and practical scores).The dataset was obtained by the authors from their class teaching, which involved a many dimensions collection of raw data. The work involved a long time frame and a relatively large amount of work. Subsequently, data preprocessing was performed on the raw data, including data cleaning, data discretization work, and data filtering. Due to the variability and diversity of the collected data expressions, data preprocessing was required: for data with characteristics such as “student course activity level,” the data were discretized and normalized, and we also cleaned up some of the abnormal data.Table1 introduces the feature information of the student performance, where the English name, Chinese meaning, and the result of each feature are explained in detail. Based on the historical records and student behavior subjects, we choose the first two years as a training set and trained the prediction model with the improved random forest algorithm, and then, we employed the the last years of data as a test set for performance prediction and made comparative analysis with the real performance. The size of the student achievement dataset (three years) is 2002.Table 1
Detailed information on the characteristics of the student performance.
No.NameFeatureIntroduction of the values1BirthplaceBirthdayNational province dictionary2GenderGender{0: male; 1: female }3Place of entrance examinationExam locationNational identity dictionary4Direct contactContact person{0: father; 1: mother; 2: other }5Student’s grade levelGrade{1, 2, 3, 4}6Teaching semesterSemeter{1, 2}7Teaching classroomClassroomClassroom dictionary8Student’s majorMajorProfessional dictionary9Faculty staffTeacherFaculty dictionary10Student course quiz resultsQuiz testAverage score/10011Students’ activity in classActivityNormalize to [0, 1]12Student absenceAbsenceNormalize to [0, 1]13Number of times students access teaching resources after classVisit resoucesNormalize to [0, 1]14Number of student discussions in the courseDiscussionNormalize to [0, 1]15Student course satisfactionSatisfactionNormalize to [0, 1]


### 3.2. Feature Normalization

First, we normalized the data features. The feature normalization method aims to reduce the numerical difference in the range of values taken by different features and reduces the impact of the training model on the units of measure. The feature normalization mostly employs linear transform, log transform, or tan transform to normalize the initial data, thus transforming it into a smaller public space.About the student performance dataset, the feature variables that require feature programming include student activity in class, student absenteeism, and student course satisfaction features, while other feature variables are calculated using raw values, such as birthplace using coded values and gender using the Boolean values coded as 0 or 1. For the student absenteeism count feature, the feature programming method is mainly a mnemonic linear transformation using the maximum-minimum normalization method. In the following, we choose the most representative example of student classroom activity for a complex feature normalization illustration.For the characteristics of students’ classroom activity, the number of times users raise their hands to answer questions, the number of times they communicate in class, and the degree of personal concentration in class are collected for comprehensive evaluation. In the process of implementation, the discrete data are first normalized to [0, 1] based on the log function normalization method, and each factor is assigned a corresponding weight coefficient to obtain the student classroom activity data. The expression of the student classroom activity level is(9)KTScore=Wjs∗log10JSxlog10maxJS+Wjl∗log10JLxlog10maxJL+Wzz∗ZZFactor.Wjs, Wjl, and Wzz denote the weighting factors of the three factors, which are set by default at 0.4, 0.3, and 0.3; ZZFactor denotes the teacher’s subjective evaluation score of students’ concentration; the range of it is 0-1.


### 3.3. Classification Algorithm Evaluation Indicator

After proposing an improved random forest model, an overall evaluation of the performance of the whole model is needed. The more common performance evaluation metrics include accuracy, precision, recall, F-value, ROC curve, and AUC value.First, we make some assumptions about the evaluation criteria: TP (True Positive) represents the correctly classified positive class, FP (False Positive) indicates the incorrectly classified negative class, FN (False Negative) denotes the incorrectly classified positive class, and TN (True Negative) is on behalf of the correctly classified negative class.N=TP+FN+FP+TN denotes the total amount of sample data.Accuracy, which represents the overall classification accuracy inside the total sample data, is calculated as follows:(10)accuracy=TP+TNN.Accuracy is the most important basic index among all evaluation indexes, which directly evaluates the efficiency of the model. The higher the value, the better the classification effect.Recall is mainly the classification accuracy of all sample data for positive classes, which means the proportion of correctly classified positive samples occupying all correctly classified samples, and the corresponding calculation formula is as follows:(11)recall=TPTP+TN.Precision mainly indicates the proportion of correctly classified positive samples to all predicted positive samples and is calculated as follows:(12)precision=TPTP+FP.The F-value is a comprehensive performance metric used to evaluate the imbalance of the dataset, mainly based on the combination of precision and recall, whereβ is used to indicate the importance of precision rate and recall rate. Only when both precision and recall are high, the F-value is higher and also indicates better classification results.(13)F=1+β2∗precision∗recallβ2∗precision+recall.ROC (Receiver Operating Characteristic) is an important criterion for judging the performance of classification models and is well suited for evaluating imbalance. When the ROC is closer to the upper left corner, it indicates better classification results. The area under the ROC is called the AUC (Area Under Curve); AUC can also be used to indicate the criterion for the superiority of the classifier; the higher the AUC, the better the classification results.


## 3.1. Data Introduction

Students’ course performance depends on many elements, including their own learning characteristics, their own professional orientation, the degree of study in the course, the degree of review after class, the teacher of the class, and other factors. This course will incorporate the improved random forest algorithm into the student course prediction data through the IRFC to enhance the predicted results for the students’ achievement in each course; based on this, managers can pay attention to the factors that affect student performance and pay attention to these factors in the course teaching process [25, 26]. By continuously cycling this predict-practice-improve approach, we can enhance the teaching and learning reform efforts to make it more scientific and efficient and at the same time make students’ performance better and enhance their competitiveness in job opportunities after graduation.The dataset for conducting experiments in this section is the student performance dataset, which is student characteristics collected from our school’s advanced mathematics course. The dataset includes 15 characteristics of students, which consists of three parts: individual basic information, individual education information, and individual behavior information. The classification of student statistical features includes features such as birthplace, gender (male or female), place of entrance examination, and direct contact (father or mother) [27–30]. The classification of individual education information includes characteristics such as student grade level, teaching semester, teaching classroom, student major, faculty members, and student course subtest scores (in-class exam scores, midterm exam scores, and practical scores).The dataset was obtained by the authors from their class teaching, which involved a many dimensions collection of raw data. The work involved a long time frame and a relatively large amount of work. Subsequently, data preprocessing was performed on the raw data, including data cleaning, data discretization work, and data filtering. Due to the variability and diversity of the collected data expressions, data preprocessing was required: for data with characteristics such as “student course activity level,” the data were discretized and normalized, and we also cleaned up some of the abnormal data.Table1 introduces the feature information of the student performance, where the English name, Chinese meaning, and the result of each feature are explained in detail. Based on the historical records and student behavior subjects, we choose the first two years as a training set and trained the prediction model with the improved random forest algorithm, and then, we employed the the last years of data as a test set for performance prediction and made comparative analysis with the real performance. The size of the student achievement dataset (three years) is 2002.Table 1
Detailed information on the characteristics of the student performance.
No.NameFeatureIntroduction of the values1BirthplaceBirthdayNational province dictionary2GenderGender{0: male; 1: female }3Place of entrance examinationExam locationNational identity dictionary4Direct contactContact person{0: father; 1: mother; 2: other }5Student’s grade levelGrade{1, 2, 3, 4}6Teaching semesterSemeter{1, 2}7Teaching classroomClassroomClassroom dictionary8Student’s majorMajorProfessional dictionary9Faculty staffTeacherFaculty dictionary10Student course quiz resultsQuiz testAverage score/10011Students’ activity in classActivityNormalize to [0, 1]12Student absenceAbsenceNormalize to [0, 1]13Number of times students access teaching resources after classVisit resoucesNormalize to [0, 1]14Number of student discussions in the courseDiscussionNormalize to [0, 1]15Student course satisfactionSatisfactionNormalize to [0, 1]


## 3.2. Feature Normalization

First, we normalized the data features. The feature normalization method aims to reduce the numerical difference in the range of values taken by different features and reduces the impact of the training model on the units of measure. The feature normalization mostly employs linear transform, log transform, or tan transform to normalize the initial data, thus transforming it into a smaller public space.About the student performance dataset, the feature variables that require feature programming include student activity in class, student absenteeism, and student course satisfaction features, while other feature variables are calculated using raw values, such as birthplace using coded values and gender using the Boolean values coded as 0 or 1. For the student absenteeism count feature, the feature programming method is mainly a mnemonic linear transformation using the maximum-minimum normalization method. In the following, we choose the most representative example of student classroom activity for a complex feature normalization illustration.For the characteristics of students’ classroom activity, the number of times users raise their hands to answer questions, the number of times they communicate in class, and the degree of personal concentration in class are collected for comprehensive evaluation. In the process of implementation, the discrete data are first normalized to [0, 1] based on the log function normalization method, and each factor is assigned a corresponding weight coefficient to obtain the student classroom activity data. The expression of the student classroom activity level is(9)KTScore=Wjs∗log10JSxlog10maxJS+Wjl∗log10JLxlog10maxJL+Wzz∗ZZFactor.Wjs, Wjl, and Wzz denote the weighting factors of the three factors, which are set by default at 0.4, 0.3, and 0.3; ZZFactor denotes the teacher’s subjective evaluation score of students’ concentration; the range of it is 0-1.


## 3.3. Classification Algorithm Evaluation Indicator

After proposing an improved random forest model, an overall evaluation of the performance of the whole model is needed. The more common performance evaluation metrics include accuracy, precision, recall, F-value, ROC curve, and AUC value.First, we make some assumptions about the evaluation criteria: TP (True Positive) represents the correctly classified positive class, FP (False Positive) indicates the incorrectly classified negative class, FN (False Negative) denotes the incorrectly classified positive class, and TN (True Negative) is on behalf of the correctly classified negative class.N=TP+FN+FP+TN denotes the total amount of sample data.Accuracy, which represents the overall classification accuracy inside the total sample data, is calculated as follows:(10)accuracy=TP+TNN.Accuracy is the most important basic index among all evaluation indexes, which directly evaluates the efficiency of the model. The higher the value, the better the classification effect.Recall is mainly the classification accuracy of all sample data for positive classes, which means the proportion of correctly classified positive samples occupying all correctly classified samples, and the corresponding calculation formula is as follows:(11)recall=TPTP+TN.Precision mainly indicates the proportion of correctly classified positive samples to all predicted positive samples and is calculated as follows:(12)precision=TPTP+FP.The F-value is a comprehensive performance metric used to evaluate the imbalance of the dataset, mainly based on the combination of precision and recall, whereβ is used to indicate the importance of precision rate and recall rate. Only when both precision and recall are high, the F-value is higher and also indicates better classification results.(13)F=1+β2∗precision∗recallβ2∗precision+recall.ROC (Receiver Operating Characteristic) is an important criterion for judging the performance of classification models and is well suited for evaluating imbalance. When the ROC is closer to the upper left corner, it indicates better classification results. The area under the ROC is called the AUC (Area Under Curve); AUC can also be used to indicate the criterion for the superiority of the classifier; the higher the AUC, the better the classification results.


## 4. Experiment Evaluation

### 4.1. Comparison of Traditional Algorithm Results

Table2 and Figure 4 show our comparative classification using a variety of basic data mining algorithms in an attempt to analyze the differences between the various data mining methods. Our comparison results indicate that the RF has a slight advantage over other data algorithms on the student feature dataset but is not significantly different overall.Table 2
Comparison of classification results of student grades dataset on each base algorithm.
LRNBSVMGRNNRFTP231214223216230FN1532132215FP2824423440TN8488808677Figure 4
The result of different algorithms of student grades.


### 4.2. Algorithm Basic Evaluation

we can find that in every index level, the improved random forest algorithm has obtained almost the minimum OOB error, which shows the proposed algorithm has strong generalization ability and higher classification accuracy. Table3 and Figure 5.Table 3
Comparison of classification results of student grades on different random forest algorithms.
RF_1RF_2RF_3RF_4GRNNIRFCN100100100100—339K11445—9Num (attribute)14141414—12Avg (W)8888—6.7Figure 5
The result of different random forest algorithms of student grades.


## 4.1. Comparison of Traditional Algorithm Results

Table2 and Figure 4 show our comparative classification using a variety of basic data mining algorithms in an attempt to analyze the differences between the various data mining methods. Our comparison results indicate that the RF has a slight advantage over other data algorithms on the student feature dataset but is not significantly different overall.Table 2
Comparison of classification results of student grades dataset on each base algorithm.
LRNBSVMGRNNRFTP231214223216230FN1532132215FP2824423440TN8488808677Figure 4
The result of different algorithms of student grades.


## 4.2. Algorithm Basic Evaluation

we can find that in every index level, the improved random forest algorithm has obtained almost the minimum OOB error, which shows the proposed algorithm has strong generalization ability and higher classification accuracy. Table3 and Figure 5.Table 3
Comparison of classification results of student grades on different random forest algorithms.
RF_1RF_2RF_3RF_4GRNNIRFCN100100100100—339K11445—9Num (attribute)14141414—12Avg (W)8888—6.7Figure 5
The result of different random forest algorithms of student grades.


## 5. Conclusion

In this study, based on random forest, a new method, IRFC, is proposed, which is proven to have a high classification accuracy in order to predict students’ performance in university courses. To enhance the execution efficiency of the method, the parallelization of the improved random forest algorithm is subsequently carried out on a big data platform to shorten the execution time of the algorithm. The main work of this paper is shown in the following: the hybrid algorithm IRFC, which combines a simulated annealing algorithm with RF, is proposed to optimize the performance of RF comprehensively. This paper analyzes the current research status of the random forest algorithm, binary encodes the relevant parameters (number of features, tree size, tree decision weights) as the target variables for algorithm optimization, uses the out-of-bag error as the optimization objective function, and then proposes the IRFC algorithm in this paper. Compared with other mainstream improved random forest algorithms for comparison, our results demonstrate that the IRFC algorithm has higher generalization ability with smaller OOB error.This paper also focuses on the parameter impact analysis of the improved RF algorithm on the student performance dataset, and the validation results confirm that the application of SA provides auxiliary effects on the effective performance of our model, which will help to carry out teaching curriculum reform. In this paper, we have completed the design and experiments of the improved random forest algorithm and fully verified the effectiveness and reliability of the improved algorithm in this paper. However, there are some shortcomings in our work; for example, the parallelization strategy is not the most fine-grained parallel working model. We will improve this in future work.



---

*Source: 9340434-2022-03-23.xml*