# Predicting Student Learning Effectiveness in Higher Education Based on Big Data Analysis

Authors: Xin Yang; Jian Ge
Journal: Mobile Information Systems
 (2022)

---

## Abstract

With the global higher education entering the era of “quality is king,” the perception and acquisition of learning experience and the evaluation of learning quality and its effect have attracted more and more attention. However, at present, most of the academic evaluations of university courses are based on final examinations, supplemented by appropriate amount of regular tests, and little attention is paid to the development and quality of students’ learning ability in the learning process, which inevitably causes academic evaluations to be equivalent to assessments and deviates from the original purpose of academic evaluations to promote continuous improvement of teaching and learning quality. On this basis, this study uses the big data analysis method to predict the teaching effect of college students, in order to improve the existing teaching problems, grasp the teaching essence, and construct a relatively perfect curriculum evaluation system combined with the course of “teaching effect guidance + teaching action,” so as to further improve the academic evaluation system and improve the teaching quality. This will provide a reference for further improving the academic evaluation system and improving the teaching quality.


---

## Body

## 1. Introduction

The traditional evaluation method is basically based on two separate lines of teaching and learning, which inevitably results in “good teaching and learning are not necessarily ideal, and good learning is not necessarily the result of teaching;” learners’ learning is often limited to “what is taught and what is learned” and “what is tested and what is learned.” “The problem of “learning, thinking, and doing are not integrated” is bound to arise. Therefore, by building a teaching-learning community and implementing learning effectiveness evaluation oriented to students’ ability development, we can overcome the unscientific evaluation orientation of “test-only, score-only, intellectual education-only” and promote the continuous improvement of higher education’s concept, classroom, mode, and evaluation. It is an important engine to realize “promoting learning by evaluation, teaching by evaluation, and teaching by teaching.”As the quality evaluation reform of higher education in China continues to deepen and the structural contradiction of college graduates becomes more and more prominent, the whole society pays more and more attention to the efficiency of using resources and the output of students’ learning outcomes. The evaluation of students’ learning outcome output is actually the evaluation of students’ learning quality and learning effectiveness. Its connotation means that education and teaching activities should aim at students’ development and adopt quantifiable, measurable, and assessable methods to evaluate and value judgment on students’ learning process, learning quality, and output results. The evaluation is not simply an assessment of the degree of knowledge mastery, but in three dimensions based on the training objectives, graduation requirements, and teaching objectives of the courses taken. It includes both explicit education and teaching activities and implicit education and should fully reflect the whole dimension, process, and elements of talent cultivation quality. Its evaluation results contain two levels of final learning outcomes and stage learning outcomes.The final learning outcome is the learning outcome that students should achieve when they graduate. The first level is the basic quality, skills, and knowledge that students should generally attain through their undergraduate studies to ensure their successful graduation, which is the “bottom line;” the second level is the quality, skills, and knowledge that students attain after completing their studies through personalized learning on the basis of maximizing their personal potential. It covers the first level of learning outcomes. It can be seen that the final learning outcome is a comprehensive evaluation of the quality of students’ abilities developed through an undergraduate study, and it is also a cumulative evaluation of the knowledge, skills, and abilities acquired by students in each academic period and course, that is, the accumulation of the evaluation of the stage learning outcomes.The design of the learning effectiveness evaluation system must be conducive to promoting students’ meaningful learning, that is, to make students feel changes in certain aspects through the learning of the course, especially to allow each student to continue and expand the experiences and gains gained. Therefore, when designing learning effectiveness evaluation indexes, we should not only stop at the knowledge dimension of “understanding + memorization” but also set multiple dimensions such as ability, personality, and value; we should not only achieve the bottom line for all students but also guide students to higher level of personalized needs, so as to truly achieve the goal, and we should also set multiple dimensions in terms of ability and value.


## 2. Related Work

As we all know, the syllabus of any course has to set the teaching objectives of the course, not the learning objectives. At present, when setting the teaching objectives, there are often very vague descriptions, making it difficult for students to have a real sense of experience and gain. In the design of teaching, the focus is on “what to teach,” but not on “how students learn well” and “what is the quality of the learning outcomes,” and little attention is paid to whether the course can provide students with effective learning. These problems arise because the concept of “student-centeredness” is not fully implemented and enforced [1]. Therefore, it is necessary to return the teaching objectives of the curriculum to the learning objectives and to design them precisely with the learning situation, so as to improve the quality of learning and the reliability and validity of the evaluation of learning effectiveness.The current first-class curriculum proposes the construction standard of “one degree of gender” (i.e., high order, innovation, and challenge), which means that the evaluation method of the curriculum should not be limited to the knowledge dimension of “memorization + understanding” but should also be evaluated from the dimensions of ability, personality, and value [2]. The evaluation should be conducted in multiple dimensions, such as competence, personality, and value [3]. In Table 1, for engineering majors, according to the twelve general standards of engineering education accreditation, the learning objectives of undergraduate courses are comprehensively sorted out from five categories: discipline-specific knowledge and skills, higher-order thinking ability, humanistic literacy and values, professional literacy and ethics, and personality development and lifelong learning [4]. In the actual setting, we should make specific details according to the support and contribution of the courses to the graduation requirements of majors [5]. However, they must reflect multidimensionality, progressiveness, high order, innovation, and challenge [6].Table 1
Common evaluation methods for each type of learning objectives.
Category of learning objectivesEvaluation methodKnowledge and skills under discipline exclusiveOrdinary learning performance, stage assessment, final examination, and practical training assessmentHigher-order thinking skillsKnowledge network, mind mapping, case study, problem exploration, and project practiceHumanistic literacy and valuesFree discussion, simulation, role play, and real-life practiceProfessionalism and ethicsIdentifying facts and opinions, engineering cognition, project practice, and practice reflection and evaluationPersonality development and lifelong learningCooperative learning, cooperative practice, project management and communication, and self-learning and evaluationIn addition, when considering the progressiveness of course objectives, we should refer to Bloom’s principle of classifying educational objectives, presenting gradients and distinctions. Figure1 shows the mapping relationship between Bloom’s objective levels and knowledge construction to competence development [7, 8]. It needs to design more innovative and challenging teaching activities, such as flipped classroom, nonstandard answer exams, problem-oriented case teaching, experiential learning, and cooperative learning, guide students to analyze the essence, compare and choose solutions, synthesize and judge, expand and apply, and improve and optimize, and set up criteria that students can understand and teachers can operate, measure, and evaluate in the activities, so as to avoid simple and sloppy teaching design and lack of credibility and validity of evaluation [9, 10].Figure 1
Mapping relationship between Bloom’s goal hierarchy and knowledge construction to competence development.Based on the premise of the design of the student learning effectiveness evaluation system is to do a good analysis of the learning situation because it is the starting point for a good evaluation and the basis for implementation and is also the implementation of student-centered teaching based on the specific initiatives [11, 12]. The analysis of learning situation should not only analyze the existing situation but also analyze the subsequent needs and not only to figure out the knowledge base but also to figure out the learning ability and learning habits already possessed, as shown in Figure 2.Figure 2
Content of learning analysis.On the basis of obtaining the learning analysis data, the evaluation system is constructed according to six aspects, such as objectives, resources, methods, processes, standards, and results, to stimulate students’ professional aspirations and devote to their ability development [13, 14]. When designing the indexes, we should design the learning objectives around the benefit of student learning and learning effect improvement, carry out a series of work on learning priorities, learning method suggestions, content reconstruction, and assessment content and standards, and organically integrate the learning process and learning results to form a learning effectiveness evaluation system with the characteristics of “result-oriented + action learning” [15, 16]. The specific framework is shown in Figure 3; the steps are shown in the flowchart, for example, 1 represents step 1.Figure 3
Framework of the learning effectiveness evaluation system based on “Outcome Orientation + Action Learning.”In the framework of the “Result-oriented + Action Learning” learning effectiveness evaluation system, by creating a teaching-learning community, teachers and students evaluate and promote each other, taking result-oriented evaluation as the main line and integrating value-added evaluation and evaluation of learning behaviors [17–20].


## 3. Methods

### 3.1. Indicator Analysis of College Student Learning Effectiveness Evaluation

In order to realize the prediction of differentiated student learning effectiveness based on BDA, it is necessary to first establish the index analysis model of college student learning effectiveness evaluation and adopt the big data analysis method for college student learning effectiveness analysis, and the combined structure model of college student learning effectiveness evaluation is shown in Figure4.Figure 4
Combinatorial structure model of college students’ learning effectiveness evaluation.According to Figure4, the prediction system design of college students’ learning effectiveness is carried out under the B/S structure system, and the multidimensional structure analysis of college students’ learning effectiveness is carried out under the analysis expert system model with the fused scheduling method, and the large data distribution set of learning effectiveness evaluation is(1)Yk=yk1,yk2,…,ykj,…,ykJk=1,2,…,N.In equation (1), ykj denotes the characteristic quantity of the regression distribution of college students’ learning effectiveness, and N is the data length; the fusion of the lower college students’ learning effectiveness degree based on the correlation between different indicators, the combination of regression analysis and test analysis methods, the differential analysis of college students, the establishment of the robustness analysis model, the discrete dataset xt of college students’ learning effectiveness distribution, the introduction of robustness evaluation factors, and the data of college students’ learning effectiveness evaluation are obtained as(2)zt=xt+iyt=ateiθt.In equation (2), zt denotes the data component of college student learning effectiveness, yt denotes the level of growth degree, i denotes the robustness evaluation factor, and the control variables for predicting college student learning effectiveness under the establishment constraint, and the big data fusion model of college student learning effectiveness is obtained as(3)maxFX=F1X,F2X,…,FnXs.t·gjX≤0j=1,2,…,phkX=0k=1,2,…,p.In equation (3), j denotes the college student learning effectiveness factor, and the big data analysis model is constructed to obtain the reconstructed iterative formula for predicting the learning effectiveness of lower college students as(4)xik+1=1−ωxik+ωanibi−∑j=1i−1aijxjk+1−∑j=i+1naijxjk,i=1,2,…,n,k=1,2,…,n.In equation (4), ω denotes the level of learning input and bi,ai denote the specific differentiated values, from which a statistical analysis model of college students’ learning effectiveness is constructed and a multiple regression model is used to test and analyze the established multiple regression model, and the learning effectiveness prediction assessment is carried out by combining the method of panel data search for excellence.The association rule set of college students’ learning effectiveness is expressed byG through the Fourier transform decomposition results, and according to the linear relationship distribution, under the condition of significance correlation, the learning effectiveness of college students is obtained. The simplified mathematical model of college student learning effectiveness prediction is described by the following equation.(5)G1=b11an+b12a2+…b1nan,G2=b21a1+b22a2+…b2nan,………,Gn=bn1a1+bn2a2+…+bnnan.In equation (5), a,b have strong correlation, indicating the deviation limit and oscillation level of the lower college student learning effectiveness, and the lower corner markers indicate the sequence of each principal component in the college student learning effectiveness, and the optimal classification set of college student learning effectiveness is described as(6)fx=sgn∑j=1lαj∗yjKx,xj+b∗,x∈Rn.In  equation  (6), b∗=yi−∑j=1lyjαjKxj,xi,i∈i0αi∗uxi. It is possible to obtain the characteristic distribution matrix for the evaluation of the learning effectiveness of students in the following universities.(7)B=b11b12⋯b1nb21b22⋯b2n⋯⋯⋯bn1bn2⋯bnn.The fuzzy degree of learning effectiveness of college students and the adaptive search model is used to reconstruct the distributed feature sequence, and the output isS=x1,y1,ux1,…,xl,yl,uxl, where xj∈Rn,uxj∈−1,1,σ≤uxj≤1, σ are the model parameters, uxj is the lower college student learning effectiveness discrimination degree, and the linear correlation factor of learning effectiveness evaluation is xj,yj,uxj. Output the affiliation of yj  = 1 (positive class) or yj  = -1 (negative class) j=1,…,l. According to the above analysis, fuzzy decision making and feature reorganization of learning effectiveness evaluation of college students are realized, and learning effectiveness prediction is performed.Chaotic neuron is the basic unit of the chaotic neural network. The structure of a single chaotic neuron is shown in Figure5, in which the function f⋅ is the activation function of the neuron and (1) is the dynamic equation of the neuron [21–23].(8)xit+1=f∑j=1MWij∑r=0tk′hjxjt−r+∑j=1NVij∑i=0tkrIjt−r−α∑r=0tkrgixit−r−θi,fxit+1=11+exp−xit+1/ε.Figure 5
Chaotic neuron model.The structure of the chaotic neural network is shown in Figure6. The feedback and hidden layers of the network use logistic and linear functions as transfer functions, respectively, to make it have chaotic characteristics, and the chaotic characteristics exhibited by the neurons can make the network have better approximation ability. The transfer functions of each layer of the network are as follows.(9)a1k=logisticIW1,1p+LW1,1a1k−1+b1,a2k=purelinLW2,1a1k+b2.Figure 6
Structure of the chaotic neural network.


## 3.1. Indicator Analysis of College Student Learning Effectiveness Evaluation

In order to realize the prediction of differentiated student learning effectiveness based on BDA, it is necessary to first establish the index analysis model of college student learning effectiveness evaluation and adopt the big data analysis method for college student learning effectiveness analysis, and the combined structure model of college student learning effectiveness evaluation is shown in Figure4.Figure 4
Combinatorial structure model of college students’ learning effectiveness evaluation.According to Figure4, the prediction system design of college students’ learning effectiveness is carried out under the B/S structure system, and the multidimensional structure analysis of college students’ learning effectiveness is carried out under the analysis expert system model with the fused scheduling method, and the large data distribution set of learning effectiveness evaluation is(1)Yk=yk1,yk2,…,ykj,…,ykJk=1,2,…,N.In equation (1), ykj denotes the characteristic quantity of the regression distribution of college students’ learning effectiveness, and N is the data length; the fusion of the lower college students’ learning effectiveness degree based on the correlation between different indicators, the combination of regression analysis and test analysis methods, the differential analysis of college students, the establishment of the robustness analysis model, the discrete dataset xt of college students’ learning effectiveness distribution, the introduction of robustness evaluation factors, and the data of college students’ learning effectiveness evaluation are obtained as(2)zt=xt+iyt=ateiθt.In equation (2), zt denotes the data component of college student learning effectiveness, yt denotes the level of growth degree, i denotes the robustness evaluation factor, and the control variables for predicting college student learning effectiveness under the establishment constraint, and the big data fusion model of college student learning effectiveness is obtained as(3)maxFX=F1X,F2X,…,FnXs.t·gjX≤0j=1,2,…,phkX=0k=1,2,…,p.In equation (3), j denotes the college student learning effectiveness factor, and the big data analysis model is constructed to obtain the reconstructed iterative formula for predicting the learning effectiveness of lower college students as(4)xik+1=1−ωxik+ωanibi−∑j=1i−1aijxjk+1−∑j=i+1naijxjk,i=1,2,…,n,k=1,2,…,n.In equation (4), ω denotes the level of learning input and bi,ai denote the specific differentiated values, from which a statistical analysis model of college students’ learning effectiveness is constructed and a multiple regression model is used to test and analyze the established multiple regression model, and the learning effectiveness prediction assessment is carried out by combining the method of panel data search for excellence.The association rule set of college students’ learning effectiveness is expressed byG through the Fourier transform decomposition results, and according to the linear relationship distribution, under the condition of significance correlation, the learning effectiveness of college students is obtained. The simplified mathematical model of college student learning effectiveness prediction is described by the following equation.(5)G1=b11an+b12a2+…b1nan,G2=b21a1+b22a2+…b2nan,………,Gn=bn1a1+bn2a2+…+bnnan.In equation (5), a,b have strong correlation, indicating the deviation limit and oscillation level of the lower college student learning effectiveness, and the lower corner markers indicate the sequence of each principal component in the college student learning effectiveness, and the optimal classification set of college student learning effectiveness is described as(6)fx=sgn∑j=1lαj∗yjKx,xj+b∗,x∈Rn.In  equation  (6), b∗=yi−∑j=1lyjαjKxj,xi,i∈i0αi∗uxi. It is possible to obtain the characteristic distribution matrix for the evaluation of the learning effectiveness of students in the following universities.(7)B=b11b12⋯b1nb21b22⋯b2n⋯⋯⋯bn1bn2⋯bnn.The fuzzy degree of learning effectiveness of college students and the adaptive search model is used to reconstruct the distributed feature sequence, and the output isS=x1,y1,ux1,…,xl,yl,uxl, where xj∈Rn,uxj∈−1,1,σ≤uxj≤1, σ are the model parameters, uxj is the lower college student learning effectiveness discrimination degree, and the linear correlation factor of learning effectiveness evaluation is xj,yj,uxj. Output the affiliation of yj  = 1 (positive class) or yj  = -1 (negative class) j=1,…,l. According to the above analysis, fuzzy decision making and feature reorganization of learning effectiveness evaluation of college students are realized, and learning effectiveness prediction is performed.Chaotic neuron is the basic unit of the chaotic neural network. The structure of a single chaotic neuron is shown in Figure5, in which the function f⋅ is the activation function of the neuron and (1) is the dynamic equation of the neuron [21–23].(8)xit+1=f∑j=1MWij∑r=0tk′hjxjt−r+∑j=1NVij∑i=0tkrIjt−r−α∑r=0tkrgixit−r−θi,fxit+1=11+exp−xit+1/ε.Figure 5
Chaotic neuron model.The structure of the chaotic neural network is shown in Figure6. The feedback and hidden layers of the network use logistic and linear functions as transfer functions, respectively, to make it have chaotic characteristics, and the chaotic characteristics exhibited by the neurons can make the network have better approximation ability. The transfer functions of each layer of the network are as follows.(9)a1k=logisticIW1,1p+LW1,1a1k−1+b1,a2k=purelinLW2,1a1k+b2.Figure 6
Structure of the chaotic neural network.


## 4. Experiments

When applying the prediction model to the prediction of the learning effectiveness of college students, the raw data of the training network are usually varied by different variables in different units and orders of magnitude. It is known from the properties of neuron activation functions that the output of neurons is usually restricted to a certain range, and the output of nonlinear activation functions used in most artificial neural network applications is limited to (0, 1) or (−1, 1). Training the network directly with raw data causes neuron saturation, so the data must be preprocessed with normalization before training the network with the formula:(10)Pi=pi−pminpmax−pmin,Ni=ni−nminnmax−nmin,where pi,ni are the original target, input data; pmin,pmax,nmin,nmax are the minimum and maximum values in P and N; Pi,Ni are the normalized target, input data [24–27].The mean absolute percentage error (MAPE) is used as the prediction result comparison criterion. Its calculation formula is(11)MAPE=1N∑i=1NPfi−PaiPai×100%.The actual data are input into different models, and the predicted data of each model are compared. The results shown in Figure7 are obtained.Figure 7
Prediction results of each model. (a) Homoscedastic analysis model with prediction results. (b) Elman network model with prediction results. (c) Chaotic neural network model with prediction results.
(a)(b)(c)The MAPE of the prediction results of each model are given in Table2.Table 2
Mean absolute percentage error (MAPE).
Prediction modelsMAPE (%)BP neural networks2.3822Elman neural network1.8401Chaotic neural network1.7211It can be seen from the prediction that the chaotic neural network prediction model based on big data analysis adopted in this study has more accurate prediction accuracy than the other two models, the accuracy of predicting students’ academic performance, and the prediction error is also higher than the other two models [28–30].


## 5. Conclusion

This study mainly aimed of implementing student development and integrating the teaching process, learning activities, and evaluation activities. Through the evaluation of students’ learning attitudes, learning behaviors, learning abilities, learning outcomes, and learning effectiveness, the prediction method of students’ learning effectiveness is established based on big data analysis methods, and very superior prediction results are achieved. It promotes teaching activities to trigger students’ higher-order learning activities, which is conducive to grasping students’ academic situation in an all-round way, as well as finding problems in teaching and effective ways to solve them and improve quality.



---

*Source: 8409780-2022-07-31.xml*