# Evaluation System for the Talent Training Quality of Higher Education Based on the Combination of the Subjective and Objective Evaluation Method and AdaBoost-SVM

Authors: Ruifeng Liu; Lijun Li
Journal: Wireless Communications and Mobile Computing
 (2022)

---

## Abstract

This paper mainly researches for the evaluation index system of college students’ cultivation quality. First, we establish a comprehensive evaluation index system of college students’ cultivation quality from three aspects of college students’ knowledge, ability, and quality, to establish the evaluation index system which consists of 3 first-level indexes, 14 second-level indexes, and 40 index observation points. Then, the method of assigning the index weight is optimized and the method of combining subjective and objective weight is adopted. Finally, this paper introduces the algorithm of machine learning as an evaluation method. In order to improve the performance of the algorithm, diversity measure is introduced to ensure that the base learner has good diversity. We proposed a new algorithm AdaBoost-support vector machine (AdaBoost-SVM); it is found that the proposed method gets a high accuracy rate of 91% with a small amount of data in our education datasets. The research results of this paper solve the problems of incomplete index creation, unreasonable index weight assignment, and low efficiency of evaluation methods in the current evaluation of college student’s cultivation quality.


---

## Body

## 1. Introduction

At the end of the last century, college enrollment policy in China has been adjusted to expand the enrollment of colleges and universities. From then on, China’s higher education has entered into a rapid development period for its rapid growth in enrollment and newly established schools. Since the enrollment expansion, the enrolled students in technical colleges have increased from 430,000 in 1998 to 3,507,000 in 2017, which is an eightfold growth. From 1998 to 2017, the enrollment of undergraduate students increased from 653,000 to 4.107 million from 1998 to 2017, which is an increase of nearly 6.3 times (data source: National Bureau of Statistics). It can be seen that since the expansion of enrollment in 1999, higher education has achieved remarkable outcome in terms of enrollment and the development of higher education has also been promoted. At the same time, more colleges and universities have been established to meet the needs of higher education expansion: the number of ordinary colleges and universities rose from 1,022 in 1998 to 2,631 in 2017. At the same time, the development of private colleges and universities in our country has also achieve its bonanza: the number of private general colleges and universities has tripled from 226 in 2004 to 727 in 2014. Among them, the number of undergraduate colleges and universities rose from 9 in 2004 to 420 and the number of private undergraduate colleges and universities expanded to 46 times the original number in ten years (data source: the National Bureau of Statistics). According to the data analysis of the National Bureau of Statistics, higher education has achieved a rapid development in the past 20 years.Some problems gradually emerged after the expansion of enrollment. The sharp expansion of enrollment resulted in a shortage of teaching resources in colleges and universities. The teaching quality and students’ quality decline further as a result of the shortage of teaching resources such as faculty in the teaching process of college students.At present, there are no authoritative methods and unified standards for the quality evaluation of college students. Some researches are far from comprehensive with unreasonable evaluation index settings, such as inappropriate amounts of index settings and outdated and incomplete index settings, resulting in the unscientific evaluation result. This article is of theoretical reference significance, and the practical value for the quality of college students’ training is assessed in the article through the establishment of the training index system, the improvement of the weight setting, and the optimization of the evaluation method.


## 2. Related Work

In the evaluation of the training quality of college students abroad, taking the United States as an example, there are few researches on the evaluation of college students’ training quality and the quality of college students’ training has always been measured by students’ learning achievements which is also used as an indicator to assess the higher education level. This method was later sought after by many European countries in the assessment of higher education. The United States assesses students’ learning achievements with the goal of improving the quality of higher education, making students’ learning goals clearer, promoting students’ learning and enabling students to develop better. At present, collegiate learning assessment (CLA) is the most influential one in the United States. Although the quality of college students’ training has no direct way of assessment in the United States, it can be indirectly reflected by the evaluation of student learning achievements and students can be evaluated through student learning evaluation. The evaluation of college students’ learning is also carried out through multiple aspects to make the evaluation more scientific and reasonable.The assessment of college students in the United States is mainly carried out through four aspects: critical thinking, problem-solving ability, written communication skills that enables college students to better express their own learning perception and learning experience and to make high-quality communication, and logical analysis and reasoning ability that enables college students to more deeply analyze problems, during which student’s cognitive competence will be further improved.The evaluation of college students abroad is mainly carried out through the assessment of college students’ learning, and for graduate students, it is evaluated mainly from the government, society, and the university itself through different evaluators’ assessments. Of course, the evaluation of graduate students in the U.S. is also the first to be systematic with diversified characteristics.The study on the learning quality indicators of nursing students in clinical practice is to obtain quantitative indicators that affect students’ practice learning by investigating the students’ experience in practice. In addition, the evaluation of engineering education majors is carried out by the fuzzy set method [1]. Besides the evaluation of specific aspects of certain specialties, there are also evaluations of some special abilities of students, such as digital literacy. With the development of the Internet, the Internet has become a very important tool for students to acquire knowledge and special skills. Through factor analysis and questionnaires, a network of indicators affecting students’ digital literacy [2] can be constructed. There are few researches on the quality of student training, and most of them focus on a certain aspect of research.Of course, there are many studies on assessment index systems abroad in terms of different issues in different industries and evaluation methods based on the established index system. For example, the environment can be evaluated by establishing an evaluation framework [3]. There are also many studies on evaluation methods, which combines specific issues to the model, such as the use of fuzzy comprehensive evaluation methods in risk assessment [4]. Similarly, research on machine learning evaluation is increasingly being applied to various evaluation and predictive models, such as the risk evaluation mentioned above [5, 6]. The evaluation method is selected according to the specific evaluation object. There is no fixed selection criterion for the evaluation method, and the evaluation method with good performance should be selected according to the specific problem.Machine learning-based methods have been widely used in educational management and other fields [7–9]. At present, many scholars have put forward different schemes for the talent training system from different perspectives. According to the division of talent training objects, scholars have studied the training system of technical innovative talents [10]. and talent training systems in different fields (such as e-commerce [11], clinical medicine [12], basic chemistry [13], urban development planning [14], traditional Chinese Medicine education [15], mathematics [16], and landscape architecture [17]). In addition, a small number of scholars have also carried out research on the talent cultivation system of scientific research [18]. However, there are few studies on the comprehensive talent training system in colleges and universities. Among the existing studies, for example, there is a talent training and evaluation system for higher vocational education [19] and a talent evaluation system for cooperation with enterprises [20] but these studies are based on the perspective of third-party institutions. In particular, AHP [21] and fuzzy comprehensive evaluation [22] have been used in some studies on the comprehensive talent training system of ordinary colleges and universities and certain results have been achieved..


## 3. Construction of the Evaluation Index System of College Student Cultivation Quality

The quality evaluation of college students is inseparable from the evaluation index system of training quality. The evaluation of college students’ training quality is analyzed in this part, and the evaluation objective was divided into three aspects: knowledge, ability, and quality for evaluation. At the same time, each item was analyzed to obtain secondary indicators and the secondary indicators were quantified through observation points. The advantages and disadvantages of the subjective weighting method and the objective weighting method are compared and analyzed in this part to propose and adopt a more reasonable combined weighting method for the weighting of college students’ quality evaluation indicators.Undergraduate training quality evaluation is the evaluation goal. In this article, there are three first-level indicators for training quality evaluation: knowledge, quality, and ability, as introduction in Figure1.Figure 1
Graph cultivation quality and first-level index.The first-level indicators can be decomposed and analyzed to obtain the second-level indicators corresponding to different first-level indicators. There are 14 secondary indicators in total, among which the knowledge indicators include the following: 4 are concluded in the knowledge indicators, 6 are in the ability indicators, and 4 are in the quality indicators.


### 3.1. The Secondary Index of the Knowledge Index

Knowledge indicators should be analyzed to obtain what kind of knowledge college students should master and then split into 4 secondary indicators, including humanities and social science knowledge, mathematical, physical and computer science knowledge, foreign language knowledge, and profession knowledge.


### 3.2. Secondary Indicators of Ability Indicators

The ability indicators of college students’ training quality mainly include knowledge acquisition ability, knowledge application ability, knowledge output ability, communication ability, innovative ability, and the career-exploring ability.


### 3.3. Secondary Indicators of Quality

The secondary quality indicators include ideological quality, cultural quality, physical and mental quality, and profession quality.The primary and secondary indicators are shown in Table1.Table 1
Hierarchical relationship.
Evaluation objectivePrimary indicatorsSecond indicatorsQuality of college student trainingKnowledgeHumanities and social science knowledgeMathematics, physics, chemistry, and computer science knowledgeForeign language knowledgeProfession knowledgeAcquisition knowledgeApplication knowledgeAbilityKnowledge expression abilityCommunication abilityInnovation abilityEntrepreneurship abilityQualityIdeological qualityCultural qualityPhysical and mental qualitiesProfession quality


### 3.4. Index Observation Points

Primary indicators can be decomposed into secondary indicators that are more detailed and specific in demonstration. But the secondary indicators still cannot satisfy the need of evaluation of the quality of college students’ training. More refined indicators can be used for evaluation, which is convenient for us to collect indicator data. Therefore, observation points, which refer to the aspects that we plan to collect data from to quantify the indicators, are introduced into the article to quantify the secondary headings and evaluate the quality of college students’ training based on the data collected from the observation points. Although the observation points are at the next level of the secondary index, there is no strict subordination relationship between the observation points and the secondary indicators. Instead of that, the observation points are used to better represent the secondary indicators and facilitate the quantification of the indicators. For the selection of observation points, there is no unified and fixed standard. The observation points are only needed to well represent the last-level indicator in different situation.


#### 3.4.1. Main Observation Points of Secondary Indicators of Knowledge

(1)
The observation points of profession knowledge mainly include class results of the professional basic courses and professional core courses(2)
The observation points of foreign language knowledge mainly include the scores of foreign language courses, CET-4, CET-6, and foreign language competitions(3)
Humanities and social science knowledge indicators refer to the situation of borrowed books or the results of cultural and sports activities, including philosophy, sociology, history, aesthetic art, literature, and psychology


#### 3.4.2. Main Observation Points of Secondary Indicators of Ability

(1)
Acquisition knowledge ability, the ability of on-site investigation, and the ability of basic data search and literature exploring(2)
Knowledge application ability: independent of engaging in production, scientific research, and management, which includes internships and ensuring the quality of the graduation project(3)
Knowledge expression ability: paper writing and academic report(4)
Communication ability: participation in club activities and performance in team competitions(5)
Innovation ability: innovation outcome and practice(6)
Entrepreneurship ability: professionalism and profession foundation


#### 3.4.3. Main Observation Points of Secondary Indicators of Quality Indicators

(1)
Ideological quality: personal quality, collective spirit, labor viewpoint, learning attitude, political attitude, and honesty(2)
Cultural quality: modern consciousness, social science and art literacy, and writing style literacy(3)
Physical and mental qualities: psychology quality and physical quality(4)
Profession quality: profession literacy and profession foundation


### 3.5. Indicator System

The evaluation index system of college students’ cultivation quality is shown in Table2.Table 2
Index system.
Primary indicatorsSecondary indicatorsObservation pointsIndicationKnowledgeHumanities and social science knowledgePhilosophy(1) The situation of borrowed books(2) The results of cultural and sports activities and grades of relevant elective coursesSociologyHistoryLiteratureArt aestheticsPsychologyMathematics, physics, and computer science knowledgeComputer rank(1) Competition results(2) Grade examination(3) Grades in computer coursesCompetitionCourseForeign language knowledgeGrades in foreign language courses and rank examinations(1) Rank and grades(2) Grades of coursesProfession knowledgeGrades of basic specialized courses and grades of core-specialized coursesGrades of coursesAcquisition knowledgeField investigation(1) Literature retrieval course(2) Teacher’s evaluationBasic data collectionAnalysis abilityLiterature reviewApplication knowledgeIndependently engaged in related(1) Internship(2) Graduation designProductionScientific researchManagementInternshipGraduation designKnowledge expressionWriting papers and reports(1) Paper publish(2) Academic reportAbilityCommunication abilityCommunication(1) The performance of group work(2) The performance of team competition in club activityOrganizationDecision makingCooperationInnovation abilityInnovation opportunity positioning(1) Results of innovation coursesInnovative practice and achievementsEntrepreneurship abilityEntrepreneurship preparation(1) Grades of entrepreneurship courses(2) Entrepreneur competitionEntrepreneurship emotional intelligenceQualityIdeology qualityPolitics attitude(1) Record of bad behavior(2) Good deeds(3) Teacher’s evaluation and peer assessmentPersonal cultivationLearning attitudeCollective ideaLabor viewpointBeing law-abidingHonesty and trustworthinessCulture qualityThe application of different writing styles(1) Teacher’s comments(2) Course grades(3) Participation in related activitiesSocial science and art literacyModern awarenessTypefacePhysical and mental qualitiesPhysical quality(1) Medical examination reportMental quality(1) Psychological testProfession qualitySolid foundation of professional knowledgeProfession performance


### 3.6. Quantitative Indicators

In this paper, the secondary indicators are decomposed according to observation points and the quantification of indicators should be worked out by the data that can reflect corresponding indicators as much as possible, as the observation points can be summarized into the following aspects and also be carried out through corresponding data.(1)
Course grades, practice performance, psychological tests, and physical examination reports which can reflect the corresponding indicators(2)
Obtained relevant certificates, such as competitions and rating certificates(3)
The publication and the rating of papers, the impact factors, and the number of published papers(4)
The number of related books read, situation of reading, and category of these borrowed books(5)
The teacher’s evaluation of the students, and the mutual evaluation among studentsAs for award scores, different levels of awards depend on different scores. The highest one is the national level, and the awards decrease successively in accord with the administrative ranks. It is arranged from the national, provincial, and municipal to departmental levels, and the highest award is always taken in the same category. The scores are 100, 90, 80, 70, 60, 50, 40, 30, 20, and 10. Each rank has three levels of awards. 10 points are added for the first prize at the department level, and no points are added below the first prize.The criterion for papers to be published such as SCI, EI academic ones regards 100 points as the benchmark, The full score of the published indicator such as articel indexed by SCI and EI is 100. The final score is accounted by multiplying 100 points by the impact factors. 80 points will be added for academic papers published in the first-level journals of the national main board, 70 points for core journals and EI conferences, and 60 points for general journals. In order to avoid someone in name only, the second place has 60% of the scores of the first place and no points for the rests.10 points is added for each book which is read and 100 points as the full mark. Books in different categories are scored by different requirements. The observation points in terms of ideological cultivation are determined by mutual evaluation of students and teacher evaluation, according to Likert scale and the evaluation of semantics scale. That is divided into five grades: excellent, good, average, poor, and fail, also marked as 100, 80, 60, 40, 20, and 10. Thus, the student’s final score is worked.


## 3.1. The Secondary Index of the Knowledge Index

Knowledge indicators should be analyzed to obtain what kind of knowledge college students should master and then split into 4 secondary indicators, including humanities and social science knowledge, mathematical, physical and computer science knowledge, foreign language knowledge, and profession knowledge.


## 3.2. Secondary Indicators of Ability Indicators

The ability indicators of college students’ training quality mainly include knowledge acquisition ability, knowledge application ability, knowledge output ability, communication ability, innovative ability, and the career-exploring ability.


## 3.3. Secondary Indicators of Quality

The secondary quality indicators include ideological quality, cultural quality, physical and mental quality, and profession quality.The primary and secondary indicators are shown in Table1.Table 1
Hierarchical relationship.
Evaluation objectivePrimary indicatorsSecond indicatorsQuality of college student trainingKnowledgeHumanities and social science knowledgeMathematics, physics, chemistry, and computer science knowledgeForeign language knowledgeProfession knowledgeAcquisition knowledgeApplication knowledgeAbilityKnowledge expression abilityCommunication abilityInnovation abilityEntrepreneurship abilityQualityIdeological qualityCultural qualityPhysical and mental qualitiesProfession quality


## 3.4. Index Observation Points

Primary indicators can be decomposed into secondary indicators that are more detailed and specific in demonstration. But the secondary indicators still cannot satisfy the need of evaluation of the quality of college students’ training. More refined indicators can be used for evaluation, which is convenient for us to collect indicator data. Therefore, observation points, which refer to the aspects that we plan to collect data from to quantify the indicators, are introduced into the article to quantify the secondary headings and evaluate the quality of college students’ training based on the data collected from the observation points. Although the observation points are at the next level of the secondary index, there is no strict subordination relationship between the observation points and the secondary indicators. Instead of that, the observation points are used to better represent the secondary indicators and facilitate the quantification of the indicators. For the selection of observation points, there is no unified and fixed standard. The observation points are only needed to well represent the last-level indicator in different situation.


### 3.4.1. Main Observation Points of Secondary Indicators of Knowledge

(1)
The observation points of profession knowledge mainly include class results of the professional basic courses and professional core courses(2)
The observation points of foreign language knowledge mainly include the scores of foreign language courses, CET-4, CET-6, and foreign language competitions(3)
Humanities and social science knowledge indicators refer to the situation of borrowed books or the results of cultural and sports activities, including philosophy, sociology, history, aesthetic art, literature, and psychology


### 3.4.2. Main Observation Points of Secondary Indicators of Ability

(1)
Acquisition knowledge ability, the ability of on-site investigation, and the ability of basic data search and literature exploring(2)
Knowledge application ability: independent of engaging in production, scientific research, and management, which includes internships and ensuring the quality of the graduation project(3)
Knowledge expression ability: paper writing and academic report(4)
Communication ability: participation in club activities and performance in team competitions(5)
Innovation ability: innovation outcome and practice(6)
Entrepreneurship ability: professionalism and profession foundation


### 3.4.3. Main Observation Points of Secondary Indicators of Quality Indicators

(1)
Ideological quality: personal quality, collective spirit, labor viewpoint, learning attitude, political attitude, and honesty(2)
Cultural quality: modern consciousness, social science and art literacy, and writing style literacy(3)
Physical and mental qualities: psychology quality and physical quality(4)
Profession quality: profession literacy and profession foundation


## 3.4.1. Main Observation Points of Secondary Indicators of Knowledge

(1)
The observation points of profession knowledge mainly include class results of the professional basic courses and professional core courses(2)
The observation points of foreign language knowledge mainly include the scores of foreign language courses, CET-4, CET-6, and foreign language competitions(3)
Humanities and social science knowledge indicators refer to the situation of borrowed books or the results of cultural and sports activities, including philosophy, sociology, history, aesthetic art, literature, and psychology


## 3.4.2. Main Observation Points of Secondary Indicators of Ability

(1)
Acquisition knowledge ability, the ability of on-site investigation, and the ability of basic data search and literature exploring(2)
Knowledge application ability: independent of engaging in production, scientific research, and management, which includes internships and ensuring the quality of the graduation project(3)
Knowledge expression ability: paper writing and academic report(4)
Communication ability: participation in club activities and performance in team competitions(5)
Innovation ability: innovation outcome and practice(6)
Entrepreneurship ability: professionalism and profession foundation


## 3.4.3. Main Observation Points of Secondary Indicators of Quality Indicators

(1)
Ideological quality: personal quality, collective spirit, labor viewpoint, learning attitude, political attitude, and honesty(2)
Cultural quality: modern consciousness, social science and art literacy, and writing style literacy(3)
Physical and mental qualities: psychology quality and physical quality(4)
Profession quality: profession literacy and profession foundation


## 3.5. Indicator System

The evaluation index system of college students’ cultivation quality is shown in Table2.Table 2
Index system.
Primary indicatorsSecondary indicatorsObservation pointsIndicationKnowledgeHumanities and social science knowledgePhilosophy(1) The situation of borrowed books(2) The results of cultural and sports activities and grades of relevant elective coursesSociologyHistoryLiteratureArt aestheticsPsychologyMathematics, physics, and computer science knowledgeComputer rank(1) Competition results(2) Grade examination(3) Grades in computer coursesCompetitionCourseForeign language knowledgeGrades in foreign language courses and rank examinations(1) Rank and grades(2) Grades of coursesProfession knowledgeGrades of basic specialized courses and grades of core-specialized coursesGrades of coursesAcquisition knowledgeField investigation(1) Literature retrieval course(2) Teacher’s evaluationBasic data collectionAnalysis abilityLiterature reviewApplication knowledgeIndependently engaged in related(1) Internship(2) Graduation designProductionScientific researchManagementInternshipGraduation designKnowledge expressionWriting papers and reports(1) Paper publish(2) Academic reportAbilityCommunication abilityCommunication(1) The performance of group work(2) The performance of team competition in club activityOrganizationDecision makingCooperationInnovation abilityInnovation opportunity positioning(1) Results of innovation coursesInnovative practice and achievementsEntrepreneurship abilityEntrepreneurship preparation(1) Grades of entrepreneurship courses(2) Entrepreneur competitionEntrepreneurship emotional intelligenceQualityIdeology qualityPolitics attitude(1) Record of bad behavior(2) Good deeds(3) Teacher’s evaluation and peer assessmentPersonal cultivationLearning attitudeCollective ideaLabor viewpointBeing law-abidingHonesty and trustworthinessCulture qualityThe application of different writing styles(1) Teacher’s comments(2) Course grades(3) Participation in related activitiesSocial science and art literacyModern awarenessTypefacePhysical and mental qualitiesPhysical quality(1) Medical examination reportMental quality(1) Psychological testProfession qualitySolid foundation of professional knowledgeProfession performance


## 3.6. Quantitative Indicators

In this paper, the secondary indicators are decomposed according to observation points and the quantification of indicators should be worked out by the data that can reflect corresponding indicators as much as possible, as the observation points can be summarized into the following aspects and also be carried out through corresponding data.(1)
Course grades, practice performance, psychological tests, and physical examination reports which can reflect the corresponding indicators(2)
Obtained relevant certificates, such as competitions and rating certificates(3)
The publication and the rating of papers, the impact factors, and the number of published papers(4)
The number of related books read, situation of reading, and category of these borrowed books(5)
The teacher’s evaluation of the students, and the mutual evaluation among studentsAs for award scores, different levels of awards depend on different scores. The highest one is the national level, and the awards decrease successively in accord with the administrative ranks. It is arranged from the national, provincial, and municipal to departmental levels, and the highest award is always taken in the same category. The scores are 100, 90, 80, 70, 60, 50, 40, 30, 20, and 10. Each rank has three levels of awards. 10 points are added for the first prize at the department level, and no points are added below the first prize.The criterion for papers to be published such as SCI, EI academic ones regards 100 points as the benchmark, The full score of the published indicator such as articel indexed by SCI and EI is 100. The final score is accounted by multiplying 100 points by the impact factors. 80 points will be added for academic papers published in the first-level journals of the national main board, 70 points for core journals and EI conferences, and 60 points for general journals. In order to avoid someone in name only, the second place has 60% of the scores of the first place and no points for the rests.10 points is added for each book which is read and 100 points as the full mark. Books in different categories are scored by different requirements. The observation points in terms of ideological cultivation are determined by mutual evaluation of students and teacher evaluation, according to Likert scale and the evaluation of semantics scale. That is divided into five grades: excellent, good, average, poor, and fail, also marked as 100, 80, 60, 40, 20, and 10. Thus, the student’s final score is worked.


## 4. Indicator Weighting

### 4.1. Common Weighting Methods

There are quite a few kinds of indicator weighting setting methods that can be divided into two categories according to different methods of determining the weight. The one which is determined by scoring through the knowledge and experience of people as the indicator is called the subjective weighting method. Common subjective weighting methods include the analytic hierarchy process (AHP) and Delphi method. The other is the objective weighting method, that is, a weight of the corresponding indicator obtained after analyzing and sorting out some data, such as entropy weight method (EWM), principal component analysis (PCA), and factor analysis (FA).The characteristics of the weights and the difference in evaluation results are determined by different weighting methods. The subjective weighting methods are relatively mature so that it is easier to be used to explain the evaluation results and follow subjective will. For example, in the assessment of cultivation quality of undergraduates, the certain indicator can be set according to the analysis of different experts on the current undergraduates and even the weight of important indicators can be increased according to the analysis. However, the weight setting will be affected by subjective factors, so the results of weighting and evaluation are not objective. As for objective methods, the weights of indicators can be worked out through the analysis of corresponding data, so the evaluation results are more objective but it is supported by the data completely, which may lead to the result that the direction of the evaluation target is inconsistent with people’s will. Moreover, it is sometimes difficult to explain the evaluation results. At the same time, the difference of results is greatly affected by the weighting method. Thus, the evaluations of some fields are hard to be accepted by the public if the objective weighting method is used only.


### 4.2. Weighting Methods and Limitations of the Evaluation Indicator

#### 4.2.1. Analytic Hierarchy Process (AHP)

The analytic hierarchy process, abbreviated as AHP, has been proposed more than 30 years ago. The method is, in short, to simplify a complex issue by decomposing it into simple parts. Thus, the original goal can be divided step by step into several goals or criterion which are easy to deal with.For example, in a comprehensive evaluation, the original evaluation goal is decomposed into several evaluation indicators or criterion. That is, the original evaluation goals can be split into some aspects from which the goals can be evaluated better. As for decomposed indicators or criterion, they still can be decomposed into more simple ones. Finally, a complex evaluation goal is decomposed into several simple indicators of evaluation and completed through the final evaluation indicators.The analytic hierarchy process falls into 3 parts including the object layer, criterion layer, and scheme layer. Their hierarchical models are shown Figure2.Figure 2
Schematic diagram of the hierarchical model.The steps of the analytic hierarchy process to determine the weight in the evaluation indicator of undergraduates’ cultivation quality:(1) Establish a Hierarchical Indicator System for the Evaluation on Undergraduates’ Cultivation Quality. In order to establish a hierarchical model of the evaluation indicator system on undergraduates’ cultivation quality, an in-depth analysis should be made in evaluation issues and an evaluation indicator system is established to determine the object layer and criterion layer, among which the criterion layer is analyzed to determine the level and the subordinate relationship of the criterion layer.(2) Construct a Judgment Matrix. In this paper, the indicators are analyzed through the way to establish an indicator system for the evaluation on undergraduates’ cultivation quality and the importance of indicators is determined by the corresponding scale comparison table, and also, a judgment matrix can be constructed according to the importance of indicators which are confirmed. The primary diagonal element of the matrix is 1, and those elements which are symmetric with the primary diagonal are reciprocal, and their all values are greater than 1.(3) The Consistency Test of the Constructed Judgment Matrix. The judgment matrix can be regarded as qualified when the consistency is tested to the criterion; on the contrary, it needs to be modified. For example, if the importance of indicator A for indicator B is 3, B for C is 2; so, what is A for C? 6 should be worked out from the condition above. However, actually, the situation is inconsistent that the relative importance given is 4. Of course, it is difficult to keep consistent completely in practical application and excessive consistency would influence the scientificity in analyzing the indicators, even in judging the indicators subjectively from consistency, rather than the importance of the indicator in practice. Thus, the judgment matrix only needs to meet a certain consistency, that is, the matrix is reasonable and can be used if it passes the consistency test. As for the one which fails to pass, it cannot be accepted until it meets the consistency by reanalyzing indicators and confirming relative importance and reperforming the test.(4) Hierarchical Single Sorting. Hierarchical single ranking refers to ranking the weights of the indicators at the upper level according to the relative importance of the indicators at the current level. That is to sort the weight values corresponding to each index calculated by the judgement matrix.(5) Hierarchical Total Sorting. Hierarchical total sorting should be based on the single sorting result and combined with the weight of the upper index and able to meet the needs of the consistency test. The hierarchical total sorting result is the final weight ranking result, and the weight determined by the hierarchical total ranking is evaluated.(6) Evaluation. The final evaluation result should be determined by using the corresponding data of the evaluation index, combining with the final weight of the index to make a general evaluation by referring to the actual situation and the final score.In general, the analytic hierarchy process (AHP) is divided into six steps. Firstly, it is establishing the index system and analyzing the relationship between each level of indicators and the lower level.The next step is analyzing the importance of the indicators relative to the target level and the upper level. According to the comparison table of the AHP scale, the importance degree is obtained by comparing the two indexes. The next step is constructing the judgment matrix, carrying on the consistency test and hierarchical single and total sort. Finally, the evaluation and the analysis to the evaluation result should be given.The flow chart of AHP evaluation is shown in Figure3.Figure 3
Flow chart of analytic hierarchy process.The advantages and limitations of the analytic hierarchy process can be seen in the process of applying it to the weight setting of the indicators in the college student training quality evaluation index system.The analytic hierarchy process is a systematic analysis method that determines the weight of the college student training quality index system through the evaluators’ judgment and quantitative analysis on the college quality evaluation index system. The method is simple and practical with strict mathematical derivation. The mathematical knowledge used in the process is simple and easy to master, and a simplified method is applied to the calculation of the matrix, which makes the derivation simpler and more efficient. What is more, the weight is determined not with a large amount of data and information. The evaluator analyzes the evaluation target and then makes a judgment on the importance of the index, which is in line with people’s thinking habits.


#### 4.2.2. Entropy Method

The entropy method is a kind of objective weighting method. Entropy is originally a thermodynamic concept, which reflects the degree of disorder in the system. Entropy was first introduced by Claude Elwood Shannon from thermodynamics to information theory. Information entropy bears some general features of thermodynamics, but it is a broader concept that has been widely used in many fields. Information entropy is a measure of the degree of disorder. The greater the information entropy, the higher the degree of disorder, and the less information is included. The entropy method is to calculate the information entropy of each indicator based on the data value of the indicator and then determine the weight of the indicator. The flow chart of the entropy method for weighting indicators is shown in Figure4.Figure 4
Flow chart of the entropy weight method.The entropy method is simple for determining the index weight. The information entropy and indicator weights are obtained by processing the indicator data, which effectively avoids the problem of insufficient objective weights caused by human subjective factors. However, the entropy method completely relies on the index data to determine the index weight, and then, the weight is obtained through calculation. The evaluation result may not meet the subjective needs of the evaluator, making it difficult for the evaluator to understand and explain.


### 4.3. Improvement of the Weighting Method

By organically combining the subjective weighting method and the objective weighting method, the weights can be optimized, so that the weights can reflect the wishes of the evaluator and can be explained without violating the objectivity of the results. Since the objective weighting method is the result of weighting calculated through indicator data, the weighting depends on the data and is objective.This article adopts the linear combination method and introduces a distance function to determine the coefficient of linear combination. The subjective weight and the objective masses are multiplied by different coefficients and summed to obtain the improved combined weight. The flow chart of the combined weighting calculation method is shown in Figure5.Figure 5
Flow chart of the combined weighting method.Based on the Delphi method, the features of the analytic hierarchy process, and the number of indicators in this article, the analytic hierarchy process is first used to subjectively weigh the indicators, and then, the entropy method is adopted to objectively weigh the indicators, and finally, the weights obtained through the two methods is combined to gain the final weight.


### 4.4. AHP(Analytic Hierarchy Process) Weights

(1)
The evaluation target is denoted asA, the evaluation indexB=B1,B2,B3,⋯,Bn, the relative importance ofBi, andBjto the evaluation target ABij; the value of which is an integer from 1 to 9; for details, see Table 3Table 3
Decision matrix scale table.
ScaleImplication1The two indicators are equally important3The former is slightly more important than the latter5The former is obviously more important than the latter7The former is more important than the latter9The former is extremely important than the latter2,4,6,8The two intermediate values in the abovementioned comparisonReciprocalThe ratio of indexi to j is Bij; then, the ratio of j to i is 1/BijThe judgment matrix of the evaluation target isA as shown in formula (1).
(1)A=b11b12⋯b1nb21b22b2n⋮⋱⋮bn1bn2⋯bnn.(2)
Formula (2) can be used to quantize the column items of matrix A and normalize b¯ijto be the normalized matrix elements(2)b¯ij=bij∑j=1nbij,j=1,2,3,⋯,n(3)
The normalized matrix is summed by row vectors to obtain column vectorW as shown in formula (3) as follows:(3)W=wi,i=1,2,3,⋯,n,wi=∑jnbij,i=1,2,3,⋯,n(4)
Normalization of column vectorW is shown in formula (4) as follows:(4)w¯i=wi∑i=1nwi,i=1,2,3,⋯,n,where feature vector W¯=w¯i,i=1,2,3,⋯,n is the weight vector of each index
(5)
The step is a consistency checkThe consistency index is calculated from the combination of formula (6) and the maximum characteristic root λmax calculated by formula (5) as follows.
(5)λmax=∑inAW¯inW¯i,(6)CR=CIRI.The conformance ratio CR was calculated by formula (7) as follows.
(7)CR=CIRI.RI is a random consistency indicator, which can be obtained by looking up the table. Table4 lists the consistency indicators. N is the index number.
(6)
Total weight and total consistency test of indicatorsTable 4
Random consistency index.
n1234567891011RI000.580.901.121.241.321.411.451.491.51According to the above steps, the weight of the first-level indicator iswiQB, the weight of the second-level indicator C to the first-level indicator B is wijBC, and the weight of the second-level indicator to Q is wij′, which can be calculated by the following formula.
(8)wij′=wiQB×wijBC.The consistency ratio of indicatorC to Q, CRz, is calculated from (8) and satisfies CRz<0.1.
(9)CRz=∑wiQBCIi∑wiQBRIi.RIi and CIi are the random consistency index and consistency index of secondary index C to Bi, respectively; for details, see Table 4.IfCR<0.1, the weight corresponding to the determined indicator will be in accord with the requirement of consistency.


### 4.5. Entropy Weighting Method

According to the index system already established in this paper, the secondary indicator is used for evaluating the cultivation quality of undergraduates. The number of indicator ism and the number of students is n to construct the matrix R as shown in (10) as follows.
(10)R=rijnxm=r11r12⋯r1mr21r22r2m⋮⋱⋱rn1rn2⋯rnm.(1)
The matrixR is normalized according to formula (11)(11)Rij=rij−minr1j,⋯,rnjmaxr1j,⋯,rnj−minrij,⋯,rnj,where the matrix R has been normalized as shown in formula (12) as follows:
(12)R¯=Rijnxm=R11R12⋯R1mR21R22R2m⋮⋱⋮Rn1Rn2⋯Rnm(2)
The proportionpij of the indicator j of the sample i should be calculated according to formula (13) as follows:(13)pij=Rij∑i=1nrij,i=1,2,⋯,nj=1,2,⋯,m(3)
Calculate the entropyej of the indicator j according to formula (14) as follows:(14)ej=−1Inn∑i=1npijInpij,j=1,⋯,m(4)
Calculate the weight w of the indicatorj according to formula (15) as follows:(15)wj=1−ej∑j=1m1−ej,where 1−ej is the discrepancy coefficient of the indicator j


### 4.6. Improved Weighting Methods to Calculate Weights

In this paper, the weighting method is improved by using the distance function to determine the coefficient of the combination weight. If the combination weight iswc, wi′ and wi″ are the weights obtained by the subjective weighting method and the objective weighting method, respectively; at the same time, α and β are coefficients; then, the combination weights are shown in formula (16) as follows:
(16)Wc=∝wi′+βwi′′.Suppose that the distance function of the subjective weighting method and the objective weighting method isdwi′,wi′′ and the distance function can be expressed as follows:
(17)dwi′,wi′′=12∑i=1nwi′−wi′′21/2.In order to ensure that the degree of weight difference is consistent with that of coefficients, letα−β2 and dwi′,wi′′ be equal to obtain formula (18):
(18)α−β2=dwi′,wi′′.The weighting distribution coefficients fit in with the formula:(19)α+β=1.Simultaneous formulas can be expressed as follows:(20)α−β2=dwi′,wi′′,α+β=1.The weight distribution coefficientsα and β can be obtained and substituted into the combined weighting expression to obtain the combined weight Wc.


## 4.1. Common Weighting Methods

There are quite a few kinds of indicator weighting setting methods that can be divided into two categories according to different methods of determining the weight. The one which is determined by scoring through the knowledge and experience of people as the indicator is called the subjective weighting method. Common subjective weighting methods include the analytic hierarchy process (AHP) and Delphi method. The other is the objective weighting method, that is, a weight of the corresponding indicator obtained after analyzing and sorting out some data, such as entropy weight method (EWM), principal component analysis (PCA), and factor analysis (FA).The characteristics of the weights and the difference in evaluation results are determined by different weighting methods. The subjective weighting methods are relatively mature so that it is easier to be used to explain the evaluation results and follow subjective will. For example, in the assessment of cultivation quality of undergraduates, the certain indicator can be set according to the analysis of different experts on the current undergraduates and even the weight of important indicators can be increased according to the analysis. However, the weight setting will be affected by subjective factors, so the results of weighting and evaluation are not objective. As for objective methods, the weights of indicators can be worked out through the analysis of corresponding data, so the evaluation results are more objective but it is supported by the data completely, which may lead to the result that the direction of the evaluation target is inconsistent with people’s will. Moreover, it is sometimes difficult to explain the evaluation results. At the same time, the difference of results is greatly affected by the weighting method. Thus, the evaluations of some fields are hard to be accepted by the public if the objective weighting method is used only.


## 4.2. Weighting Methods and Limitations of the Evaluation Indicator

### 4.2.1. Analytic Hierarchy Process (AHP)

The analytic hierarchy process, abbreviated as AHP, has been proposed more than 30 years ago. The method is, in short, to simplify a complex issue by decomposing it into simple parts. Thus, the original goal can be divided step by step into several goals or criterion which are easy to deal with.For example, in a comprehensive evaluation, the original evaluation goal is decomposed into several evaluation indicators or criterion. That is, the original evaluation goals can be split into some aspects from which the goals can be evaluated better. As for decomposed indicators or criterion, they still can be decomposed into more simple ones. Finally, a complex evaluation goal is decomposed into several simple indicators of evaluation and completed through the final evaluation indicators.The analytic hierarchy process falls into 3 parts including the object layer, criterion layer, and scheme layer. Their hierarchical models are shown Figure2.Figure 2
Schematic diagram of the hierarchical model.The steps of the analytic hierarchy process to determine the weight in the evaluation indicator of undergraduates’ cultivation quality:(1) Establish a Hierarchical Indicator System for the Evaluation on Undergraduates’ Cultivation Quality. In order to establish a hierarchical model of the evaluation indicator system on undergraduates’ cultivation quality, an in-depth analysis should be made in evaluation issues and an evaluation indicator system is established to determine the object layer and criterion layer, among which the criterion layer is analyzed to determine the level and the subordinate relationship of the criterion layer.(2) Construct a Judgment Matrix. In this paper, the indicators are analyzed through the way to establish an indicator system for the evaluation on undergraduates’ cultivation quality and the importance of indicators is determined by the corresponding scale comparison table, and also, a judgment matrix can be constructed according to the importance of indicators which are confirmed. The primary diagonal element of the matrix is 1, and those elements which are symmetric with the primary diagonal are reciprocal, and their all values are greater than 1.(3) The Consistency Test of the Constructed Judgment Matrix. The judgment matrix can be regarded as qualified when the consistency is tested to the criterion; on the contrary, it needs to be modified. For example, if the importance of indicator A for indicator B is 3, B for C is 2; so, what is A for C? 6 should be worked out from the condition above. However, actually, the situation is inconsistent that the relative importance given is 4. Of course, it is difficult to keep consistent completely in practical application and excessive consistency would influence the scientificity in analyzing the indicators, even in judging the indicators subjectively from consistency, rather than the importance of the indicator in practice. Thus, the judgment matrix only needs to meet a certain consistency, that is, the matrix is reasonable and can be used if it passes the consistency test. As for the one which fails to pass, it cannot be accepted until it meets the consistency by reanalyzing indicators and confirming relative importance and reperforming the test.(4) Hierarchical Single Sorting. Hierarchical single ranking refers to ranking the weights of the indicators at the upper level according to the relative importance of the indicators at the current level. That is to sort the weight values corresponding to each index calculated by the judgement matrix.(5) Hierarchical Total Sorting. Hierarchical total sorting should be based on the single sorting result and combined with the weight of the upper index and able to meet the needs of the consistency test. The hierarchical total sorting result is the final weight ranking result, and the weight determined by the hierarchical total ranking is evaluated.(6) Evaluation. The final evaluation result should be determined by using the corresponding data of the evaluation index, combining with the final weight of the index to make a general evaluation by referring to the actual situation and the final score.In general, the analytic hierarchy process (AHP) is divided into six steps. Firstly, it is establishing the index system and analyzing the relationship between each level of indicators and the lower level.The next step is analyzing the importance of the indicators relative to the target level and the upper level. According to the comparison table of the AHP scale, the importance degree is obtained by comparing the two indexes. The next step is constructing the judgment matrix, carrying on the consistency test and hierarchical single and total sort. Finally, the evaluation and the analysis to the evaluation result should be given.The flow chart of AHP evaluation is shown in Figure3.Figure 3
Flow chart of analytic hierarchy process.The advantages and limitations of the analytic hierarchy process can be seen in the process of applying it to the weight setting of the indicators in the college student training quality evaluation index system.The analytic hierarchy process is a systematic analysis method that determines the weight of the college student training quality index system through the evaluators’ judgment and quantitative analysis on the college quality evaluation index system. The method is simple and practical with strict mathematical derivation. The mathematical knowledge used in the process is simple and easy to master, and a simplified method is applied to the calculation of the matrix, which makes the derivation simpler and more efficient. What is more, the weight is determined not with a large amount of data and information. The evaluator analyzes the evaluation target and then makes a judgment on the importance of the index, which is in line with people’s thinking habits.


### 4.2.2. Entropy Method

The entropy method is a kind of objective weighting method. Entropy is originally a thermodynamic concept, which reflects the degree of disorder in the system. Entropy was first introduced by Claude Elwood Shannon from thermodynamics to information theory. Information entropy bears some general features of thermodynamics, but it is a broader concept that has been widely used in many fields. Information entropy is a measure of the degree of disorder. The greater the information entropy, the higher the degree of disorder, and the less information is included. The entropy method is to calculate the information entropy of each indicator based on the data value of the indicator and then determine the weight of the indicator. The flow chart of the entropy method for weighting indicators is shown in Figure4.Figure 4
Flow chart of the entropy weight method.The entropy method is simple for determining the index weight. The information entropy and indicator weights are obtained by processing the indicator data, which effectively avoids the problem of insufficient objective weights caused by human subjective factors. However, the entropy method completely relies on the index data to determine the index weight, and then, the weight is obtained through calculation. The evaluation result may not meet the subjective needs of the evaluator, making it difficult for the evaluator to understand and explain.


## 4.2.1. Analytic Hierarchy Process (AHP)

The analytic hierarchy process, abbreviated as AHP, has been proposed more than 30 years ago. The method is, in short, to simplify a complex issue by decomposing it into simple parts. Thus, the original goal can be divided step by step into several goals or criterion which are easy to deal with.For example, in a comprehensive evaluation, the original evaluation goal is decomposed into several evaluation indicators or criterion. That is, the original evaluation goals can be split into some aspects from which the goals can be evaluated better. As for decomposed indicators or criterion, they still can be decomposed into more simple ones. Finally, a complex evaluation goal is decomposed into several simple indicators of evaluation and completed through the final evaluation indicators.The analytic hierarchy process falls into 3 parts including the object layer, criterion layer, and scheme layer. Their hierarchical models are shown Figure2.Figure 2
Schematic diagram of the hierarchical model.The steps of the analytic hierarchy process to determine the weight in the evaluation indicator of undergraduates’ cultivation quality:(1) Establish a Hierarchical Indicator System for the Evaluation on Undergraduates’ Cultivation Quality. In order to establish a hierarchical model of the evaluation indicator system on undergraduates’ cultivation quality, an in-depth analysis should be made in evaluation issues and an evaluation indicator system is established to determine the object layer and criterion layer, among which the criterion layer is analyzed to determine the level and the subordinate relationship of the criterion layer.(2) Construct a Judgment Matrix. In this paper, the indicators are analyzed through the way to establish an indicator system for the evaluation on undergraduates’ cultivation quality and the importance of indicators is determined by the corresponding scale comparison table, and also, a judgment matrix can be constructed according to the importance of indicators which are confirmed. The primary diagonal element of the matrix is 1, and those elements which are symmetric with the primary diagonal are reciprocal, and their all values are greater than 1.(3) The Consistency Test of the Constructed Judgment Matrix. The judgment matrix can be regarded as qualified when the consistency is tested to the criterion; on the contrary, it needs to be modified. For example, if the importance of indicator A for indicator B is 3, B for C is 2; so, what is A for C? 6 should be worked out from the condition above. However, actually, the situation is inconsistent that the relative importance given is 4. Of course, it is difficult to keep consistent completely in practical application and excessive consistency would influence the scientificity in analyzing the indicators, even in judging the indicators subjectively from consistency, rather than the importance of the indicator in practice. Thus, the judgment matrix only needs to meet a certain consistency, that is, the matrix is reasonable and can be used if it passes the consistency test. As for the one which fails to pass, it cannot be accepted until it meets the consistency by reanalyzing indicators and confirming relative importance and reperforming the test.(4) Hierarchical Single Sorting. Hierarchical single ranking refers to ranking the weights of the indicators at the upper level according to the relative importance of the indicators at the current level. That is to sort the weight values corresponding to each index calculated by the judgement matrix.(5) Hierarchical Total Sorting. Hierarchical total sorting should be based on the single sorting result and combined with the weight of the upper index and able to meet the needs of the consistency test. The hierarchical total sorting result is the final weight ranking result, and the weight determined by the hierarchical total ranking is evaluated.(6) Evaluation. The final evaluation result should be determined by using the corresponding data of the evaluation index, combining with the final weight of the index to make a general evaluation by referring to the actual situation and the final score.In general, the analytic hierarchy process (AHP) is divided into six steps. Firstly, it is establishing the index system and analyzing the relationship between each level of indicators and the lower level.The next step is analyzing the importance of the indicators relative to the target level and the upper level. According to the comparison table of the AHP scale, the importance degree is obtained by comparing the two indexes. The next step is constructing the judgment matrix, carrying on the consistency test and hierarchical single and total sort. Finally, the evaluation and the analysis to the evaluation result should be given.The flow chart of AHP evaluation is shown in Figure3.Figure 3
Flow chart of analytic hierarchy process.The advantages and limitations of the analytic hierarchy process can be seen in the process of applying it to the weight setting of the indicators in the college student training quality evaluation index system.The analytic hierarchy process is a systematic analysis method that determines the weight of the college student training quality index system through the evaluators’ judgment and quantitative analysis on the college quality evaluation index system. The method is simple and practical with strict mathematical derivation. The mathematical knowledge used in the process is simple and easy to master, and a simplified method is applied to the calculation of the matrix, which makes the derivation simpler and more efficient. What is more, the weight is determined not with a large amount of data and information. The evaluator analyzes the evaluation target and then makes a judgment on the importance of the index, which is in line with people’s thinking habits.


## 4.2.2. Entropy Method

The entropy method is a kind of objective weighting method. Entropy is originally a thermodynamic concept, which reflects the degree of disorder in the system. Entropy was first introduced by Claude Elwood Shannon from thermodynamics to information theory. Information entropy bears some general features of thermodynamics, but it is a broader concept that has been widely used in many fields. Information entropy is a measure of the degree of disorder. The greater the information entropy, the higher the degree of disorder, and the less information is included. The entropy method is to calculate the information entropy of each indicator based on the data value of the indicator and then determine the weight of the indicator. The flow chart of the entropy method for weighting indicators is shown in Figure4.Figure 4
Flow chart of the entropy weight method.The entropy method is simple for determining the index weight. The information entropy and indicator weights are obtained by processing the indicator data, which effectively avoids the problem of insufficient objective weights caused by human subjective factors. However, the entropy method completely relies on the index data to determine the index weight, and then, the weight is obtained through calculation. The evaluation result may not meet the subjective needs of the evaluator, making it difficult for the evaluator to understand and explain.


## 4.3. Improvement of the Weighting Method

By organically combining the subjective weighting method and the objective weighting method, the weights can be optimized, so that the weights can reflect the wishes of the evaluator and can be explained without violating the objectivity of the results. Since the objective weighting method is the result of weighting calculated through indicator data, the weighting depends on the data and is objective.This article adopts the linear combination method and introduces a distance function to determine the coefficient of linear combination. The subjective weight and the objective masses are multiplied by different coefficients and summed to obtain the improved combined weight. The flow chart of the combined weighting calculation method is shown in Figure5.Figure 5
Flow chart of the combined weighting method.Based on the Delphi method, the features of the analytic hierarchy process, and the number of indicators in this article, the analytic hierarchy process is first used to subjectively weigh the indicators, and then, the entropy method is adopted to objectively weigh the indicators, and finally, the weights obtained through the two methods is combined to gain the final weight.


## 4.4. AHP(Analytic Hierarchy Process) Weights

(1)
The evaluation target is denoted asA, the evaluation indexB=B1,B2,B3,⋯,Bn, the relative importance ofBi, andBjto the evaluation target ABij; the value of which is an integer from 1 to 9; for details, see Table 3Table 3
Decision matrix scale table.
ScaleImplication1The two indicators are equally important3The former is slightly more important than the latter5The former is obviously more important than the latter7The former is more important than the latter9The former is extremely important than the latter2,4,6,8The two intermediate values in the abovementioned comparisonReciprocalThe ratio of indexi to j is Bij; then, the ratio of j to i is 1/BijThe judgment matrix of the evaluation target isA as shown in formula (1).
(1)A=b11b12⋯b1nb21b22b2n⋮⋱⋮bn1bn2⋯bnn.(2)
Formula (2) can be used to quantize the column items of matrix A and normalize b¯ijto be the normalized matrix elements(2)b¯ij=bij∑j=1nbij,j=1,2,3,⋯,n(3)
The normalized matrix is summed by row vectors to obtain column vectorW as shown in formula (3) as follows:(3)W=wi,i=1,2,3,⋯,n,wi=∑jnbij,i=1,2,3,⋯,n(4)
Normalization of column vectorW is shown in formula (4) as follows:(4)w¯i=wi∑i=1nwi,i=1,2,3,⋯,n,where feature vector W¯=w¯i,i=1,2,3,⋯,n is the weight vector of each index
(5)
The step is a consistency checkThe consistency index is calculated from the combination of formula (6) and the maximum characteristic root λmax calculated by formula (5) as follows.
(5)λmax=∑inAW¯inW¯i,(6)CR=CIRI.The conformance ratio CR was calculated by formula (7) as follows.
(7)CR=CIRI.RI is a random consistency indicator, which can be obtained by looking up the table. Table4 lists the consistency indicators. N is the index number.
(6)
Total weight and total consistency test of indicatorsTable 4
Random consistency index.
n1234567891011RI000.580.901.121.241.321.411.451.491.51According to the above steps, the weight of the first-level indicator iswiQB, the weight of the second-level indicator C to the first-level indicator B is wijBC, and the weight of the second-level indicator to Q is wij′, which can be calculated by the following formula.
(8)wij′=wiQB×wijBC.The consistency ratio of indicatorC to Q, CRz, is calculated from (8) and satisfies CRz<0.1.
(9)CRz=∑wiQBCIi∑wiQBRIi.RIi and CIi are the random consistency index and consistency index of secondary index C to Bi, respectively; for details, see Table 4.IfCR<0.1, the weight corresponding to the determined indicator will be in accord with the requirement of consistency.


## 4.5. Entropy Weighting Method

According to the index system already established in this paper, the secondary indicator is used for evaluating the cultivation quality of undergraduates. The number of indicator ism and the number of students is n to construct the matrix R as shown in (10) as follows.
(10)R=rijnxm=r11r12⋯r1mr21r22r2m⋮⋱⋱rn1rn2⋯rnm.(1)
The matrixR is normalized according to formula (11)(11)Rij=rij−minr1j,⋯,rnjmaxr1j,⋯,rnj−minrij,⋯,rnj,where the matrix R has been normalized as shown in formula (12) as follows:
(12)R¯=Rijnxm=R11R12⋯R1mR21R22R2m⋮⋱⋮Rn1Rn2⋯Rnm(2)
The proportionpij of the indicator j of the sample i should be calculated according to formula (13) as follows:(13)pij=Rij∑i=1nrij,i=1,2,⋯,nj=1,2,⋯,m(3)
Calculate the entropyej of the indicator j according to formula (14) as follows:(14)ej=−1Inn∑i=1npijInpij,j=1,⋯,m(4)
Calculate the weight w of the indicatorj according to formula (15) as follows:(15)wj=1−ej∑j=1m1−ej,where 1−ej is the discrepancy coefficient of the indicator j


## 4.6. Improved Weighting Methods to Calculate Weights

In this paper, the weighting method is improved by using the distance function to determine the coefficient of the combination weight. If the combination weight iswc, wi′ and wi″ are the weights obtained by the subjective weighting method and the objective weighting method, respectively; at the same time, α and β are coefficients; then, the combination weights are shown in formula (16) as follows:
(16)Wc=∝wi′+βwi′′.Suppose that the distance function of the subjective weighting method and the objective weighting method isdwi′,wi′′ and the distance function can be expressed as follows:
(17)dwi′,wi′′=12∑i=1nwi′−wi′′21/2.In order to ensure that the degree of weight difference is consistent with that of coefficients, letα−β2 and dwi′,wi′′ be equal to obtain formula (18):
(18)α−β2=dwi′,wi′′.The weighting distribution coefficients fit in with the formula:(19)α+β=1.Simultaneous formulas can be expressed as follows:(20)α−β2=dwi′,wi′′,α+β=1.The weight distribution coefficientsα and β can be obtained and substituted into the combined weighting expression to obtain the combined weight Wc.


## 5. Data Processing

The experimental data is evaluated in this article with the data of all students from 2013 to 2014 in the major of agricultural water conservancy engineering. The data generated by the students during their studies at school are collected from the following aspects, so as to obtain the original data of the corresponding indicators.According to the analysis of the observation points in the evaluation index system of college students’ training quality, there are five sources of data. Academic performance and practical results can be obtained from the educational administration system, and the category and quantity of books to be read can be obtained from the library management system. The ideological quality of students can be obtained through questionnaire surveys of teachers’ evaluations of students and the results of students’ mutual evaluations. The competition and awards are from the college’s annual awards and scientific research result evaluation survey. Finally, 262 pieces of data were collected and invalid data is manually deleted.


### 5.1. Combined Weight

According to formula (2)–(20), α=0.56 and β=0.44 can be calculated to obtain the combination weight WC of the indicator, as shown in Table 5.Table 5
Combination weights.
Second level indicatorsWeightWCHumanities and social sciences0.062974Mathematics, physics, and computer knowledge0.055505Foreign language knowledge0.061202Profession knowledge0.059634Acquisition knowledge0.094187Applied knowledge0.072923Knowledge expression0.072443Communication0.05155Innovation0.147197Entrepreneurship0.066417Ideological quality0.085431Cultural quality0.038563Physical and mental quality0.104138Profession quality0.048537


### 5.2. Sample Category Annotation

The student’s final scoreSQ is calculated by the following formula (21), where Sij is the index value and Wc is the combined weight.
(21)SQ=∑SijWc.Mark the data according to the final scoreSQ and divide the students into A, B, C, and D. Mark the level L, which is calculated by formula (22) as follows:
(22)L=A90≤SQ,B80≤SQ<90,C80≤SQ<90,DSQ<70.


## 5.1. Combined Weight

According to formula (2)–(20), α=0.56 and β=0.44 can be calculated to obtain the combination weight WC of the indicator, as shown in Table 5.Table 5
Combination weights.
Second level indicatorsWeightWCHumanities and social sciences0.062974Mathematics, physics, and computer knowledge0.055505Foreign language knowledge0.061202Profession knowledge0.059634Acquisition knowledge0.094187Applied knowledge0.072923Knowledge expression0.072443Communication0.05155Innovation0.147197Entrepreneurship0.066417Ideological quality0.085431Cultural quality0.038563Physical and mental quality0.104138Profession quality0.048537


## 5.2. Sample Category Annotation

The student’s final scoreSQ is calculated by the following formula (21), where Sij is the index value and Wc is the combined weight.
(21)SQ=∑SijWc.Mark the data according to the final scoreSQ and divide the students into A, B, C, and D. Mark the level L, which is calculated by formula (22) as follows:
(22)L=A90≤SQ,B80≤SQ<90,C80≤SQ<90,DSQ<70.


## 6. Model Training and Test

Python language and machine learning framework scikit-learn are used to edit the code, and the labeled 160 sample data is used for model training.The final error rate output of the AdaBoost-SVM model is 0.09, that is, the accuracy rate isO. 91. The training results of the AdaBoost-SVM model are shown in Figure 6 as follows.Figure 6
Output error rate of AdaBoost-SVM.According to the running results, when the number of component learners is 7, the error rate of the model in the test set and training set does not change. That is, when seven base learners are constructed, the error rate of the model reaches the optimal result under the given parameters. When the curve is balanced, the error rate of the model on the test set is 0.09. After increasing the number of base classifiers, the curve does not change. So, the number of base classifiers is 7.The AdaBoost-SVM evaluation model was tested according to the sample data with the exception of the training data from the student data samples of agricultural and water conservancy engineering. During the process, four sets of data were used, each with 25 samples. The test result contingency table is shown in Table6 as follows.Table 6
Contingency table.
Student categorySample sizeTest resultABCDA2521220B2502410C2501231D2500223In the abovementioned contingency table, there are four categories of student sample data and each category has 25 samples to test the model. After inputting the model, the number of correctly classified students inA is 21 and the number of incorrectly classified students is 4, among which, two are mistakenly classified into category B, and two samples are incorrectly classified into category C. Twenty-four of the B students were correctly classified, and one sample was incorrectly classified into C. Among the C students, 23 samples were correctly classified and the other two samples were incorrectly classified into B and D. The number of correctly classified samples in the sample of D students is 23, the number of samples incorrectly classified is 2, and they are classified into class C incorrectly.Suppose that the accuracy rate on the test setT is ACC and the accuracy rate can be calculated by formula (23).
(23)acc=1T∑x∈TIc^x=cx,whereI. is an indicative function and the parameter value is of the Boolean type. When the value is true, the function value is 1, and when the parameter value is false, the value is 0.The accuracy rate can be obtained from the results in the contingency table:(24)acc=1T∑x∈TIc^x=cx=0.91.It can be concluded that the AdaBoost-SVM evaluation model has good performance in accuracy and recall and its accuracy rate reaches 0.91. After the training, the model can be easily used for student evaluation, and compared with the analytic hierarchy process, it can avoid a lot of tedious work of processing data.


## 7. Conclusion

This paper researches the current situation of university students’ cultivation quality evaluation at home and abroad and finds that there are problems of incomplete creation of indicators, unreasonable assignment of indicator weights, and inefficient evaluation methods in the current university students’ cultivation quality evaluation. Therefore, the research in this paper improves the evaluation of college student cultivation quality from three aspects: cultivation quality evaluation index system, assignment method, and evaluation method. This paper mainly does the following works: (1) The study of the index system of college students’ cultivation quality evaluation establishes a comprehensive index system of college students’ cultivation quality evaluation from three aspects of college students’ knowledge, ability, and quality and establishes the index system of college students’ cultivation quality evaluation consisting of 3 primary indexes, 14 secondary indexes, and 40 index observation points. (2) Optimize the assignment method of index weights, analyze the advantages and disadvantages of the subjective assignment method and objective assignment method, and adopt the combination of the subjective and objective assignment method. The combination of subjective and objective weighting methods is used. The combination weighting method is used to determine the weights, which can make the weights have good interpretation and reduce the influence of subjective factors. The distance function is used to determine the linear combination coefficients of the combination weights. (3) The evaluation method is analyzed by introducing a machine learning algorithm as the evaluation method. Since the support vector machine performs well on small datasets and there is still room for improvement in accuracy, this paper selects the support vector machine and integrates it with the AdaBoost algorithm to further improve the accuracy and generalization ability. Since AdaBoost requires a relatively low accuracy as its base learner and SVM alone has a high performance, this paper selects Gaussian kernel function and adjusts the parameters of the support vector machine to meet the accuracy requirements of the AdaBoost algorithm when using the support vector machine as the base learner of AdaBoost algorithm. To improve the performance of the algorithm, a diversity measure is introduced to ensure that the base learner has good diversity.



---

*Source: 8022386-2022-04-07.xml*