# Multilevel Evaluation of Teaching Quality in Higher Education Using Single-Valued Neutrosophic Set

Authors: Fuzhong Wu; Yiming Fang
Journal: Mobile Information Systems
 (2022)

---

## Abstract

Evaluation of teaching quality is essential for teachers’ promotion, students’ course selection, and institutes’ standing. A multilevel evaluation framework for teaching quality in higher education was investigated by combining the Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) with the single-valued neutrosophic set (SVNS). An indicator system was constructed, including the teaching performance and the students’ learning outcomes. For the qualitative indicator values, an SVNS representation method was proposed, aiming to describe the uncertainty and improve the credibility and validity of the evaluation. Then, both the qualitative data and quantitative data were applied to the TOPSIS-based multilevel evaluation framework, which consisted of an overall assessment and five specific evaluations. The former assessment would provide a final rank and determine the best lecturers to be given priority in awards and promotion. The latter would focus on identifying areas where the lecturers could do better and would give them tips to overcome their challenges. Finally, a descriptive example was provided to verify the proposed framework and demonstrate its practicality.


---

## Body

## 1. Introduction

Assessment of teaching quality in higher education is a topic of discussion worldwide. An accurate evaluation of teaching quality can be beneficial to decision-making and help promote educational outcomes. It also has a significant impact on employment, tenure, and promotion of lecturers. However, presently the process of evaluation is complex and challenging, and there are no widely accepted methods [1]. Additional efforts are required to find better ways to address this problem.Spooren et al. provided an extensive review of the high impact studies [2].The construction of the assessment instrument is a significant point of discussion. Baghdadi attempted to achieve the betterment of teaching quality by optimizing the choice of valid indicators to evaluate the teaching quality. Based on the analysis of an exploratory questionnaire, a new indicator system was proposed, which included four factors. It was claimed that the new indicators would enable students to evaluate teaching in terms of their learning progress [3]. Matosas-López et al. analyzed the limitations of the current evaluation systems in assessing the university teachers in blended learning modalities and constructed an assessment indicator system with behavioral scales. The proposed indicator system highlighted the teaching aspects of blended learning models and reinforced the formative purpose of assessments [4].Another important issue is the methods and tools employed for efficient assessment. Nurdin et al. reported a new way to evaluate teaching quality using an online mobile system. An android system was developed and applied in an Islamic higher education institution. The results showed that the online mobile system could increase the participation rate in the assessment of teaching quality [5]. Esmael refuted the unreliability of the online teaching evaluation by investigating the difference between the manual course evaluation and online assessment. Totally, 4678 students and 180 courses were utilized for the assessments. The results showed no significant difference between the feedback received on the manual and online assessments; therefore, the use of the online teaching evaluation continued to be encouraged [6]. Literature [7] focused on the problem of assessing teaching quality in the massive open online courses (MOOC) modality. The authors construed a Hadoop platform and developed a video player to collect and calculate the learning time. Based on these, a naive Bayesian model was designed and applied to the collected data. Finally, the learning cost coefficient was obtained to describe the teaching quality.Although previous works have achieved success in various applications, we found that there were some issues worthy of in-depth research. First, most existing methods put too much emphasis on teachers’ attributes, such as their academic level, teaching ability, and teaching skills. The students’ learning outcomes were often excluded from those indicators that were used to evaluate teaching quality [8]. As a result, the evaluation results were not essentially credible, and high teaching quality was not intrinsically related to higher learning outcomes. Outcome-based education (OBE), put forward by Spady [9], is a new and widely used education model. The changes introduced by OBE emphasized what the students learned rather than what they were taught. Everything in an educational system should be organized to ensure that students achieve the expected goals [10]. According to this theory, the students’ learning outcomes should be considered in evaluating the teaching quality [11].Another significant issue was the method to deal with the qualitative indicator values. Unlike quantitative data, qualitative data often mirror the pattern of how people think and make judgments. Linguistic variables are usually employed to express qualitative data rather than numbers. The values are based on opinions, feelings, or viewpoints. There exists much incomplete and uncertain information [12]. During the past decades, many approaches have been investigated to deal with the qualitative evaluation results, aiming to improve the validity and credibility of teaching quality evaluation [13]. Among them, fuzzy-based methods seem to be an appealing tool. It has been widely accepted that reasoning based on fuzzy approaches provides an attractive way to deal with imprecise data [14]. Motivated by this, Li established a fuzzy model to evaluate the quality of teaching English in colleges, based on the analytic hierarchy process (AHP) method and Grey system theory. The author claimed the model could effectively assess the quality of English teaching, and the evaluation results, in turn, greatly promoted the quality of teaching English in colleges [15]. Similarly, a framework based on the multiattribute fuzzy measurement model was investigated to assess the teaching quality of DanceSport Major. The model was also believed to offer an excellent assurance to improve the teaching quality of DanceSport Major [16]. Recently, neutrosophic set (NS), defined as the generalization of interval fuzzy sets, has been used extensively in many fields, such as medical image processing, decision-making, and social judgments. It has been widely accepted that the NS-based technique supplies an effective tool for handling inconsistent and vague data [17–21]. Motivated by this, NS was increasingly employed to represent the qualitative indicator values in various fields, aiming for improving the credibility of the applications.The last concern was the use of the evaluation result. Most of the existing studies provided only the final rank of the teaching quality, for each lecturer. In literature [22], the authors combined the Grey correlation with the TOPSIS to provide a modified Grey-TOPSIS method and applied it to evaluate the teaching quality in five colleges. The main disadvantage was that the evaluation only provided a final rank for the five colleges. The evaluated colleges received no valuable feedback and could not benefit from the assessment. We believe that if the problems causing the poor ranks could be identified, the evaluation would be more beneficial. Therefore, we struggled to propose a multilevel evaluation framework, which not only provided the final grade but also identified problems and provided timely feedback for each lecturer.The main contributions of this study are as follows: (1) an evaluation indicator system was constructed, which covers the teaching performance and the students’ learning outcomes and provides a strong foundation for efficient assessment. (2) The qualitative indicator values were analyzed from the SVNS perspective, which would be useful to describe the uncertainty of the qualitative data and to improve the credibility and validity of the evaluation. (3) A multilevel evaluation framework of teaching quality in higher education was proposed, which could not only provide a final rank for the lecturers but also timely feedback, which would be beneficial to improve their teaching activities.The remainder of this paper is organized as follows: Section2 provides a brief introduction of the techniques utilized in this study. Section 3 discusses the multilevel evaluation method using SVNS and TOPSIS. Section 4 discusses a typical case to illustrate how to apply the evaluation approach followed by the conclusion in Section 5.


## 2. Preliminaries

### 2.1. Definition of NS

NS was proposed by Smarandache [18]. Each element of NS has a degree of truth, indeterminacy, and falsity, which are independent. Let X denote a universe of discourse and x∈X. The NS A can be defined by a truth membership function TAx, an indeterminacy membership function IAx, and a falsity membership function FAx. The definition can be depicted as follows:
(1)A=X,TAx,IAx,FAx∣x∈XTAx, IAx, and FAx are real standard or real nonstandard subsets of ]0−,1+[, i.e.,
(2)TAx:X⟶0−,1+,IAx:X⟶0−,1+,FAx:X⟶0−,1+.There exist no restrictions on the sum of the functions ofTAx, IAx, and FAx; therefore,
(3)0−≤supTAx+supIAx+supFAx≤3+,where sup∙ indicates the supremum operation of a set.


### 2.2. Single-Valued Neutrosophic Set

Single-valued neutrosophic set (SVNS) is a case of NS. It was presented by Wang et al. to handle indeterminate, inconsistent, and incomplete information [23]. The interval [0, 1] was used instead of ]0−,1+[ to achieve better representation and application to practical issues. The SVNS B can be constructed as follows:
(4)B=X,TBx,IBx,FBx∣x∈X,where
(5)TBx:X⟶0,1,IBx:X⟶0,1,FBx:X⟶0,1.And the limitation of the three membership functions changes to be(6)0≤TBx+IBx+FBx≤3.The systematical operational rule for SVNS can be found in literature [20, 21]. In this study, we focused on the application of SVNS in processing the evaluation data of teaching quality.


### 2.3. TOPSIS Ranking Method

TOPSIS was introduced by Hwang and Yoon. According to this technique, the ideal point denotes the one consisting of all the best indicator values possible, and the point that consists of all the worst indicator values possible is called the negative-ideal point. The solution, which approaches the ideal point and is distant from the negative-ideal point, is usually ranked as a better one [24].LetI+ denote the ideal point and I− represent the negative-ideal point. The closeness coefficient of the solution k, which can be used to describe its superiority, is defined in Equation (7). The alternatives having a higher closeness coefficient would be considered as the superior ones.
(7)μk=WDk−WDk++WDk−,where WDk+ is the distance from the solution k to the ideal point I+ and WDk− denotes the distance from the solution k to the negative-ideal point I−.


## 2.1. Definition of NS

NS was proposed by Smarandache [18]. Each element of NS has a degree of truth, indeterminacy, and falsity, which are independent. Let X denote a universe of discourse and x∈X. The NS A can be defined by a truth membership function TAx, an indeterminacy membership function IAx, and a falsity membership function FAx. The definition can be depicted as follows:
(1)A=X,TAx,IAx,FAx∣x∈XTAx, IAx, and FAx are real standard or real nonstandard subsets of ]0−,1+[, i.e.,
(2)TAx:X⟶0−,1+,IAx:X⟶0−,1+,FAx:X⟶0−,1+.There exist no restrictions on the sum of the functions ofTAx, IAx, and FAx; therefore,
(3)0−≤supTAx+supIAx+supFAx≤3+,where sup∙ indicates the supremum operation of a set.


## 2.2. Single-Valued Neutrosophic Set

Single-valued neutrosophic set (SVNS) is a case of NS. It was presented by Wang et al. to handle indeterminate, inconsistent, and incomplete information [23]. The interval [0, 1] was used instead of ]0−,1+[ to achieve better representation and application to practical issues. The SVNS B can be constructed as follows:
(4)B=X,TBx,IBx,FBx∣x∈X,where
(5)TBx:X⟶0,1,IBx:X⟶0,1,FBx:X⟶0,1.And the limitation of the three membership functions changes to be(6)0≤TBx+IBx+FBx≤3.The systematical operational rule for SVNS can be found in literature [20, 21]. In this study, we focused on the application of SVNS in processing the evaluation data of teaching quality.


## 2.3. TOPSIS Ranking Method

TOPSIS was introduced by Hwang and Yoon. According to this technique, the ideal point denotes the one consisting of all the best indicator values possible, and the point that consists of all the worst indicator values possible is called the negative-ideal point. The solution, which approaches the ideal point and is distant from the negative-ideal point, is usually ranked as a better one [24].LetI+ denote the ideal point and I− represent the negative-ideal point. The closeness coefficient of the solution k, which can be used to describe its superiority, is defined in Equation (7). The alternatives having a higher closeness coefficient would be considered as the superior ones.
(7)μk=WDk−WDk++WDk−,where WDk+ is the distance from the solution k to the ideal point I+ and WDk− denotes the distance from the solution k to the negative-ideal point I−.


## 3. Methodology

### 3.1. Indicator Selection

The selected indicators for the evaluation of teaching quality are shown in Table1, which shows that this indicator system covered the aspects regarding teaching performance and those concerning the students’ learning outcomes. This would make the evaluation complete and provide more information on the teaching quality.Table 1
Selected indicators for the teaching quality evaluation.
AspectIndicatorDescriptionTypeTeachers’ attitudeC1Class punctualityQuantitativeC2Dismissing the class earlyQuantitativeC3Number of times the class schedule was changedQuantitativeC4Degree of concentration on teachingQualitativeC5Efforts to monitor students’ learningQualitativeStudents’ responseC6Number of teacher-student interactionsQuantitativeC7Number of students looking at the blackboardQuantitativeTeaching contentC8Consistency with syllabusQualitativeC9NoveltyQualitativeTeaching abilityC10Familiarity with teaching contentQualitativeC11Suitability of teaching methodsQualitativeC12Effectiveness in getting students’ attentionQualitativeLearning outcomeC13Degree of achieving learning goalsQualitativeC14Improvement of the learning abilityQualitativeIn addition, scientific methods were also developed to collect indicator values. For the quantitative indicators, the assessment data were imported from the Center for Teaching Quality Assessment of the university. We believe that this is the most authoritative way to collect the indicator values. For the qualitative ones, questionnaires were employed to collect the assessment of experts and students. The questionnaires contained all the qualitative indicators listed in Table1. Nine answer options were provided for each item. These were as follows: extremely good/high, very good/high, good/high, medium good/high, medium/fair, medium bad/low, bad/low, very bad/low, and extremely bad/low. Experts and the students who took the evaluated courses would be invited to make their choices.


### 3.2. Input Data Processing

#### 3.2.1. Tendency Treatment

According to the principle of TOPSIS, the values of the cost indicators were first transformed into beneficial values. In this study, the original values of the cost indicatorsC1, C2, and C3were applied to Equation (8), and the corresponding benefit values were obtained.
(8)Cik,P=maxkCik−CikmaxkCik−minkCik,where Cik denotes the original value of the indicator Ci for the lecturer k and Cik,P is the obtained beneficial value of Cik.


#### 3.2.2. Normalization for Quantitative Data

To avoid comparing indicator values on different scales, all the quantitative data are normalized according to Equation (9).
(9)NCik=Cik∑kCik2,where NCik is the normalization result of the original value Cik.


#### 3.2.3. Quantification for Qualitative Data

The input data, expressed in a nonnumerical way, was quantified through the SVNS representation technique. Table2 shows the SVNS representation for the fuzzy linguistic variables.Table 2
SVNS representation of the fuzzy linguistic variables.
Linguistic term (x)SVNS<TBx,IBx,FBx>Extremely good/high<0.99, 0.01, 0.01>Very good/high<0.90, 0.10, 0.10>Good/high<0.80, 0.20, 0.15>Medium good/high<0.70, 0.30, 0.30>Medium/fair<0.50, 0.50, 0.50>Medium bad/low<0.30, 0.65, 0.60>Bad/low<0.20, 0.75, 0.80>Very bad/low<0.10, 0.90, 0.90>Extremely bad/low<0.01, 0.99, 0.99>Suppose there wereT evaluators (including experts and students) providing their feedback for each lecturer regarding each qualitative indicator. The SVNS weight aggregation operator was utilized to aggregate the different representations given by different evaluators according to the given weights λ. Equation (10) illustrates the aggregation operation.
(10)FCik=λ1FCik,1⨁λ2FCik,2⨁⋯⨁λTFCik,T=1−∏t=1T1−NTik,tλt,∏t=1TNIik,tλt,∏t=1TNFik,tλt,whereFCik,t is the SVNS representation of the indicator Ci for the kth lecturer provided by the tth(t=1,2,⋯,T) evaluator. FCik denotes the aggregated indicator value.


### 3.3. Multilevel Evaluation Framework

In this study, a multilevel evaluation framework was proposed using TOPSIS technique. It consisted of an overall evaluation and five specific evaluations, as shown in Figure1.. The overall evaluation was performed using all the indicators listed in Table 1 and provided a credible rank for each evaluated lecturer. The specific evaluations only used the particular indicators of one aspect. Their functions were to identify areas for improvement and individualizing professional development.Figure 1
Multilevel evaluation framework.Suppose there wereM quantitative indicators involved in an evaluation. Let DI+ denote the quantitative subset of the ideal point I+ and DI− represent the quantitative subset of the negative-ideal point I−. The definition of DI+ and DI− were described as Equations (11) and (12).
(11)DI+=Qm+=maxkNCmk,m=1,2,⋯M,(12)DI−=Qm−=minkNCmk,m=1,2,⋯M.Correspondingly, the quantitative parts of the distancesWDk+ and WDk− can be expressed as Equations (13) and (14).
(13)DDk+=∑m=1Mwm∙Qm+−NCmk2,(14)DDk−=∑m=1Mwm∙Qm−−NCmk2,where wm is the weight for the mth indicator and can be determined using the method of the SVNS cross-entropy [25].Similarly, suppose there wereN qualitative indicators involved in the evaluation.FI+ and FI− were the qualitative subsets of I+ and I−, respectively. Their definitions were as Equations (15) and (16).
(15)FI+=Qn+=NTn+,NIn+,NFn+=maxkNTnk,minkNInk,minkNFnk,n=1,2,⋯N,(16)FI−=Qn−=NTn−,NIn−,NFn−=minkNTnk,maxkNInk,maxkNFnk,n=1,2,⋯N.The qualitative part of the distancesWDk+ and WDk− can be expressed as Equations (17) and (18).
(17)FDk+=13N∑n=1Nwn∙NTn+−NTnk2+NIm+−NInk2+NFn+−NFnk2,(18)FDk−=13N∑n=1Nwn∙NTn−−NTnk2+NIn−−NInk2+NFn−−NFnk2,where wn is the weight for the nth indicator and can be determined in the same way as wm in Equations (13) and (14).Letwd and wf be the weights of the quantitative part and the qualitative aspect of the distances, subject to 0≤wd,wf≤1,wd+wf=1. The comprehensive weighted distancesWDk+ and WDk− could be obtained according to Equations (19) and (20). If only quantitative indicators were involved in the evaluation, wd equaled to 1 and wf was 0. In contrast, when the assessment only relied on the qualitative indicators, wd was set as 0 and wf was 1.
(19)WDk+=wd∙DDk++wf∙FDk+,(20)WDk−=wd∙DDk−+wf∙FDk−.Finally, the closeness coefficient can be obtained according to Equation (7).


## 3.1. Indicator Selection

The selected indicators for the evaluation of teaching quality are shown in Table1, which shows that this indicator system covered the aspects regarding teaching performance and those concerning the students’ learning outcomes. This would make the evaluation complete and provide more information on the teaching quality.Table 1
Selected indicators for the teaching quality evaluation.
AspectIndicatorDescriptionTypeTeachers’ attitudeC1Class punctualityQuantitativeC2Dismissing the class earlyQuantitativeC3Number of times the class schedule was changedQuantitativeC4Degree of concentration on teachingQualitativeC5Efforts to monitor students’ learningQualitativeStudents’ responseC6Number of teacher-student interactionsQuantitativeC7Number of students looking at the blackboardQuantitativeTeaching contentC8Consistency with syllabusQualitativeC9NoveltyQualitativeTeaching abilityC10Familiarity with teaching contentQualitativeC11Suitability of teaching methodsQualitativeC12Effectiveness in getting students’ attentionQualitativeLearning outcomeC13Degree of achieving learning goalsQualitativeC14Improvement of the learning abilityQualitativeIn addition, scientific methods were also developed to collect indicator values. For the quantitative indicators, the assessment data were imported from the Center for Teaching Quality Assessment of the university. We believe that this is the most authoritative way to collect the indicator values. For the qualitative ones, questionnaires were employed to collect the assessment of experts and students. The questionnaires contained all the qualitative indicators listed in Table1. Nine answer options were provided for each item. These were as follows: extremely good/high, very good/high, good/high, medium good/high, medium/fair, medium bad/low, bad/low, very bad/low, and extremely bad/low. Experts and the students who took the evaluated courses would be invited to make their choices.


## 3.2. Input Data Processing

### 3.2.1. Tendency Treatment

According to the principle of TOPSIS, the values of the cost indicators were first transformed into beneficial values. In this study, the original values of the cost indicatorsC1, C2, and C3were applied to Equation (8), and the corresponding benefit values were obtained.
(8)Cik,P=maxkCik−CikmaxkCik−minkCik,where Cik denotes the original value of the indicator Ci for the lecturer k and Cik,P is the obtained beneficial value of Cik.


### 3.2.2. Normalization for Quantitative Data

To avoid comparing indicator values on different scales, all the quantitative data are normalized according to Equation (9).
(9)NCik=Cik∑kCik2,where NCik is the normalization result of the original value Cik.


### 3.2.3. Quantification for Qualitative Data

The input data, expressed in a nonnumerical way, was quantified through the SVNS representation technique. Table2 shows the SVNS representation for the fuzzy linguistic variables.Table 2
SVNS representation of the fuzzy linguistic variables.
Linguistic term (x)SVNS<TBx,IBx,FBx>Extremely good/high<0.99, 0.01, 0.01>Very good/high<0.90, 0.10, 0.10>Good/high<0.80, 0.20, 0.15>Medium good/high<0.70, 0.30, 0.30>Medium/fair<0.50, 0.50, 0.50>Medium bad/low<0.30, 0.65, 0.60>Bad/low<0.20, 0.75, 0.80>Very bad/low<0.10, 0.90, 0.90>Extremely bad/low<0.01, 0.99, 0.99>Suppose there wereT evaluators (including experts and students) providing their feedback for each lecturer regarding each qualitative indicator. The SVNS weight aggregation operator was utilized to aggregate the different representations given by different evaluators according to the given weights λ. Equation (10) illustrates the aggregation operation.
(10)FCik=λ1FCik,1⨁λ2FCik,2⨁⋯⨁λTFCik,T=1−∏t=1T1−NTik,tλt,∏t=1TNIik,tλt,∏t=1TNFik,tλt,whereFCik,t is the SVNS representation of the indicator Ci for the kth lecturer provided by the tth(t=1,2,⋯,T) evaluator. FCik denotes the aggregated indicator value.


## 3.2.1. Tendency Treatment

According to the principle of TOPSIS, the values of the cost indicators were first transformed into beneficial values. In this study, the original values of the cost indicatorsC1, C2, and C3were applied to Equation (8), and the corresponding benefit values were obtained.
(8)Cik,P=maxkCik−CikmaxkCik−minkCik,where Cik denotes the original value of the indicator Ci for the lecturer k and Cik,P is the obtained beneficial value of Cik.


## 3.2.2. Normalization for Quantitative Data

To avoid comparing indicator values on different scales, all the quantitative data are normalized according to Equation (9).
(9)NCik=Cik∑kCik2,where NCik is the normalization result of the original value Cik.


## 3.2.3. Quantification for Qualitative Data

The input data, expressed in a nonnumerical way, was quantified through the SVNS representation technique. Table2 shows the SVNS representation for the fuzzy linguistic variables.Table 2
SVNS representation of the fuzzy linguistic variables.
Linguistic term (x)SVNS<TBx,IBx,FBx>Extremely good/high<0.99, 0.01, 0.01>Very good/high<0.90, 0.10, 0.10>Good/high<0.80, 0.20, 0.15>Medium good/high<0.70, 0.30, 0.30>Medium/fair<0.50, 0.50, 0.50>Medium bad/low<0.30, 0.65, 0.60>Bad/low<0.20, 0.75, 0.80>Very bad/low<0.10, 0.90, 0.90>Extremely bad/low<0.01, 0.99, 0.99>Suppose there wereT evaluators (including experts and students) providing their feedback for each lecturer regarding each qualitative indicator. The SVNS weight aggregation operator was utilized to aggregate the different representations given by different evaluators according to the given weights λ. Equation (10) illustrates the aggregation operation.
(10)FCik=λ1FCik,1⨁λ2FCik,2⨁⋯⨁λTFCik,T=1−∏t=1T1−NTik,tλt,∏t=1TNIik,tλt,∏t=1TNFik,tλt,whereFCik,t is the SVNS representation of the indicator Ci for the kth lecturer provided by the tth(t=1,2,⋯,T) evaluator. FCik denotes the aggregated indicator value.


## 3.3. Multilevel Evaluation Framework

In this study, a multilevel evaluation framework was proposed using TOPSIS technique. It consisted of an overall evaluation and five specific evaluations, as shown in Figure1.. The overall evaluation was performed using all the indicators listed in Table 1 and provided a credible rank for each evaluated lecturer. The specific evaluations only used the particular indicators of one aspect. Their functions were to identify areas for improvement and individualizing professional development.Figure 1
Multilevel evaluation framework.Suppose there wereM quantitative indicators involved in an evaluation. Let DI+ denote the quantitative subset of the ideal point I+ and DI− represent the quantitative subset of the negative-ideal point I−. The definition of DI+ and DI− were described as Equations (11) and (12).
(11)DI+=Qm+=maxkNCmk,m=1,2,⋯M,(12)DI−=Qm−=minkNCmk,m=1,2,⋯M.Correspondingly, the quantitative parts of the distancesWDk+ and WDk− can be expressed as Equations (13) and (14).
(13)DDk+=∑m=1Mwm∙Qm+−NCmk2,(14)DDk−=∑m=1Mwm∙Qm−−NCmk2,where wm is the weight for the mth indicator and can be determined using the method of the SVNS cross-entropy [25].Similarly, suppose there wereN qualitative indicators involved in the evaluation.FI+ and FI− were the qualitative subsets of I+ and I−, respectively. Their definitions were as Equations (15) and (16).
(15)FI+=Qn+=NTn+,NIn+,NFn+=maxkNTnk,minkNInk,minkNFnk,n=1,2,⋯N,(16)FI−=Qn−=NTn−,NIn−,NFn−=minkNTnk,maxkNInk,maxkNFnk,n=1,2,⋯N.The qualitative part of the distancesWDk+ and WDk− can be expressed as Equations (17) and (18).
(17)FDk+=13N∑n=1Nwn∙NTn+−NTnk2+NIm+−NInk2+NFn+−NFnk2,(18)FDk−=13N∑n=1Nwn∙NTn−−NTnk2+NIn−−NInk2+NFn−−NFnk2,where wn is the weight for the nth indicator and can be determined in the same way as wm in Equations (13) and (14).Letwd and wf be the weights of the quantitative part and the qualitative aspect of the distances, subject to 0≤wd,wf≤1,wd+wf=1. The comprehensive weighted distancesWDk+ and WDk− could be obtained according to Equations (19) and (20). If only quantitative indicators were involved in the evaluation, wd equaled to 1 and wf was 0. In contrast, when the assessment only relied on the qualitative indicators, wd was set as 0 and wf was 1.
(19)WDk+=wd∙DDk++wf∙FDk+,(20)WDk−=wd∙DDk−+wf∙FDk−.Finally, the closeness coefficient can be obtained according to Equation (7).


## 4. An Example of Teaching Quality Evaluation

The applicability of the proposed framework was illustrated with the help of an example, followed by a discussion based on the evaluation results. Eighty-four lecturers were involved in this evaluation. The Center for Teaching Quality Assessment provided the quantitative data for the lectures. Twenty experts and all the students who took the related courses were invited to make their qualitative assessment of the lecturers. While transforming the qualitative assessments made by the experts and students into SVNS representations, the weightλ for each expert was set as 0.4 and the weight for each student evaluator was set as 0.6. When calculating the final closeness coefficient, the weights wd and wf were set as 0.3 and 0.7, respectively. The evaluation was performed on a self-developed software. Figure 2 shows four typical interfaces of the software.Figure 2
The typical interfaces of the self-developed software for teaching quality evaluation.
(a)
User login(b)
Quantitative data collection(c)
Qualitative data collection(d)
Evaluation selectionFigure3(a) shows the overall evaluation results of the 84 lecturers. Most lecturers’ closeness coefficient fluctuates around 0.5, indicating that the settings of the ideal solution and the negative ideal solution are reasonable. The five lecturers of No. 25, No. 35, No. 49, No. 66, and No. 82 have high closeness coefficients, which are all greater than 0.7. It means that the teaching quality of these five lecturers is the best among the 84 lecturers. The five lecturers were requested to share their practices throughout the university and were recommended to be given priority in awards and promotion.Figure 3
The evaluation results of the 84 lecturers.
(a)
Overall evaluation(b)
Teachers’ attitude evaluation(c)
Students’ response evaluation(d)
Teaching content evaluation(e)
Teaching ability evaluation(f)
Learning effect evaluationFigures3(b)–3(f) illustrate the results of the five specific evaluations. It was observed that there were 49 lecturers who had closeness coefficients in at least one specific evaluation lower than the preset threshold, which equaled to 0.2. These lecturers would receive a written feedback and needed to participate in mentoring sessions on teaching skills. The instructional coach would review the evaluation feedback with them and help them identify the areas of improvement in their own teaching skills. Thereafter, a data-driven action plan would be created for improving the 49 lecturers’ teaching practices. If a lecturer achieved low scores in two consecutive semesters, they would be required to suspend teaching for a short period. Only after passing an additional examination could they restart their teaching practice.


## 5. Conclusion

Teaching quality is an intangible concept, and there is an increasing interest in its evaluation methods. This study demonstrates that SVNS, combined with TOPSIS, provides a novel tool in addressing this problem. A case study was also provided to demonstrate how to employ the method to assess the teaching quality. The teaching quality of 84 lecturers was evaluated with the quantitative and qualitative data collected. As expected, the overall evaluation clearly indicated the five lecturers who achieved the best teaching performance. Additionally, the five specific evaluations identified the issues related to lecturers’ growth areas that they might need to address. That would help them improve their teaching practice.However, the weights, such asλ in Equation (10) and ω in Equations (17)–(20), were specified by experts before the beginning of the evaluation in this study. It is a simple method, but the values may not be the optimal one. Further efforts are suggested to find the best way to optimize the value of the weights. In addition, the proposed method needs to be tested on more teaching practices from different higher education institutions.



---

*Source: 6217191-2022-03-17.xml*