# A Study of Efficiency Evaluation of National Quality Online Courses during the Epidemic: Based on Fuzzy Logic Calculation and Bootstrap-DEA

Authors: Zhen-Hui Li; Jia-Jia Yang; Hai-Qing Qin; Yi-Wei Xia; Mei-er Zhuang
Journal: Mathematical Problems in Engineering
 (2021)

---

## Abstract

The study combines fuzzy logic calculation method and bootstrap-DEA to explore the efficiency of National Quality Online Courses (NQOOCs) during the new coronavirus epidemic. We choose Project 985 universities in China as our sample considering their academic influence. The results show that the efficiency value of NQOOCs calculated by bootstrap-DEA differs from the efficiency value measured by traditional DEA. The difference of universities` ranking is more pronounced in bootstrap-DEA. The amount of input resources (including human resources, science and technology resources, and financial resources) may not be directly reflected on the efficiency of NQOOCs. There is still much room for improvement in the efficiency of NQOOCs in most universities.


---

## Body

## 1. Introduction

Under the influence of the new coronavirus epidemic, most of universities have actively responded to the Ministry of Education’s initiative of “no suspension of classes, no suspension of classes” and launched online courses. During this period, “National Quality Online Open Courses (NQOOCs)” played a leading role in demonstration. Many universities have opened high-quality online courses and virtual simulation experimental teaching resources free of charge to students across the country through online education websites such as “Chinese Universities MOOC.” However, due to the lack of online teaching experience and the different input levels and methods of online courses, the effectiveness of online education construction in various universities is uneven. In order to improve the quality of online courses education in universities, it is particularly important to explore the relationship between the investment and effectiveness of online courses in universities.Although there have been studies on the advantages and disadvantages of online courses, the development model, and the problems faced by online teaching during the epidemic, most of them are qualitative analysis, and there is a lack of quantitative evaluation of the construction methods and effects of online education in different universities. As the pioneer of online education in China, universities sponsored by Project 985 have accumulated relative experience. The National Quality Online Open Courses represent the best quality in online courses. Therefore, this article takes universities sponsored by Project 985 as the representative and uses the DEA-SBM model to analyze the relationship between the input elements and output elements of the national quality online open courses during the epidemic in 2020 and obtains the optimal benchmarking model by comparison. It is hoped that this research can provide reference suggestions for improving the quality of online education courses in universities.


## 2. Literature Review

E-learning, also known as distance education and online learning, is a learning method that can spread content across time and space through the use of information technology and Internet technology, bringing great convenience to our lives, learning, and work [1, 2]. Its forms include micro-classes, MOOCs, SPOC, and so on. Among them, MOOCs are the main form of China’s online education designated quality online courses. In 2015, the “Opinions of the Ministry of Education on Strengthening the Application and Management of Online Open Curriculum Construction in Colleges and Universities” pointed out that China’s universities should strengthen the construction of an online open curriculum system and platform with Chinese characteristics, and actively promote the update of college education concepts and the optimization of teaching models [3]. Major universities in China actively respond to the call of the Ministry of Education, and many famous universities led by Peking University and Tsinghua University actively participate in the construction of online open courses. In 2019, the Ministry of Education identified 801 courses as 2018 national quality online open courses, including 690 undergraduate education courses and 111 vocational higher education courses.Online education in colleges and universities has emerged as a teaching model in recent years, and many scholars have analyzed its advantages, disadvantages, and development models. Online education can increase the interactivity of courses through the information database management technology and two-way interactive function of the computer network, and the asynchronous or synchronous learning network model reduces the time and space constraints [4, 5], and the learners of the course have no background and experience restrictions [6], which provides participants with convenient learning opportunities. However, the existing online courses have the problems of insufficient wisdom, low teachers’ new media literacy, and lack of innovation in instructional design [7]. Zhang and Wang found that there is still a lot of room for improvement in the quantity, technology, and quality of the MOOC and MOOC platform through the observation of relevant data of the national quality online open courses [8]. In response to the above problems, a lot of research has begun to focus on the factors that affect the efficiency of online education and the ways and means to improve online education [9]. The study mainly explores the influence of learner factors, teacher factors, and technical factors on the satisfaction of online education courses from the TAM model [10, 11]. The exploration of the efficiency of online courses in colleges and universities under the background of new coronavirus is also very important. For example, Zou et al. combined with the situation in the Guangdong-Hong Kong-Macao Greater Bay Area to build an online teaching model for universities and implemented it in emergency to solve the urgent need for online teaching organization and management during the outbreak [12].However, the relevant research on online education, especially MOOCs, mainly focuses on the influencing factors of course efficiency, such as course design factors. However, such factors mainly affect learners’ satisfaction and the willingness to learn. Besides, the measurement of the outcome of MOOCs is mostly based on singular variable, and the influence of the learner factor is exaggerated [12]. For the national quality online courses, the input of universities has a more significant impact on their quantity and quality, so it is of great practical significance to explore the relationship between university inputs and the output of quality online courses. In addition, the existing literature has proposed many strategies for online education in colleges and universities, but it has not obtained the optimal model by comparing the efficiency of online education construction in many colleges.Therefore, this article takes universities sponsored by Project 985 as the representative and combines the fuzzy logic calculation method and bootstrap-DEA model to analyze the relationship between the input elements and output elements of the national excellent courses from January to May 2020 to find the optimal benchmarking model. This article will also provide reference suggestions for universities to improve the quality of online education courses.


## 3. Data Sources and Variables

Data for the input of universities for education are derived from panoramic data platform of universities, the platform comprehensively covers more than 150 high-interest development related data indicators, and the collected data cover double first-class construction, faculty, scientific research, courses construction, education and teaching, and so on. The data are accurate and provide a good data source for researchers.The data of Project 985 national high-quality online courses are mainly from the Chinese University MOOC website. This website is a domestic high-quality Chinese MOOC learning platform, which was created by the Love Course Network and Netease Cloud Classroom. The platform has more than a thousand courses including Project 985 universities, among which are the first batch of certified national quality online open courses data, including the number of courses, course ratings, and student participation. It can provide data support for our analysis of the efficiency of national quality online courses (https://www.icourse163.org/).Combining the data from the two data platforms and the list of 985 national universities, a total of 39 universities were identified. After that, we excluded three schools that did not open online courses on the MOOC platform of Chinese universities during the epidemic, namely, Tsinghua University, Northwest A&F University, Lanzhou University, South China University of Technology, and National University of Defense Technology that lacked input data. Finally, 34 Chinese Project 985 universities were identified in our research sample.The input variables we choose are human resources, science, and technology resources and financial resources universities invested in 2019, which are essential elements for universities` courses design and construction. We use high-end talent (HET), tech talent (TT), and course professionals (CP) to measure human resources. And to measure science and technology resources, we use scientific research projects (SRP), research scale (RS), and discipline construction (DC) as variables. Financial resources are measured by project investment (PI), social donation (SD), and fund investment (FI). Besides, to measure the output of NQOOC, we use total courses (TC), teaching quality (TP), and total number of participants (TNP) during the epidemic situation (from Jan 2020 to May 2020). Definitions and statistic descriptions of the above variables are shown in Table1.Table 1
The definitions and summary statistics of the variables.
VariablesDefinitionMeanStd. dev.MinMaxHETTotal number of academicians of Chinese Academy of Sciences and Chinese Academy Of Engineering1.2351.4780.0007.000TTTotal number of highly cited scholars of Elsevier China30.73525.8261.000107.000CPTotal number of members of the College Teaching Steering Committee of the Ministry of Education49.35323.6049.000111.000SRPTotal number of National Natural Science Foundation Projects416.853279.22715.0001261.000RSTotal number of published research papers included in the Scopus database2.3891.2420.0905.445DCTotal number of double first-class construction disciplines in colleges and universities8.0597.6511.00041.000PIThe average amount of scientific and technological research funds in universities (ten thousand yuan)35.97416.42212.35879.926SDSchool foundation annual social donation income (ten thousand yuan)11.14812.5600.00057.050FITotal amount of National Natural Science Foundation (100 million yuan)3.0532.2520.18311.039TCThe number of available “National Quality Online Open Courses” during the epidemic13.02910.0501.00042.000TQThe viewer rating of the course on “China University MOOC” (0–5).4.6890.0844.4004.857TNPThe number of participants in the course during the epidemic, that is, the total number of participants in the course divided by the times of the specific courses opened2263.3641736.402558.9928408.004HET represents scholars who have made outstanding contributions to the development of related fields. TT are scholars who have a high world influence. CP are members of the College Teaching Steering Committee of the Ministry of Education. These experts have strong teaching ability and high academic attainments and can provide consultation for the discipline construction of universities.SRP are projects supported by the National Natural Science Foundation of China usually have high research value and significance. RS refers to number of papers included in the Scopus database, which is used as an important data source in various domestic and foreign university rankings. The selection of “double first-class” DC is a great affirmation of the effectiveness of college discipline construction.Funding for scientific research projects is an important material basis for the smooth development of scientific research in colleges and universities. So, we choose PI as the representor of scientific and technological research funds. SD is one of the important sources of funding for colleges and universities, and it can play a role in helping colleges and universities. Through FI, we can see the overall scale and overall scientific research strength of basic research and scientific research workers in each school.


## 4. Mathematical Model and Data Analysis

### 4.1. Fuzzy Logic Calculation

Based on multi-valued logic, fuzzy logic is a science using fuzzy sets to study uncertainty concept and ambiguity phenomena as well as their laws. Fuzzy logic can imitate the way of thinking of human brain, express qualitative knowledge and experience with unclear boundaries, and judge and reason about uncertain concepts. It uses membership function to distinguish fuzzy sets and handle fuzzy relationships. For description systems where the model is unknown or uncertain, fuzzy sets and fuzzy rules are used for reasoning and expressing transitional boundaries or qualitative knowledge experience. Meanwhile, fuzzy comprehensive judgment is carried out and solves regular fuzzy information problems that conventional methods are difficult to deal with.In this paper, by using fuzzy logic, the second-level indicators of input and output can be reasoned and judged comprehensively, thus scientifically and effectively merged into the first-level indicators of input and output, which lays the foundation for the efficiency evaluation below. Specific steps are as follows.First, each column of sample data of each secondary index is subjected to min-max normalization processing. The min-max normalization method is to linearly transform the original data. The formula is new data = (original data-minimum value)/(maximum value-minimum value).Second, define the variables in the fuzzy logic and determine the degree of membership. Convert the input value of fuzzy logic into the degree of membership of each set.Third, conduct the process of fuzzification. Determine the relationship between the input value and membership, so that the input value can find the membership of the corresponding set at any point. Define Gaussian, trapezoidal, and triangular membership function parameters, and add input and output variable membership functions to the fuzzy inference system.Fourth, design the fuzzy logic judgment operation. Each membership degree decomposed, imitating human judgment using the concept of fuzzy, the fuzzy rule base and fuzzy inference of the previous step are used to obtain fuzzy logic control result signal. In this study, we use the minimum membership method (MIN implication) for processing.Fifth, add fuzzy logic decision rules. After the input is fuzzified, rules need to be set and recombined with the operation of fuzzy logic.Finally, conduct the process of de-fuzzification. This step is to convert the fuzzy value of the inference result into a clear control signal value. After the fuzzy logic converts the input value into the membership of each set through fuzzy, then several outputs can be obtained through rules and operations (see Table2), and the membership function graphs are drawn (see Figure 1).Table 2
Results of fuzzy logic calculation.
UniversityHuman resourceTechnology resourceFinancial resourceNQOOCsPeking University15.00015.00015.00012.021Beihang University5.2508.0665.00211.379Beijing Institute of Technology5.0005.00010.00012.518Beijing Normal University7.9835.0005.00011.423Dalian University of Technology7.1048.7345.00011.928University of Electronic Science and Technology of China5.0005.0005.0008.724Northeastern University5.0005.0009.18111.128Southeast University8.6059.0125.00011.438Fudan University11.18210.27910.24910.798Harbin Institute of Technology9.06210.6469.99610.056Hunan University5.0005.0005.00010.927East China Normal University5.0005.0005.00010.183Huazhong University of Science and Technology9.86010.4718.81310.890Jilin University9.85711.3505.00014.759Nanjing University10.1019.8247.51615.000Nankai University5.0005.0005.00010.003Xiamen University7.7235.00010.00710.892Shandong University9.95112.8285.0008.486Shanghai Jiao Tong University14.16110.29510.8499.922Sichuan University9.99511.0276.88312.037Tianjin University9.7059.9725.00010.032Tongji University10.10310.4385.0009.020Wuhan University9.9729.9125.00013.032Xi’an Jiaotong University9.64010.3865.00010.662Northwestern Polytechnical University5.0005.0005.00010.778Zhejiang University14.31510.47413.65314.086Ocean University of China5.0005.0005.00010.173University of Science and Technology Of China9.4289.3628.00110.032China Agricultural University5.0005.0005.00010.142Renmin University of China7.9836.3765.0005.659Central South University6.30911.8535.00010.419Sun Yat-sen University11.50710.0689.71010.251Minzu University of China5.0005.0005.0009.139Chongqing University6.7336.6105.00010.079Figure 1
Membership functions of fuzzy logic.
(a)(b)(c)(d)As can be seen from Figure1 and Table 2, after fuzzy logic processing, the number of variables is effectively reduced to three inputs and one output. Since the set data output interval is between 0 and 20, the index results after fuzzy logic control are all within 20, which provides good data conditions for further empirical analysis.


### 4.2. NQOOCs Efficiency Analysis of Universities Based on the Bootstrap-DEA Method

Bootstrap-DEA is an extension of the traditional DEA method. Although the DEA method for some technical parameters estimation method has several advantages, the estimation results are susceptible to random interference factors, with sample sensitivity [13], and the bootstrap-DEA is very essential to overcome the efficiency value of the inner dependency. It makes statistical inferences from the raw data, without making any assumptions about the unknown population. And it generates pseudorandom numbers by taking back samples from existing samples, thus inferring the characteristics of the population [14]. Efficiency value calculation with bootstrap-DEA includes the following steps:Firstly, each DMUXk,Yk, k=1,…,n, uses the traditional DEA method to calculate the efficiency value θk of the sample data.Secondly, based on the efficiencyθk, k=1,…,n the bootstrap method produces n efficiency value θ1b∗,θ2b∗,…,θnb∗, where b represents the b-th iteration using the bootstrap methodThirdly, calculate bootstrap simulation sampleXkb∗,Yk, K=1,…,n, where Xab∗=θk×xk/xab∗, (k = 1, …,n)Fourthly, the traditional DEA method is used to simulate each bootstrap method sample, and the efficiency valueθkb, k=1,…,n, is calculated againFinally, repeat steps 2–4 B times to generate the efficiency valueθkb, b=1,…nGenerally speaking, the more iterations of bootstrap-DEA, the more accurate the calculation results of efficiency value will be. The greater the confidence, the greater the distance between the upper and lower limits of the confidence interval [15]. According to existing studies, without loss of generality, this paper sets the number of bootstrap-DEA analysis iterations as 2000 and the confidence as 95%.In the previous paper, the fuzzy logic calculation method is used to obtain the fuzzy values of input variables and output variables. On this basis, bootstrap-DEA was used to analyze the operational efficiency of online education in 34 Project 985 universities. The results are shown in Table3.Table 3
Efficiency evaluation results of Project 985 universities NQOOCs, with traditional DEA and bootstrap-DEA.
DMUEfficiency valueOriginal rankingEfficiency value after rectifying deviationRanking after rectifying deviationDeviationLower limitUpper limitPeking University0.36730.0000.34234.0000.0250.3270.376Beihang University1.0001.0000.9173.0000.0830.8730.991Beijing Institute of Technology1.0001.0000.84512.0000.1550.7260.978Beijing Normal University1.0001.0000.9024.0000.0980.8400.983Dalian University of Technology0.9413.0000.8696.0000.0720.8310.920University of Electronic Science and Technology of China0.79816.0000.74420.0000.0540.7110.819Northeastern University0.9089.0000.78417.0000.1240.6890.898Southeast University0.85112.0000.78515.0000.0660.7460.852Fudan University0.47826.0000.44630.0000.0320.4290.485Harbin Institute of Technology0.50025.0000.46529.0000.0350.4450.505Hunan University1.0001.0000.9321.0000.0680.8911.026East China Normal University0.9324.0000.8687.0000.0630.8300.956Huazhong University of Science and Technology0.54124.0000.50628.0000.0350.4890.542Jilin University1.0001.0000.8795.0000.1210.7961.000Nanjing University0.83515.0000.78416.0000.0510.7590.825Nankai University0.9158.0000.85310.0000.0620.8160.939Xiamen University0.87010.0000.76419.0000.1060.6850.862Shandong University0.57521.0000.51027.0000.0650.4630.583Shanghai Jiao Tong University0.42629.0000.39033.0000.0350.3700.423Sichuan University0.68819.0000.63523.0000.0520.6070.673Tianjin University0.71518.0000.65322.0000.0620.6130.730Tongji University0.63220.0000.57324.0000.0590.5330.646Wuhan University0.9316.0000.85211.0000.0790.8010.951Xi’an Jiaotong University0.74817.0000.67621.0000.0720.6290.761Northwestern Polytechnical University0.9862.0000.9192.0000.0670.8791.012Zhejiang University0.57422.0000.52125.0000.0530.4880.563Ocean University of China0.9315.0000.8688.0000.0630.8290.955University Of Science and Technology of China0.54623.0000.51326.0000.0330.4970.547China Agricultural University0.9287.0000.8659.0000.0630.8270.952Renmin University of China0.46928.0000.43432.0000.0350.4140.458Central South University0.85711.0000.79213.0000.0660.7570.836Sun Yat-Sen University0.47327.0000.44131.0000.0320.4250.475Minzu University of China0.83614.0000.77918.0000.0570.7450.858Chongqing University0.84113.0000.79014.0000.0510.7640.831From the perspective of ranking, the ranking of universities before and after rectifying deviation has changed greatly. From a comprehensive perspective, except for Hunan University and Northwestern Polytechnical University, the ranking of other universities has decreased by different degrees compared with the traditional DEA results. Except for Hunan University, the former top universities all dropped in different degrees. Beihang University, Beijing Normal University, and Jilin University ranked third, fourth, and fifth, respectively. The ranking of Beijing Institute of Technology dropped to 12, and the efficiency value after rectifying deviation was 0.845, which was the university with the largest decline. In addition, Northwestern Polytechnical University ranked second both before and after the correction.The NQOOCs of Hunan University was 0.932 after rectifying deviation, and the lowest value was 0.342 after rectifying deviation of Peking University. There is still much room for improvement in the efficiency of NQOOCs in most universities.Among the top 10 ranking universities, there are three polytechnic universities, namely, Northwestern Polytechnical University, Beihang University, and Dalian University of Technology, which indicates that the inputs of these universities are well utilized to courses construction. Three comprehensive universities, namely, Hunan University, Jilin University, and Nankai university, also perform well. Besides, the only two normal universities of Project 985 universities, namely, Beijing Normal University and East China Normal University, are both on the top 10 list. The possible reason is that normal universities have more teaching techniques and experience to improve the courses effect, and can better promote the conversion of invested resources in teaching performance. Moreover, Ocean University of China and China Agricultural University achieved the 8th and 9th rankings, which suggests that course characteristics and professionalism are also important factors in ensuring course quality.


## 4.1. Fuzzy Logic Calculation

Based on multi-valued logic, fuzzy logic is a science using fuzzy sets to study uncertainty concept and ambiguity phenomena as well as their laws. Fuzzy logic can imitate the way of thinking of human brain, express qualitative knowledge and experience with unclear boundaries, and judge and reason about uncertain concepts. It uses membership function to distinguish fuzzy sets and handle fuzzy relationships. For description systems where the model is unknown or uncertain, fuzzy sets and fuzzy rules are used for reasoning and expressing transitional boundaries or qualitative knowledge experience. Meanwhile, fuzzy comprehensive judgment is carried out and solves regular fuzzy information problems that conventional methods are difficult to deal with.In this paper, by using fuzzy logic, the second-level indicators of input and output can be reasoned and judged comprehensively, thus scientifically and effectively merged into the first-level indicators of input and output, which lays the foundation for the efficiency evaluation below. Specific steps are as follows.First, each column of sample data of each secondary index is subjected to min-max normalization processing. The min-max normalization method is to linearly transform the original data. The formula is new data = (original data-minimum value)/(maximum value-minimum value).Second, define the variables in the fuzzy logic and determine the degree of membership. Convert the input value of fuzzy logic into the degree of membership of each set.Third, conduct the process of fuzzification. Determine the relationship between the input value and membership, so that the input value can find the membership of the corresponding set at any point. Define Gaussian, trapezoidal, and triangular membership function parameters, and add input and output variable membership functions to the fuzzy inference system.Fourth, design the fuzzy logic judgment operation. Each membership degree decomposed, imitating human judgment using the concept of fuzzy, the fuzzy rule base and fuzzy inference of the previous step are used to obtain fuzzy logic control result signal. In this study, we use the minimum membership method (MIN implication) for processing.Fifth, add fuzzy logic decision rules. After the input is fuzzified, rules need to be set and recombined with the operation of fuzzy logic.Finally, conduct the process of de-fuzzification. This step is to convert the fuzzy value of the inference result into a clear control signal value. After the fuzzy logic converts the input value into the membership of each set through fuzzy, then several outputs can be obtained through rules and operations (see Table2), and the membership function graphs are drawn (see Figure 1).Table 2
Results of fuzzy logic calculation.
UniversityHuman resourceTechnology resourceFinancial resourceNQOOCsPeking University15.00015.00015.00012.021Beihang University5.2508.0665.00211.379Beijing Institute of Technology5.0005.00010.00012.518Beijing Normal University7.9835.0005.00011.423Dalian University of Technology7.1048.7345.00011.928University of Electronic Science and Technology of China5.0005.0005.0008.724Northeastern University5.0005.0009.18111.128Southeast University8.6059.0125.00011.438Fudan University11.18210.27910.24910.798Harbin Institute of Technology9.06210.6469.99610.056Hunan University5.0005.0005.00010.927East China Normal University5.0005.0005.00010.183Huazhong University of Science and Technology9.86010.4718.81310.890Jilin University9.85711.3505.00014.759Nanjing University10.1019.8247.51615.000Nankai University5.0005.0005.00010.003Xiamen University7.7235.00010.00710.892Shandong University9.95112.8285.0008.486Shanghai Jiao Tong University14.16110.29510.8499.922Sichuan University9.99511.0276.88312.037Tianjin University9.7059.9725.00010.032Tongji University10.10310.4385.0009.020Wuhan University9.9729.9125.00013.032Xi’an Jiaotong University9.64010.3865.00010.662Northwestern Polytechnical University5.0005.0005.00010.778Zhejiang University14.31510.47413.65314.086Ocean University of China5.0005.0005.00010.173University of Science and Technology Of China9.4289.3628.00110.032China Agricultural University5.0005.0005.00010.142Renmin University of China7.9836.3765.0005.659Central South University6.30911.8535.00010.419Sun Yat-sen University11.50710.0689.71010.251Minzu University of China5.0005.0005.0009.139Chongqing University6.7336.6105.00010.079Figure 1
Membership functions of fuzzy logic.
(a)(b)(c)(d)As can be seen from Figure1 and Table 2, after fuzzy logic processing, the number of variables is effectively reduced to three inputs and one output. Since the set data output interval is between 0 and 20, the index results after fuzzy logic control are all within 20, which provides good data conditions for further empirical analysis.


## 4.2. NQOOCs Efficiency Analysis of Universities Based on the Bootstrap-DEA Method

Bootstrap-DEA is an extension of the traditional DEA method. Although the DEA method for some technical parameters estimation method has several advantages, the estimation results are susceptible to random interference factors, with sample sensitivity [13], and the bootstrap-DEA is very essential to overcome the efficiency value of the inner dependency. It makes statistical inferences from the raw data, without making any assumptions about the unknown population. And it generates pseudorandom numbers by taking back samples from existing samples, thus inferring the characteristics of the population [14]. Efficiency value calculation with bootstrap-DEA includes the following steps:Firstly, each DMUXk,Yk, k=1,…,n, uses the traditional DEA method to calculate the efficiency value θk of the sample data.Secondly, based on the efficiencyθk, k=1,…,n the bootstrap method produces n efficiency value θ1b∗,θ2b∗,…,θnb∗, where b represents the b-th iteration using the bootstrap methodThirdly, calculate bootstrap simulation sampleXkb∗,Yk, K=1,…,n, where Xab∗=θk×xk/xab∗, (k = 1, …,n)Fourthly, the traditional DEA method is used to simulate each bootstrap method sample, and the efficiency valueθkb, k=1,…,n, is calculated againFinally, repeat steps 2–4 B times to generate the efficiency valueθkb, b=1,…nGenerally speaking, the more iterations of bootstrap-DEA, the more accurate the calculation results of efficiency value will be. The greater the confidence, the greater the distance between the upper and lower limits of the confidence interval [15]. According to existing studies, without loss of generality, this paper sets the number of bootstrap-DEA analysis iterations as 2000 and the confidence as 95%.In the previous paper, the fuzzy logic calculation method is used to obtain the fuzzy values of input variables and output variables. On this basis, bootstrap-DEA was used to analyze the operational efficiency of online education in 34 Project 985 universities. The results are shown in Table3.Table 3
Efficiency evaluation results of Project 985 universities NQOOCs, with traditional DEA and bootstrap-DEA.
DMUEfficiency valueOriginal rankingEfficiency value after rectifying deviationRanking after rectifying deviationDeviationLower limitUpper limitPeking University0.36730.0000.34234.0000.0250.3270.376Beihang University1.0001.0000.9173.0000.0830.8730.991Beijing Institute of Technology1.0001.0000.84512.0000.1550.7260.978Beijing Normal University1.0001.0000.9024.0000.0980.8400.983Dalian University of Technology0.9413.0000.8696.0000.0720.8310.920University of Electronic Science and Technology of China0.79816.0000.74420.0000.0540.7110.819Northeastern University0.9089.0000.78417.0000.1240.6890.898Southeast University0.85112.0000.78515.0000.0660.7460.852Fudan University0.47826.0000.44630.0000.0320.4290.485Harbin Institute of Technology0.50025.0000.46529.0000.0350.4450.505Hunan University1.0001.0000.9321.0000.0680.8911.026East China Normal University0.9324.0000.8687.0000.0630.8300.956Huazhong University of Science and Technology0.54124.0000.50628.0000.0350.4890.542Jilin University1.0001.0000.8795.0000.1210.7961.000Nanjing University0.83515.0000.78416.0000.0510.7590.825Nankai University0.9158.0000.85310.0000.0620.8160.939Xiamen University0.87010.0000.76419.0000.1060.6850.862Shandong University0.57521.0000.51027.0000.0650.4630.583Shanghai Jiao Tong University0.42629.0000.39033.0000.0350.3700.423Sichuan University0.68819.0000.63523.0000.0520.6070.673Tianjin University0.71518.0000.65322.0000.0620.6130.730Tongji University0.63220.0000.57324.0000.0590.5330.646Wuhan University0.9316.0000.85211.0000.0790.8010.951Xi’an Jiaotong University0.74817.0000.67621.0000.0720.6290.761Northwestern Polytechnical University0.9862.0000.9192.0000.0670.8791.012Zhejiang University0.57422.0000.52125.0000.0530.4880.563Ocean University of China0.9315.0000.8688.0000.0630.8290.955University Of Science and Technology of China0.54623.0000.51326.0000.0330.4970.547China Agricultural University0.9287.0000.8659.0000.0630.8270.952Renmin University of China0.46928.0000.43432.0000.0350.4140.458Central South University0.85711.0000.79213.0000.0660.7570.836Sun Yat-Sen University0.47327.0000.44131.0000.0320.4250.475Minzu University of China0.83614.0000.77918.0000.0570.7450.858Chongqing University0.84113.0000.79014.0000.0510.7640.831From the perspective of ranking, the ranking of universities before and after rectifying deviation has changed greatly. From a comprehensive perspective, except for Hunan University and Northwestern Polytechnical University, the ranking of other universities has decreased by different degrees compared with the traditional DEA results. Except for Hunan University, the former top universities all dropped in different degrees. Beihang University, Beijing Normal University, and Jilin University ranked third, fourth, and fifth, respectively. The ranking of Beijing Institute of Technology dropped to 12, and the efficiency value after rectifying deviation was 0.845, which was the university with the largest decline. In addition, Northwestern Polytechnical University ranked second both before and after the correction.The NQOOCs of Hunan University was 0.932 after rectifying deviation, and the lowest value was 0.342 after rectifying deviation of Peking University. There is still much room for improvement in the efficiency of NQOOCs in most universities.Among the top 10 ranking universities, there are three polytechnic universities, namely, Northwestern Polytechnical University, Beihang University, and Dalian University of Technology, which indicates that the inputs of these universities are well utilized to courses construction. Three comprehensive universities, namely, Hunan University, Jilin University, and Nankai university, also perform well. Besides, the only two normal universities of Project 985 universities, namely, Beijing Normal University and East China Normal University, are both on the top 10 list. The possible reason is that normal universities have more teaching techniques and experience to improve the courses effect, and can better promote the conversion of invested resources in teaching performance. Moreover, Ocean University of China and China Agricultural University achieved the 8th and 9th rankings, which suggests that course characteristics and professionalism are also important factors in ensuring course quality.


## 5. Conclusion

Based on the fuzzy logic calculation analysis and bootstrap-DEA analysis, this study evaluated the efficiency of national quality online open courses of Chinese Project 985 universities. The results have some implications.The combination of fuzzy logic calculation analysis and bootstrap-DEA analysis may provide some reference for the process of dealing with variables that are of similarities. And the use of bootstrap-DEA can overcome several disadvantages of traditional DEA, especially that there are significant difference in the ranking results of bootstrap-DEA, which is essential for us to probe into the possible causes.There are some practical implications of this paper. Firstly, universities with abundant resources should pay more attention to the efficiency of curriculum construction, because there are a large number of participants that rely on the NQOOCs to gain knowledge. Secondly, ensuring the distinguishing feature and professional level of NQOOCs is a vital way to promote teaching efficiency. What is more, the focus of this research is online education, which can provide reference value for scholars in this field in the future, enriching literature materials.Additionally, this study still has some limitations that may offer future research direction. Although we have evaluated the efficiency of NQOOCs during the new coronavirus epidemic, the previous invested resources may be for a whole year of output. Future research can investigate the efficiency of NQOOCs measured by the whole year outputs. Besides, the results of bootstrap-DEA only offer the efficiency of NQOOCs, but it is not clear how different input elements produce a marked effect. Using other data-mining methods is necessary to test the mechanism of input elements and outcomes.



---

*Source: 5534583-2021-04-12.xml*