Evaluation of the Effectiveness of the Online Teaching and Learning Process at the University: Based on Kirkpatrick’s Model

The industrial revolution 4.0 is an era where humans can enjoy the ease of doing many things.However, it is undeniable that the development of technology and information in this era also resulted in world disruption or the disruptive era.In the disruptive era, students get easier access to information, more global networks, and shift from teacher-centered or one-way learning to two-way learning.Besides, this disruptive era is characterized by the use of social media platforms for communication and no seniority level in looking at other students.Even, some emerging innovations with advanced technology provide better access to information and sources of knowledge from anywhere and anytime without having to attend a class.In responding to changes in the disruptive era by utilizing the development of technology and information, many lecturers as educators have transformed the traditional learning process into an online learning process.However, it does not mean that the online learning process does not encounter obstacles or runs smoothly without any deficiencies in its implementation.Therefore, evaluation should be considered to find out the value of a program or system that is implemented/applied.This study aims to evaluate the effectiveness of online teaching and learning process, to analyze the strengths and weaknesses of the online learning process, and to evaluate the achievement of learning objectives, particularly in IAKN Manado.This study uses Kirkpatrick's evaluation model which consists of 4 levels with different measurements/assessments at each level.At level 1, the reaction level, the study used a questionnaire distributed to 100 students and analyzed the satisfaction level using the Customer Satisfaction Index (CSI).The result shows that the satisfaction level of students on the online learning process reaches 71%, or in other words, it is in the dissatisfied level.At level 2, the learning level, it analyzed the results of pre-test and post-test before and after the students participated in online learning.Then, based on the Wilcoxon Signed Rank test, the z value is -8.363 with a p-value (Asymp.Sig.2-tailed) of 0.000 which is less than the critical limit value of 0.05.Therefore, H is accepted meaning that there are significant differences between the pre-test and post-test groups.At level 3, the potential of behavior change level, the data were obtained from interviews with 2 lecturers who have conditional courses with previous courses using online learning.The data were also collected from lecturer observations about changes in student behavior including changes in mindset, knowledge, and utilization of skills and knowledge after the online learning process.At this level, the researcher also distributed questionnaires (categorical statements) with Guttman scale analysis to 100 sample students.The result of level 3 shows that students' behavior that follows the online learning process almost changes.It means that the behavior changes are not felt thoroughly and optimally.At level 4, the result level, the data used were the final results of the online learning process.The result shows that it reaches a satisfactory level.Thus, based on all four levels, it can be concluded that the online learning process is not effective yet.Some components of the online learning process need to be optimized to make the learning run effectively and to achieve the learning objectives.Supporting facilities and the skills of lecturers as educators in creating and presenting creative, effective, and fun learning strategies, models, and methods are essential components of the learning process.This study is