The Agreement of Peer Assessment and Self-Assessment of Learning Processes in Problem-Based Learning

ABSTRACT: Problem-based learning (PBL) has become a widely utilized curricular design in health care professional education. One of the principal features of PBL that distinguishes it from more conventional curricular models is that its outcome objectives emphasize the processes of learning as much as the content of learning. However, there is a lack of literature that addresses the assessment of learning processes in PBL. The purpose of this study was to determine the value of a multiple-measurement method of assessing a student's learning processes in small-group work in PBL case problems. For each of six case problems utilized in an entry-level kinesiology course, students completed peer assessments and self-assessments of learning activities within the group setting. Intraclass correlation coefficients (ICCs) demonstrated poor to moderate agreement between peer assessments and self-assessments for the case problems, and in two cases the lack of between-group variance invalidated the use of the ICC. In addition, there was no trend in the change in the level of agreement as the course progressed. The low level of agreement was related to the restricted range of scores that students assigned to themselves and their peers. Although this particular multiple-measurement format was not a valid method for assessing student learning processes, further study with the use of an alternative scoring system is indicated. Introduction One of the most significant curricular innovations in health care professional education in the past 30 years has been the development of problem-based learning (PBL). 1-3 Problem-based learning has been adopted, in whole or in part, by numerous health care professional educational disciplines, including physical therapy.2,4-7 Problem-based learning is characterized by a fundamental shift in educational philosophy, from teaching a set of distinct, separate subjects (called the "conventional curricular model") to presenting students with case problems in which basic and clinical sciences are learned in an integrated, situational context.2. 7.8 The PBL approach is also distinguished from the conventional curricular design in that the processes of learning are emphasized at least as much as the content of learning and student learning is designed to be self-directed.7-9 The development of the PBL design comes at a critical time for health care professionals. Clinical practice is becoming increasingly "evidence-based," where clinicians are being compelled to guide and justify their patient care decisions less on the teachings of "experts" and unsubstantiated judgments and more on the foundation of published clinical evidence and problem-solving abilities.lo The development of these professional skills are central to the PBL design, where students are required to critically analyze existing knowledge and use it to justify their problem-solving decisions. " ' The research evidence that has compared the PBL approach with the conventional curricular model has not established that PBL, despite its theoretical merits, is superior to the more traditional model in terms of academic achievement or performance on professional licensing examinations.l5 There is also no research support for the notion that PBL facilitates the development of a generic problemsolving skill any better than the conventional curricular model.9. 14,16,17 However, many health care professional education programs have implemented PBL, in part, because faculty and students tend to give it highly favorable ratings in program evaluation studies.1,s8s-zo A fundamental component of the PBL design is that students are responsible for solving their case problems in a group of peers.5. 21 Although students are expected to be self-directed, independent learners, they are also expected to share their knowledge with other students and reach a consensus about the best solution to their case problem. There are several reasons why the promotion of small-group work is a goal of PBL. …