Quality Measures that Matter

Introduction According to periodic surveys by the National Center for Education Statistics (2008), the number of degree-granting postsecondary institutions in the United States that offer distance education courses has increased from 56% in 2001 to 66% in 2007. Most notably, by 2007, 77% of the enrollments in distance education courses were reported in online courses, corresponding to an estimated 9.4 million US students enrolled in courses delivered entirely online. Such growth in online education has spurred concern over the quality of the learning experience. Over the past decade, there has been a significant effort to develop research-based criteria for rigorous evaluation of online courses and academic programs. This effort has resulted in an increasing emphasis on assessment, as manifested in the standards put forth by national and international accrediting agencies (see, for example, the assurance of learning standards established by the Association to Advance Collegiate Schools of Business or AACSB International described in AACSB, 2009). Coinciding with this emphasis on assessment, the growth in online learning has compelled institutions in higher education to define what quality means for an online course. Drawing from a literature review of 160 articles and twelve books, Chaney (2006) identified 14 quality indicators of distance education, involving aspects of the student-teacher relationship (e.g., frequency of feedback, respect for diverse ways of learning), institutional support (such as faculty support services, program evaluation and assessment, institutional mission), and technology (e.g., appropriate tools and media, reliability of the course technology). Other studies on the quality of distance education have focused on the students' perspective (Chaney, Eddy, Dorman, Glessner, Green & LaraAlecio, 2007), attempting to measure students' attitudes, opinions, and perceptions. Still others were motivated by a pragmatic need to develop reliable course evaluation instruments (Achtemeier, Morris & Finnegan, 2003). In 2003, a consortium of colleges in Maryland named Maryland Online obtained a three-year grant from the Fund for the Improvement of Post-Secondary Education (FIPSE) of the United States Department of Education to build a program for faculty development and continuous improvement in online education, with the specific goal of developing a quality assurance process leading to the quality certification of online courses. Thus the Quality Matters (QM) Rubric was developed, and, after the FIPSE grant expired in 2006, Quality Matters became a self-supporting organization offering course reviews for QM certification and faculty training programs. This paper aims at illustrating the use of the QM rubric in the case of a graduate online course. The next section reviews the main characteristics of the QM certification process and standards, and summarizes the theoretical underpinnings of the QM standards. The third section of the paper applies the 40 QM standards to an online accounting course required for students in the Masters of Business Administration program of a state university in the US, followed by a fourth section on the ease of use and difficulties encountered when applying the QM standards to this particular course. The conclusion section summarizes the main benefits gained from the process of applying for QM certification, and suggests topics for further research in the area of measuring quality of online courses. QM Rubric The QM certification process was developed based on four principles: * Centered: the QM rubric draws from research findings and is updated periodically according to developments in the instructional design literature. * Collaborative: the peer review team (composed of three members, one of which is an expert in the subject matter) and the course developer evaluate the course based on evidence from the course website, instead of opinions of individual reviewers. …