Using Statistical Process Control to Enhance Student Progression

Public interest in educational outcomes has markedly increased in the most recent decade; however, quality management and statistical process control have not deeply penetrated the management of academic institutions. This paper presents results of an attempt to use Statistical Process Control (SPC) to identify a key impediment to continuous improvement in the academic setting and to facilitate continual improvement of instructional performance. An illustration from an undergraduate degree program in business administration is provided. Mark D. Hanna, Nilesh Raichura, & Ednilson Bernardes 72 Spring 2012 (Volume 8 Issue 1) agement. Similarly, in the last decade various studies have discussed the application of quality management principles to higher education (e.g., Baile and Bennett, 1996; Ensby and Mahmoodi, 1997; Mehrez et al., 1997; Vazzana et al., 1997; Willis and Taylor, 1999). However, although assessing educational processes is necessary for continual improvement of educational outcomes, institutions of higher learning seem to stand in need of examples demonstrating actual application of quality methods in their environment and greater expertise in the techniques to perform such assessment. For example, Lawrence, et al. (2008, p. 453) seeks to “stimulate thinking about the application of a proven quality management methodology to academic settings where formal improvement programs such as six sigma are not commonly found. ” W. Edwards Deming, a leading proponent of quality management and the practice of continuous improvement, promoted an improvement cycle known variously as the Shewhart cycle, Deming cycle, PDCA (Plan-Do-Check-Act) Cycle, or PDSA (Plan-Do-Study-Act) cycle (Evans and Lindsay, 2002). Like TQM, this approach has been widely used in industry, however has not deeply penetrated the administration of instruction in American universities (Mergen et al., 2000). While there is great interest in improvement and innovation, continuity of effort seems to be lacking. It is at least possible that this continuous improvement practice has not penetrated academic administration because of a lack of awareness of the way in which statistical control of processes can be used to establish an understanding of the range of outcomes one might expect under normal circumstances and a baseline for ongoing improvement. Conversely, statistical control may not be widely applied in academic administration because continuous improvement – in a form such as the P-D-S-A cycle is not practiced. Regardless, statistical process control is ideal for capturing educational processes such as retention-progression-graduation, because it focuses not only on measuring outcomes, but also, and importantly, on measuring processes, and on prevention of nonconformance rather than ex post detection. (Montgomery, 2000). More recently, improvement efforts at universities have come in the form of enrollment management efforts directed toward student retention, progression and graduation (RPG). These efforts are becoming increasingly important as our society is becoming more and more efficiency and value conscious. As student cohorts progress from the freshmen to senior level, some individuals are lost due to attrition. While there may be many factors that play a role in student attrition, including student characteristics and institutional practices, attrition does cause reduced efficiency and lost opportunities in the educational system. As such, RPG represents a significant family of performance measures for universities today, and will likely continue to grow in importance. Retention, Progression and Graduation are terms that have been operationally defined by the Integrated Postsecondary Educational Data System (IPEDS). Specifically, a first time freshman is a degree-seeking undergraduate who enters college for the first time in the fall term or who has graduated from high school within the same calendar year of that fall term. Cohorts are groups of first time, full time, degree-seeking freshmen who enter a specific university during a given fall term. A university’s retention rate for a given cohort is defined as the percentage of that group that returns to the same institution the following fall. Many universities focus primarily on freshman retention, meaning this statistic is not always tracked as the cohort matures into their third year and beyond. Progression, however, is monitored throughout the academic career of the cohort. Progression is defined as the fall cohort who progress to the next classification by the subsequent fall. The rate of progression for a cohort in a given fall would be reflected in a histogram presenting the percentage of students in each classification. Graduation rates are defined as the percentage of a fall cohort that graduates by the summer of their sixth year. While the above describes an entry cohort based methodology, Lee and Buckthorpe (2008) present an exit cohort method. For study programs in a moreor-less steady state operational situation, their method has the benefit of providing a current indicator of the current state of affairs in a course of study, rather than requiring a lag of several years while an entry cohort progresses. In addition, it has been tested over several years and “has proved Using Statistical Process Control to Enhance Student Progression Journal of Learning in Higher Education 73 to be a robust method” of calculating non-completion rates. The measurement and use of RPG has recently become quite pervasive among global universities. In the U.S., it is used by the National Collegiate Athletic Association (NCAA) to monitor athletic programs and ensure that universities do not field athletes who are not making academic progress. The consequence of not meeting the NCAA’s RPG targets for an athletic team is a reduction in the number of scholarships that may be offered to members of that team. Similarly, university systems around the world have used RPG or non-completion rates as a way of evaluating the relative effectiveness of their member institutions and to establish performance benchmarks. They have also adapted their funding formulas to financially reward institutions that meet RPG targets and penalize those that do not. In this paper, we present the results of a project that seeks to address the resistance of academic environments to the use of continuous improvement techniques. The project involves developing an illustrative example of Statistical Process Control (SPC) applied to the supervision of instruction. Our example shows that SPC is effective in demonstrating the need for standardization of course outcomes prior to efforts at continuous improvement. It also illustrates the importance of SPC in any improvement cycle that may be used by academic administrators who seek to fulfill the AACSB mandates described above, establish a continual improvement process for the courses they supervise, and/or address demands for improved RPG. STATISTICAL PROCESS CONTROL SPC was first proposed by Walter A. Shewhart. In the context of the mass production of parts used by his employer, Shewhart (1931, p. 34) suggested that SPC would lead to at least five favorable outcomes, which he labeled: 1. Reduction in the cost of inspection, 2. Reduction in the cost of rejections, 3. Attainment of maximum benefits from quantity production, 4. Attainment of uniform quality even though inspection test is destructive, 5. Reduction in tolerance limits where quality measurement is indirect. Shewhart noted that “it is often more economical to throw out defective material at some of the initial stages of production rather than to let it pass on to the final stage where it would likely cause the rejection of a finished unit of product. ”(Shewhart, 1931, p. 28) Shewhart’s perspective, which has now been validated by nearly a century of industrial practice, was that “by eliminating assignable causes of variability, we arrive at a limit to which it is feasible to go in reducing the fraction defective. ”(Shewhart, 1931, p. 28) Shewhart developed SPC as a method for identifying the assignable causes of variability – those that affect a process at sometimes but not at others so that they could be removed. The control charting techniques associated with SPC are now widely used in business and other public organizations. However, they are far less prevalent in university administration of instructional processes. While there are substantial differences between the factories addressed by Shewhart’s research in the early 20th century, and universities in the early 21st century, the outcomes envisioned by Shewhart are now being openly promoted in public higher education policy in the United States. In higher education: 1. The call for “reduction in the cost of inspection” is reflected in concerns about the rapid growth in assessment requirements and the cost associated with this growth. 2. The call for “reduction in the cost of rejections” is reflected in efforts by accrediting and funding agencies to assure RPG, or reduce non-completion rates. It is also reflected in the prevalent attempts among undergraduate programs to create “first year experience” programs that demonstrably enhance RPG. 3. The call for “attainment of maximum benefits from quantity production” is reMark D. Hanna, Nilesh Raichura, & Ednilson Bernardes 74 Spring 2012 (Volume 8 Issue 1) flected in the widespread expectation of access to higher education and financial support for the same. 4. The call for “attainment of uniform quality even though inspection test is destructive” is reflected in the fact that the decision to hire a specific graduate cannot be redone. One primary current emphasis of virtually all accrediting agencies seems to be on program assessment. This reflects the need to ensure that instructional processes consistently attain program objectives. 5. The call for “reduction in tolerance limits where quality measurement is indire