A Tale of Three Classes: Case Studies in Course Complexity

Introduction Pedagogical research is unusual within academic research in that nearly all the researchers in the area are also practitioners, which is to say they teach as well as research teaching. For this reason, interest in the answers to the research questions is personal as well as professional. Will distance learning teaching be as effective as face-to-face techniques? Is the case method really more effective than lecture? Should laptops be allowed in the classroom? The number of questions that might be posed is essentially unbounded. No one would dispute that research on teaching and learning can be challenging. After all, there are many variables that must be considered. Who could plausibly argue, for example, that factors such as the experience of the instructor, the characteristics of the students, the form of content being presented, the method of delivery, and the setting of the class are irrelevant to learning? Nonetheless, often for causal relationships that involve many variables, the individual effects of specific factors can be teased out using techniques such as regression or structural equation modeling (SEM). In such cases, the underlying process can be described as nearly decomposable (Simon, 1981). Additionally, sometimes the interrelationship between variables is so great that such decomposition is impossible. In such cases, the relationship is complex. Where such complexity exists, the research strategy needs to be reevaluated, since an individual variable's impact on overall effectiveness can be highly dependent upon the values of other variables. A particularly significant implication of complexity relates to the value of quantitative analytical techniques, such as those just mentioned. Recent research has demonstrated that, under the reasonable assumption that individuals continuously attempt to improve fitness, complex underlying relationships can produce statistically significant yet entirely misleading results (Gill & Sincich, 2008). Thus, the assumption of decomposability needs to be carefully tested prior to applying these techniques. At the present time, such tests require qualitative analysis of the process; quantitative tests for this form of complexity have yet to be devised (Gill, 2008). The present paper considers the question of the decomposability of teaching situations by presenting a qualitative analysis of three case studies of MIS courses. The cases themselves are intrinsically interesting--all three illustrate innovative teaching techniques, 2 of the 3 were winners of the Decision Science Institute's (DSI) Innovative Curriculum Competition, and all demonstrated substantial evidence of learning and student satisfaction. The research also finds that by comparing the three cases side-by-side considerable insight is gained into the complexity of the relationship between teaching approach, course setting, and outcome. The paper begins by introducing the concept of a rugged fitness landscape, taken directly from a model proposed in evolutionary biology (Kauffman, 1993). Then the research design is presented, which involves a qualitative search for interactions across four key areas of the course context: instructor characteristics, content characteristics, design/delivery characteristics, and student characteristics. Each class is presented, with details provided in two appendices, and the key interactions that were observed are identified. Because the first of these courses--referred to as Ism3232.A--evolved dramatically over time, it is presented in both longitudinal and cross sectional terms. The remaining two courses--Ism3232.B and Ism6155.A--experienced relatively few design changes from the time they were first offered. Both are therefore presented only in cross sectional terms. By comparing time slices and cross sectional observations, observational evidence of high levels of interaction between areas is acquired. The paper concludes by considering how this evidence might change the conduct of future research into IT education. …