Evaluations That Respond: Prescription, Application, and Implications of Responsive Evaluation Theory for Community College Instructional Support Programs

This study examines two community college instructional support programs to explore the effectiveness of an evaluation model - responsive evaluation theory - that may ease the tensions between a concern over programs' processes and reporting requirements for program outcomes. The study uses a comparative qualitative case study design and applies responsive evaluation's prescriptive steps to assess the research questions: How effectively does responsive evaluation theory operate as an evaluation model? How does responsive evaluation theory articulate with systematic evaluation theories? Results indicate that responsive evaluation can be an effective model if evaluators consult program faculty and staff, who in turn express an interest in building a collaborative evaluation, and if the purpose of the evaluation is to examine process-oriented issues. Results further indicate that responsive and systematic evaluation models articulate well in that outcomes-oriented issues can be examined within the context of a responsive evaluation. Finally, results demonstrate that the responsive evaluation process can be highly politicized and, consequently, addresses the concerns of stakeholders to varying degrees. Evaluation Theory for Community College Instructional Support Programs In the health, education, and social services fields, program evaluation has been dominated by objectives-oriented, outcomes-based evaluation (Rossi & Freeman, 1993; Shaddish et al, 1991). In higher education, evaluation is an outcomes-driven, student-achievement-focused institutional tool often used to build systematic models to assess program effectiveness. In recent years, the reliance on systematic approaches has intensified with the rise of regional accreditation standards and state accountability systems, which support an institutional focus on and resources for the evaluation of academic programs and services and assessment of student learning outcomes. In most sectors of higher education, faculty and staff frequently conduct evaluations collaboratively (Green, 1981). However, competing evaluation approaches have tended to bring tension to the evaluation process. This tension could be related to the observation that faculty tend to have a more "qualitative orientation for the justification of educational programs," in contrast to administrator concerns with meeting internal and external reporting requirements (Perry, 1972). In the public sector, this tension has been exacerbated by a tendency to utilize more systematic approaches that are less resource intensive (Cohen & Brawer, 1996). This study examines two community college instructional support programs to explore the effectiveness of an evaluation model- responsive evaluation theory- that may ease the tensions between a concern over the programs' processes and reporting requirements for program outcomes. Two primary research questions are asked in the study: How does responsive evaluation theory operate as a model to evaluate instructional support programs? How does responsive evaluation articulate with systematic evaluation theories? The two programs are evaluated using responsive evaluation theory. Data sources include interviews with program staff, faculty, administrators, and participants. Results from the evaluations and a discussion of the results are presented. The study ends by drawing conclusions about responsive evaluation theory's effectiveness and usefulness for community colleges. Related Literature Evaluation Research and Program Evaluation Evaluation research is "the systematic application of social research procedures for assessing the conceptualization, design, implementation, and utility of social intervention programs" (Rossi and Freeman, 1993, p. 5). Thus, evaluation research uses qualitative, quantitative, or mix-methods research designs and methods to examine programs and services. Program evaluations frequently use the principles of evaluation research to support adrninistrative and managerial decisions. …