Program Evaluation Studies: Strategic Learning Delivery Model Suggestions.

Because the academic tasks vary so dramatically from high school to college, many entering freshmen have difficulties during the lst year of college (Martin & Arendale, 1994; Nist & Simpson, 2000). According to the extant literature, one reason for these difficulties can be traced to the fact that many college freshmen tend to be passive learners rather than active, strategic learners (Alexander & Jetton, 2000; Hofer, Yu, & Pintrich, 1999). In order to assist college students with these higher level thinking tasks, many colleges have established academic assistance programs that offer generic learning strategy courses as well as content-embedded models such as paired or linked courses (Warkentin, Stallworth-Clark, & Nolen, 1999). Although these academic assistance programs are not recent innovations, only a few evaluation studies of such have appeared in the literature (Weinstein, Husman, & Dierking, 2000). The lack of quality programmatic research is not surprising given that Boylan, Bliss, and Bonham (1997) have found in a survey that approximately 25% of the 4-year institutions complete on-going and systematic evaluation of their academic assistance programs. Much of the program evaluation research that does exist has often been"cursory" (Maxwell, 1997) and has overlooked important issues. More specifically, three criticisms have been leveled against the studies on strategic learning delivery models. The first criticism of the studies on strategic learning delivery models is that researchers have typically failed to make explicit the theoretical grounding of their studies (O'Hear & MacDonald, 1995). In particular, there seems to be a mismatch between the theory underpinning a program and the questions and instruments used to evaluate that program. Without a consistent theoretical grounding to a program such as a learning strategy course or a paired course, it becomes extremely difficult to gather valid and useful information. The second criticism is that many studies have not examined students' academic performance using a constellation of dependent variables (Gebelt, Parilis, Kramer, & Wilson, 1996). Rather than posing a variety of questions and employing a battery of instruments that provide a more comprehensive picture of students, many studies seem to focus on one variable and one measure (Boylan, Bonham, White, & George, 2000). The third criticism is that many program evaluation studies have overlooked the critical questions addressing students' transfer and modification of strategies to their own academic tasks (Simpson, Hynd, Nist, & Burrell, 1997). That is, students who complete a learning strategy or paired course should have acquired the declarative, procedural, and conditional strategy knowledge necessary to become active learners across the academic disciplines (Hofer, Yu, & Pintrich, 1999; McKeough, Lupart, & Marini, 1995). Admittedly, transfer is difficult to measure and to insure, but in-depth studies using qualitative measures should be attempted if we really believe that the transfer of learning is the ultimate aim of teaching (McKeough, et al., p. vii). These criticisms of the literature were particularly important to me because I have been coordinating the Adjunct Study Strategy Seminar, an experimental program that was a modification of the paired course model. In that capacity I have spent a considerable amount of time determining what questions I wanted answered about the program, designing instruments to address those questions, collecting and analyzing data, and writing reports. Rather than share the mounds of data I have collected over a period of 8 semesters, I would rather focus on what I have learned from conducting research on the Adjunct Study Strategy Seminar Program. Hence, this article will share seven suggestions, using the experiences and data from my studies to illustrate each point. Such an approach is somewhat unconventional in that a typical journal article describes in depth one program evaluation effort. …