Reconsidering Research on Learning from Media

Recent meta-analyses and other studies of media's influence on learning are reviewed, Consistent evidence is found for the generalization that there are no learning benefits to be gained from employing any specific medium to deliver instruction. Research showing performance or timesaving gains from one or another medium are shown to be vulnerable to compelling rival hypotheses concerning the uncontrolled effects of instructional method and novelty. Problems with current media attribute and symbol system theories are described and suggestions made for more promising research directions. Studies of the influence of media on learning have ken a fixed feature of educational research since Thorndike (1912) recommended pictures as a laborsaving device in instruction, Most of this research is buttressed by the hope that learning will be enhanced with the proper mix of medium, student, subject matter content, and learning task. A typical study compares the relative achievement of groups who have received similar subject matter from different media. This research has led to so-called 'media selection" schemes or models 1982). These models generally promise to incorporate existing research and practice into procedures for selecting the best medium or mix of media to deliver instruction. Most of these models base many of their prescriptions on presumed learning benefits from media (Jamison, Suppes, & Welles, 1974). However, this article will argue that most current summaries and meta-analyses of media comparison studies clearly suggest that media do not influence learning under any conditions. Even in the few cases where dramatic changes in achievement or ability have followed the introduction of a medium, as was the case with television in El Salvador (Schramm, 1977), it was not the medium that caused the change but rather a curricular reform that accompanied the change. The best current evidence is that media are mere vehicles that deliver instruction but do not influence student achievement any more than the truck that delivers our groceries causes changes in our nutrition. Basically, the choice of vehicle might influence the cost or extent of distributing instruction, but only the content of the vehicle can influence achievement. While research often shows a slight learning advantage for newer media over more conventional instructional vehicles, this advantage will be __________________ The author wishes to acknowledge the substantive advice of Gavriel Salomon, William Winn, and anonymous reviewers without making them responsible for errors. ___________ p 445 shown to be vulnerable to compelling rival hypotheses. Among these rival explanations is evidence of artifact and confounding in existing studies and biased editorial decisions which may favor research showing larger effect sizes for newer media. After summarizing evidence from current meta-analyses of media research, I will discuss advantages and problems with current "media attribute" and 'symbol system" theories and will conclude by suggesting tentative solutions to past problems and future directions for research involving media. Media Comparison Studies In the 1960s, Lumsdaine (1963) and others (e.g,, Mielke, 1968) argued that gross media comparison and selection studies might not pay off. They implied that media, when viewed as collections of mechanical instruments, such as television and computers, were sample delivery devices. Nevertheless, earlier reviewers also held the door open to learning effects from media by attributing much of the lack of significance in prior research to poor design and lack of adequate models or theory. Lumsdaine (1963) dealt primarily with adequate studies that had used defensible methodology, and had found significant differences between treatments. With the benefit of hindsight it is not surprising that most of the studies he selected for review employed media as simple vehicles for instructional methods, such as text organization, size of step in programming, cueing, repeated exposures, and prompting. These studies compared the effects of. for example, different step size in programmed instruction via television. It was step size (and other methods), not television (or other media), which were the focus of thew studies. This is an example of what Salomon and Clark (1977) called research with media. In these studies media are mere conveyances for the treatments being examined and are not the focus of the study, though the results are often mistakenly interpreted as suggesting benefits for various media. An example of instructional research with media would be a study which contrasted a logically organized audiotutorial lesson on photosynthesis with a randomly sequenced presentation of the same frames (cf. Clark & Snow, 1975; Salomon & Clark, 1977, for a review of similar studies). Perhaps as a result of this confusion, Lumsdaine (1963) reached few conclusions beyond the suggestion that media might reduce the cost of instruction when many students are served because “the cost of perfecting it can be prorated in terms of a denominator representing thousands of students" (p. 670). A decade later, Glaser and Cooley (1973) and Levie and Dickie (1973) were cautious about media comparison studies, which apparently were still being conducted in large numbers. Glaser and Cooley (1973) recommended using acceptable medium as "a vehicle for making available to schools what psychologists have learned about learning" (p. 855). Levie and Dickie (1973) noted that most media comparison studies to that date had been fruitless and suggested that objectives can be attained through "instruction presented by any of a variety of different media" (p. 859). At that time televised education was still a lively topic and studies of computerized instruction were just beginning to appear. During the past decade, television research seems to have diminished considerably, but computer learning studies are now popular. This current h belongs to the familiar but generally fruitless media comparison approach or is concerned _____________ p 446 with different contents or methods being presented via different media (e.g., science teaching via computers). Generally, each new medium seems to attract its own set of advocates who make claims for improved learning and stimulate research questions which are similar to those asked about the previously popular medium. Most of the radio research approaches suggested in the 1950s (e.g., Hovland, Lumsdaine, & Sheffield, 1949) were very similar to those employed by the television movement of the 1960s (e.g., Schramm, 1977) and to the more recent reports of the computer-assisted instruction studies of the 1970s and 1980s (e.g., Dixon & Judd, 1977). It seems that similar research questions have resulted in similar and ambiguous data. Media comparison studies, regardless of the media employed, tend to result i n "no significant difference" conclusions (Mielke, 1968). These findings were incorrectly offered-as-evidence that different media were “equally effective” as conventional means in promoting learning. No significant difference results simply suggest that changes in the outcome scores (e.g., learning) did not result from any systematic differences in the treatments compared. Occasionally a study would find evidence for one or another medium. When this happens, Mielke (1968) has suggested that the active ingredient might be some uncontrolled aspect of the content or instructional strategy rather than the medium. When we investigate these positive studies, we find that the treatments are confounded. The evidence for this confounding may be found in the current meta-analyses of media comparison studies. The next section argues that it is the uncontrolled effects of novelty and instructional method which account for the existing evidence for the effects of various media on learning gains. Reviews and Meta-analyses of Media Research One of the most interesting trends in the past decade has been a significant increase in the number of excellent reviews and meta-analyses of research comparing the learning advantage of various media. The results of these overviews of past comparison studies seem to be reasonably unambiguous and unanimous. Taken together, they provide strong evidence that media comparison studies that find causal connections between media and achievement are confounded. Size of Effect of Media Treatments A recent series of meta-analyses of media comparison studies have been conducted by James Kulik and his colleagues at the University of Michigan (Cohen, Ebling & Kulik, 1981; C. Kulik, Kulik, & Cohen, 1980; J. Kulik, Bangert, & Williams, 1983; J. Kulik, Kulik, & Cohen, 1979). These reviews employ the relatively new technology of meta-analysis (Glass, 1976), which provides more precise estimates of the effect size of various media treatments than were possible a few years ago. Previous reviews dealing primarily with "box score" sums of significant findings for media versus conventional instructional delivery were sometimes misleading. Effect size estimates often were expressed in portions of standard score advantages for one or another type of treatment. This discussion will express effects in one of two ways: (a) the number of standard deviations separating experimental and control groups, and (b) as improvements in percent) scores on a final examination. ____________ p 447 Box Scores versus Effect Size An illustration of the advantage of meta-analytical effect size descriptions of past research over "box scores" is available in a recent review of Postlethwait's audiotutorial instruction studies (J. Kulik, Kulik, & Cohen,, 1979). The authors found 42 adequate studies, of which 29 favored audiotutorial instruction and only 13 favored conventional instruction. Of those 42, only 15 reported significant differences, but 11 of the 15 favored audiotutorial and only 4 favored conventional instruction. This type of box