CANADIAN JOURNAL OF EDUCATIONAL COMMUNICATION

Volume 18, Number 2, Summer 1989
ISSN 0710-4340

Can Instructional Variables Be Combined Effectively to Enhance Learning Achievement?

Judith L. Amato

Robert M. Bernard

Miranda D'Amico

Bette DeBellefeuille

Improving Idea Generation During Decision Making
In Small Group Computer Conferences

Norman P. Archer

Responses to: "The Future of Educational Technology is Past"

Andrew Agostino

Don Beckwith

Ivor K. Davies

Philippe Duchastel

Michael J. Hannafin

Stephen T. Kerr

G. M. Torkelson

William Winn

Next Issue: Interactive Learning Technologies

Guest Editor: Richard A. Schwier

An AMTEC Publication

<figure><img src="image_1.png" /></figure>
## EDITORIAL BOARD

**Jon Baggaley**
Concordia University

**F. Barry Brown**
University of Saskatchewan

**Elaine Bruce**
Concordia University

**Bruce Clark**
University of Calgary

**Dan Coldeway**
A thabusca University

**George L. Geis**
OISE

**W. D. Godfrey**
Softwords

**William Hanson**
Calgary Board of Education

**William Hillgartner**
McGill University

**Denis Hlynka**
University of Manitoba

**Robert Jones**
Sheridan College of Applied
Arts and Technology

**Richard F. Lewis**
University of Windsor

**Hanna Mayer**
A thabasca University

**Tom Rich**
Prince Edward Island Department
of Education

**Alexander J. Romiszowski**
Syracuse University

**Lauran Sandals**
University of Calgary

**Richard F. Schmid**
Concordia University

**Richard A. Schwier**
University of Saskatchewan

**Mariela Tovar**
Concordia University

**Jesus Vazquez-Abad**
Les Technologies EDINOV

**Cynthia B. Weston**
McGill University

**Laura R. Winer**
APO Quebec

## AMTEC BOARD OF DIRECTORS

### President

**Mary Kennedy**
Memorial University

### Past President

**Wayne Blair**
Alberta Education

### President Elect

**Bruce McLean**
Vancouver Community
College

### Secretary/Treasurer

**Al LeBlanc**
Separate School Board
Sault Ste. Marie, ON

### Director

**Margaret Sadler**
University of Alberta

### Director

**John Godfreyson**
School District # 68
Nanaimo, BC

### Director

**Dave Bieman**
Huron County Board of
Education
# Canadian Journal of Educational Communication

## Volume 18, Number 2
Summer 1989

### Editor

Robert M. Bernard

### Editor-Elect

Richard A. Schwier

### Assistant Editor

Mandie Aaron

### Production Manager

Mary Genova, WRITEWORKS

### Editorial Assistant

Jacques LeCavalier

### Production Assistant

Michèle Gour

ISSN 0710-4340

The Canadian Journal of Educational Communication is published quarterly by the Association for Media and Technology in Education in Canada; Media Resource Centre; 67 McMeeken Street; Sault Ste. Marie, ON; P6A 3L6; Attention: Mr. Al LeBlanc. All articles are copyright by AMTEC and may be reproduced for non-profit use without permission provided credit is given to CJEC. Back issues of CJEC are $10 Canadian and may be obtained by contacting the Editor. CJEC is indexed in the Canadian Education Index and ERIC.

Second Class Mail Registration No. 6956

## ARTICLES

Can Instructional Variables Be Combined Effectively to Enhance Learning Achievement?
Judith L. Amato
Robert M. Bernard
Miranda D'Amico
Bette DeBellefeuille
85

Improving Idea Generation During Decision Making in Small Group Computer Conferences
Norman P. Archer
111

## RESPONSES

Mitchell's Wake
Andrew Agostino
125

Fashioning the Future Educational Technology: An Invitation
Don Beckwith
127

Total Educational Technology (TET): Challenging Current Limits
Ivor K. Davies
132

The Upcoming of Learning Technology
Philippe Duchastel
137

The Death of Educational Technology Has Been Greatly Exaggerated
Michael J. Hannafin
140

There Are No Ends, Only Means
Stephen T. Kerr
143

The Future of Educational Technology is Past - A Reaction
G. M. Torkelson
146

Revitalizing Educational Technology: A Response to Mitchell
William Winn
149
Contents, cont'd

BOOK REVIEWS

153

Enquiries about previously submitted manuscripts or other business should be sent to:

Dr. Robert M. Bernard, Editor
Department of Education
Concordia University
1455 de Maisonneuve Blvd. W.
Montreal, PQ, Canada
H3G 1M8

New Manuscripts should be sent to:

Dr. Richard A. Schwier,
Editor Elect
College of Education
University of Saskatchewan
Saskatoon, SK
S7N 0W0

## Acknowledgement

The *Canadian Journal of Educational Communication* is supported by a grant from the Social Sciences and Humanities Research Council of Canada.

CJEC is typeset on an Apple Macintosh Plus™ in PageMaker 3.0™. Galley proofs to authors and final camera-ready impressions are output on an Apple LaserWriter Plus™.

Concordia University Printing Services

## AMTEC Membership Form

Send to: **AMTEC**
Media Resource Centre
67 McMeeken Street
Sault Ste. Marie, ON
P6A 3L6

Name

Title

Address

City

Prov./Postal Code

Home Phone

Business Phone

### Type of Membership

☐ Student $ 20.00

☐ Individual $ 50.00

☐ Institutional $ 75.00

☐ Commercial $ 125.00

### CJEC

Available to non-members on a subscription basis.

☐ Three issues $ 40.00

### Special Interest Groups

☐ Visual Literacy

☐ Interactive Video

☐ Desktop Publishing

☐ Microcomputers in Education

☐ Distance Education

☐ Other:

### Work Environment

☐ Elementary

☐ Business

☐ Secondary

☐ Government

☐ College

☐ Health Care

☐ University

☐ Other:

Please duplicate this page.
Can Instructional Variables Be
Combined Effectively to Enhance
Learning Achievement?

Judith L. Amato

Robert M. Bernard

Miranda D'Amico

Beite DeBellefeuille

Abstract: The purpose of this study was to explore an approach to group instruction whereby alterable variables or instructional strategies are combined, in an attempt to achieve the effects associated with one-to-one tutoring (two standard deviations better than group instruction). The design of the project followed principles of instructional systems technology, and incorporated three Instructional strategies with known effect sizes: cooperative learning (Effect size = .80), enhanced student classroom participation (ES = 1.0), and advance organizers (ES = .20). Cognitive, affective and sociometric measures were used to assess the outcomes of the combined treatments, compared to a control group which received conventional lecture-based instruction. Analysis of the data revealed that the approach of combining the three instructional strategies did not result in an effect size of two sigmas. The approach did, however, improve the cognitive achievement of low-aptitude learners, and did not adversely affect the cognitive achievement of middle- and high-aptitude learners. The combined instructional strategies were also found to positively influence students' attitudes toward cooperative learning, and heightened social interaction within the classroom.

INTRODUCTION

Can instructional strategies be combined in higher education courses to produce methods of group instruction which are as effective as one-to-one tutoring? Over the past eight years, a small group of research endeavors have been directed towards answering this question at the primary and secondary school levels. The results of these experiments seem to suggest that the combination of certain strategies may indeed enhance group instruction, so that it becomes as effective in promoting cognitive gains as one-to-one tutoring (Bloom, 1984; Leyton, 1983; Nordin, 1981; Tenenbaum, 1982,1986). Exposed to maximally effective instructional conditions in which instructional variables have been combined, low aptitude students have achieved final cognitive scores surpassing those of high aptitude students under conventional group instruction.
The educational implications of these findings are extremely important. If they can be generalized, practical methods of group instruction may emerge which can be widely applied with little more cost and time than conventional instruction. A change may also result in presently held notions about human potential for learning. Due to the comparatively small number of studies that have thus far been conducted in this area, however, the question of whether or not the impressive results obtained in the lower grades can be replicated in higher education remains, as yet, unanswered. Moreover, little wisdom has yet emerged as to which two or three strategies can best be combined (Bloom, personal communication, April, 1988). It is these issues that the present study attempts to explore.

## THE TWO SIGMA (σ) EFFECT

In educational technology research the ultimate question is often how to provide a specific learner or group of learners, with the best possible instruction, so as to maximize the amount and the quality of the resultant learning. One suggested solution which appears to be highly effective is one-to-one tutoring (Lippitt & Lippitt, 1968), a method which involves a teacher or teacher's aide working directly on a body of information, using a strategy specifically developed to meet the individual learning needs of that student. Under these conditions, feedback from tutor to student is constant, immediate and accurately responsive to the needs of the student. Studies by Anania (1983) and Burke (1983) have demonstrated overwhelming support for the superiority of tutorial over group instruction. After just three weeks, initially similar students were observed to exhibit dramatic differences in their capabilities in the subject being taught and their attitudes toward it.

Of interest here were the differences in final achievement under the conditions of tutorial instruction, mastery learning and group instruction. The average tutored student was at a level above approximately 98% of the conventionally instructed students, while the average student under mastery learning attained final achievement above approximately 84% of the students in conventional group instruction. In other words, tutored students achieved two standard deviations (two sigmas) above the conventionally instructed control group; mastery students achieved one standard deviation. Moreover, these results held regardless of the level of learning objective.

While one-to-one tutoring is clearly an effective method of instruction, it is too costly to be considered a ubiquitous instructional strategy. For this reason, it has not been widely accepted and applied in schools and universities, except in special circumstances. Instead, more economical methods have been proposed to deal with differences in the learning needs of individuals. These methods, known collectively as "individualized instruction," include Glaser's Individually Prescribed Instruction (IPI), Computer Aided Instruction (CAI), Keller's Personalized System of Instruction (PSI), and the various forms of
Mastery Learning proposed by Carroll (1963), Bloom (1968) and Block (1971). Recently, intelligent tutoring has emerged in the literature of educational computing, for essentially the same purpose – to mimic the characteristics that render one-to-one tutoring educationally effective. None of these, however, has achieved the high level of effectiveness – the “two sigma effect” – associated with one-to-one human tutoring.

Work by Bloom (1980) and others to achieve the "two sigma effect" has taken the direction of combining what he refers to as alterable variables (i.e., instructional strategies which can be applied by the teacher or instructional designer), for the purpose of affecting different "objects" of the instructional process. These objects include: a) the learner; b) the instructional materials; c) the home environment or peer group; and d) the teacher. Table 1 (see following page), adapted by Bloom (1984) from Walberg (1984), summarizes the effect sizes (ES) of selected alterable variables and shows Bloom's classification of the object that they influence or change. Further, Bloom speculates that it might be possible to combine certain variables so that their effect is additive. In theory, this could yield enhanced learning, which approaches the "two sigma effect" associated with tutorial instruction. For example, the separate effects of mastery learning (ES = 1.0) and another variable, such as cues and explanations (ES = 1.0), might yield, when combined, an effect size of 2.0.

## Combining Instructional Strategies

To date, the literature addressing the notion of combining instructional strategies is, in Bloom's own words, "still very crude" (B. S. Bloom, personal communication, April, 1988). Indeed, there exists a small number of studies which have thus far attempted to combine instructional strategies in an effort to devise methods of group instruction as effective as one-to-one tutoring. The following is a brief description of these studies.

Nordin (1981) – sixth grade subjects in rural Malaysia were exposed to either: a) enhanced cues (ES = 1.0; change object = teacher), b) enhanced student classroom participation (ES = 1.0; change object = learner), c) enhanced cues + participation, d) feedback-corrective (mastery), or e) traditional group instruction. Results indicated that the average student in the enhanced cues + participation condition outperformed the control condition by 1.5 sigma.

Tenenbaum (1982,1986) -sixth and ninth grade subjects studying algebra and science were exposed to either: a) enhanced cues, participation, reinforcement and feedback-corrective procedure (a maximal instructional condition), conventional group instruction (a minimal instructional condition), or c) a mastery learning condition (lying between the two extremes). Results indicated an advantage of 1.7 sigma for the combined treatment over the control condition.
<table><caption>Table 1
Effect of Selected Alterable Variables on Student Achievement</caption><thead><tr><th>Object of change<sup>1</sup></th><th>Variable</th><th>Effect size<sup>2</sup></th><th>Percentile equivalent<sup>3</sup></th></tr></thead><tbody><tr><td>D</td><td>Tutorial instruction</td><td>2.00</td><td>98</td></tr><tr><td>D</td><td>Reinforcement</td><td>1.20</td><td></td></tr><tr><td>A</td><td>Feedback-corrective (Mastery Learning)</td><td>1.00</td><td>84</td></tr><tr><td>D</td><td>Cues and explanations</td><td>1.00</td><td></td></tr><tr><td>A/D</td><td>Student classroom participation</td><td>1.00</td><td></td></tr><tr><td>A</td><td>Student time on task</td><td>1.00</td><td></td></tr><tr><td>A</td><td>Improved reading/study skill</td><td>1.00</td><td></td></tr><tr><td>C</td><td>Cooperative learning</td><td>.80</td><td>79</td></tr><tr><td>D</td><td>Homework (graded)</td><td>.80</td><td></td></tr><tr><td>D</td><td>Classroom morale</td><td>.60</td><td>73</td></tr><tr><td>A</td><td>Initial cognitive prerequisites</td><td>.60</td><td></td></tr><tr><td>C</td><td>Home environment intervention</td><td>.50</td><td>69</td></tr><tr><td>D</td><td>Peer and cross-age remedial tutoring</td><td>.40</td><td>66</td></tr><tr><td>D</td><td>Homework (assigned)</td><td>.30</td><td>62</td></tr><tr><td>D</td><td>Higher order questions</td><td>.30</td><td></td></tr><tr><td>D/B</td><td>New science & math curricula</td><td>.30</td><td></td></tr><tr><td>D</td><td>Teacher expectancy</td><td>.30</td><td></td></tr><tr><td>C</td><td>Peer group influence</td><td>.20</td><td>58</td></tr><tr><td>B</td><td>Advance organizers</td><td>.20</td><td></td></tr></tbody></table>

Note: From Bloom, B. S. (1984) and Walberg, H. J. (1984).

¹ A = the learner; B = the instructional materials; C = the home environment or peer group; D = the teacher

² Effect size = $\mu_{experimental group} - \mu_{control group} / SD_{control}$

³ Percentile equivalent = percentage of experimental distribution that exceeds the mean of the control distribution.

* Leyton (1983) — ninth grade subjects studying French as a Second Language and algebra were exposed to either: a) enhanced initial cognitive prerequisites (ES = .60; change object = learner) + mastery learning, b) conventional instruction, c) mastery learning, or d) conventional group instruction supplemented with enhanced cognitive prerequisites. Results indicated that the mean of the prerequisites + mastery condition exceeded the control mean by 1.6 sigma.

* Mevarech (1985) — a) cooperative learning (ES = .80; change object = the home environment or peer group) + mastery learning (ES = 1.0; change object = learner), b) cooperative learning alone, c) mastery learning alone or d) traditional group
instruction. Results indicated that the average student in the
mastery learning condition outperformed the control condition by .5
sigma and the combined strategy condition performed at the .8
sigma level.

More or less, these studies have confirmed the underlying premise that alterable variables can be combined successfully to enhance learning. Of possibly greater interest is the fact that in some studies, lower achieving students (determined on the basis of pretest results) were found to equal or outperform so called higher achieving students (Burke, 1983; Leyton, 1983; Tenenbaum, 1986).

In addition to determining whether the combining strategy works, it is also important to know the conditions under which it works. Several tentative "rules" have been devised to aid in selecting variables for combination. Bloom (1984) speculates that variables involving different objects of the change process (see Table 1) are more likely to produce additive results than variables involving the same object. In several studies this was used as a rule of thumb for selecting variables (Nordin, 1981; Mevarech, 1985; Tenenbaum, 1982, 1986). In another study (Leyton, 1983), a different approach was used – that variables implicating the same object be combined as long as they occur at different times in the teaching/learning process. From the limited results reported above it tentatively appears that both rules are appropriate.

QUESTIONS THAT REMAIN

Despite the impressive cognitive outcomes attained in the forgoing litera-
ture, there are several issues regarding the combination of instructional
variables which remain unclear. First of all, it is not yet known whether the
cumulative effect sizes obtained in the above studies can be replicated with
older university-level students. Unfortunately, little help can be derived from
the meta-analyses upon which the effect size estimates were based (Lysakow-
ski & Walberg, 1981; Walberg, 1984) since the literature reflects the bias of
almost exclusive sampling from elementary and secondary students. Thus, the
extent to which the calculated effect sizes are representative of the results
achievable in higher education is unknown.

Second, it remains to be determined how to go about choosing variables.
Because not all variables in Table 1 have been investigated, the "rules"
provided by Bloom (1984) are little more than suggestions at the present time.
Here are some of the questions that remain regarding the appropriateness of
various combinations: Are all variables equally combinable, or are some
particularly good or bad matches? Since only variables with effect sizes of 5 or
more have been investigated, how will variables with lower effect sizes per-
form? All but one study (Nordin, 1981) has used mastery learning as one of the
constituents. Is mastery a necessary ingredient to achieve the "two sigma
effect," or can other variables with high effect sizes, like cooperative learning,
be applied successfully?
A final issue which requires clarification in future research, concerns Bloom's classification of instructional variables into four distinct categories. Closer inspection of the variables listed in Table 1 raises the question whether the direct object of the change process for each variable is as absolute and exclusive as Bloom implies. For instance, cooperative learning may not only affect changes in the peer group, but also alter the role of both teacher and student in the instructional process. Even the materials may be implicated in designing cooperative learning sessions. Similarly, peer and cross-age remedial tutoring may be seen as involving dramatic changes in peer group relations, possibly to the same extent as it affects changes in the role of the teacher.

## VARIABLES IN THIS STUDY

Three alterable variables from Table 1 were selected for inclusion in the study. Cooperative Learning (ES = .80, object of change = peer group relations), Student Classroom Participation (ES = 1.0, object of change = the learner and teacher) and Advance Organizers (ES = .20, object of change = the instructional materials), when added together, equal the hypothetical effect size of one-to-onetutoring (ES =2.0). Following is a brief review of these variables considered separately.

### Cooperative Learning

According to Johnson and Johnson (1983) there are three ways that instructional goals can be structured in the classroom: cooperatively, competitively and individually. The literature of cooperative learning, the strategy used in this study to affect peer group relations, suggests that through its use, a wide variety of academic and social outcomes may be achieved (Moskowitz, Malvin, Schaeffer & Schaps, 1985). The reported benefits of the approach include improved interpersonal relations (Blaney, Stephen, Rosenfield, Aronson & Sikes, 1977; DeVries & Slavin, 1978; Garibaldi, 1979) such as cross-ethnic relationships (Cook, 1978; Hansell & Slavin, 1981; Weigel, Wiser & Cook, 1975), cross-sex relationships (Slavin, 1985) and greater acceptance of handicapped students (Johnson & Johnson, 1983; Johnson, Johnson & Rynders, 1981; Madden & Slavin, 1983). Cooperative learning has also been found to increase students' attitudes toward themselves (Blaney, et al., 1977), and their peers, their teachers and their schools (Duin, 1984; Sharan, 1980). There is less agreement about the benefits of cooperative learning for improving cognitive performance of students, however. Several meta-analyses have asserted the benefits of cooperative learning for all but the most concrete, repetitive tasks (Johnson & Johnson, 1974; Johnson, Maruyama, Johnson, Nelson & Skon, 1981; Sharan, 1980). However, cognitive achievement results reported in several studies have indicated the possible presence of aptitude x treatment interactions. These have been as likely to favor high ability learners (Hulten & DeVries, 1976; Webb & Kenderski, 1982) as low ability learners
(Edwards, DeVries & Snyder, 1972; Slavin & Oickle, 1981). Some studies have noted the presence of a curvilinear interaction whereby high and low learners profit from the treatment, but middle ability learners perform best on their own (Peterson, Janicki & Swing, 1981; Webb, 1977).

Only a small proportion of the studies of cooperative learning have been performed in college and university classrooms. These studies (Fraser, Bea-man, Diener & Kelem, 1977; Haines & McKeachie, 1967; McClintock & Sonquist, 1976; Smith, Johnson & Johnson, 1981; Cox, 1984; Duin, 1984; Hamilton, 1976) tend to indicate a positive influence for cooperative learning on the achievement and attitudes of older students.

The cooperative learning instructional strategy is actually made up of a variety of different methods. The most extensively researched methods are the Student Team Learning methods developed by DeVries, Slavin and Edwards (Slavin, 1980). These methods include Student Teams Achievement Divisions (STAD), Teams-Games-Tournament (TGT), Jigsaw II, and Team Assisted Individualization (TAD). Other methods include the original Jigsaw strategy, the Learning Together model, and the Group-Investigation model.

STAD, TGT and TAI are highly structured, and entail clearly specified group tasks and group rewards. Group Investigation and Learning Together, by contrast, grant greater autonomy to students, and have a less well specified reward structure. The original form of Jigsaw also does not include formal group rewards. Generally, the literature suggests that methods which employ specific group rewards, based on group members' individual learning performances, and which stress individual accountability, are more effective at promoting cognitive achievement than methods which do not (Slavin, 1983).

## Student Classroom Participation

Lysakowski and Walberg (1982) define participation as, "the extent to which the student actively participates or engages in the learning process" (p. 560). In so far as this occurs within a classroom, it may be considered classroom participation. Elsewhere referred to as "active learning" (Bouton & Garth; Brothen, 1986) or "student involvement" (Mallor, Near & Sorcenelli, 1981), student participation typically involves the use of small learning groups, and requires students to work together on tasks in order to learn a prescribed set of concepts or skills. As students use their own resources and each other to work through the content to be learned, a process of active discovery takes place (Brothen, 1986).

The literature on student participation suggests that the technique may be applied effectively across a wide variety of learning situations. Ameta-analysis by Lysakowski & Walberg (1982) has shown the positive effects of participation to be constant from elementary school through college, and across socio-economic levels, races, private and public schools, and community types. Bouton and Garth (1983) maintain that in order for student classroom participation to effectively influence learning, two major elements must be present: a) an active learning process, promoted by student conversation in
groups; and b) instructor expertise and guidance through structured learning tasks. It is not sufficient to simply increase discussion or replace lectures with group work. Both elements – structured tasks and peer participation must be present.

### *Advance Organizers*

The premise behind the application of advance organizers is that, “the learning and retention of unfamiliar but meaningful verbal material can be facilitated by the advance introduction of relevant subsuming concepts (organizers)” (Ausubel, 1960, p. 267). There has been disagreement over the effectiveness of advance organizers in promoting learning compared with no advance organizer control conditions. Initial studies, as well as some reported recently (Ausubel & Youseff, 1963; Ausubel & Fitzgerald, 1961, 1962; Fitzgerald & Ausubel, 1963; Allen, 1969; Scandura & Wells, 1967; Grotelueschen & Sjogren, 1968; Levine & Loerinc, 1985; Krahn & Blanchaer, 1986; Green, 1986), have found advance organizers generally useful in promoting learning and retention over a variety of contents. However, several studies have found the opposite (Barron, 1971; Bauman, Glass & Harrington, 1969; Feller, 1973). Ameta-analysis of 135 advance organizer studies (Luiten, Ames & Akerson, 1980) found a mean effect size of .21 to be associated with the use of advance organizers.

The purpose of the present study was to assess the cognitive, affective and sociometric outcomes of a university-level unit of group instruction, developed by way of instructional systems design, and incorporating the three instructional strategies just reviewed: cooperative learning, enhanced student participation, and advance organizers. The extent to which this theoretically “maximal” mix of variables approximated the effectiveness of one-to-one tutoring was of primary concern. The study also sought to clarify three issues mentioned previously: a) the effectiveness of the technique with older students; b) the effectiveness of cooperative learning combined with other lower-order variables; and c) the use of variables other than mastery learning.

## METHOD

### *Subjects*

Subjects were 133 undergraduate students taking an educational psychology course in the Department of Education at Concordia University. They were predominantly female anglophones with a modal age of 19 years.

### *Design*

The study may be characterized as pre-post non-equivalent control group design (Campbell & Stanley, 1963) -random assignment of subjects to treat-
ments was impossible. There were two levels of treatment (i.e., the combined variables), taught by different instructors. A control group, taught by one of the two instructors, received conventional lecture-based instruction. The comparison between the conditions with a common instructor was of greatest interest, because it was in this comparison that instructor was held constant. The comparison between the two treatment conditions was of interest because it attempted to assess the differential results that might accrue from different instructors using the same procedure. This was an attempt to generalize the findings beyond a single instructor.

## Materials

**Instructional unit.** A systematically designed instructional unit, incorporating cooperative learning, enhanced student classroom participation, and advance organizers, constituted the instruction for the treatment condition. The unit was developed according to the principles of instructional systems design, and addressed the issue of learning theories, the content normally covered during the first five weeks of the winter semester. The unit consisted of twelve and one-half hours of instruction, A series of lectures, covering the same course content, and based on the same instructional analysis as the treatment conditions, was developed by one of the two course instructors. The rationale for and the procedures used in designing the instructional treatments are described in the Procedures section of this article.

**Prior achievement.** Grades from the first term of this course were used to construct a measure of prior achievement. Expressed as a percentage, this score was calculated for each student based upon the combined scores on two exams and a term paper.

**Cognitive measures.** Subjects' cognitive knowledge of the content covered by the instructional unit was measured by way of a cognitive post-test, administered to both treatments and the control group. It consisted of 38 multiple-choice items, drawn from the content of the instructional unit.

**Affective measure.** A pencil-and-paper instrument assessing subjects' attitudes toward cooperative group work was administered to the treatment and control groups both prior to and following the five-week instructional period. The instrument consisted of 16 statements designed to assess the extent to which subjects agreed or disagreed, on a five-point scale, with commonly held notions concerning cooperative learning, group work and cooperation in general. The instrument was pilot tested and then modified, before it was presented to the target group.

**Sociometric measure.** Changes in social interaction among class members were assessed by way of a paper-and-pencil sociometric test, administered to the treatment groups only, both before and after exposure to the instructional treatments. The test was designed according to established sociometric principles, as defined by Northway (1967), and was used to determine the degree to which subjects in the two treatments were accepted by their groups. The instrument consisted of four questions which asked subjects to state with
whom, among the members of the class, they preferred to associate for specific activities, and in particular situations. Based on subjects' responses, two scores were calculated: a "social acceptance score" and an "emotional expansion score." The former is based on the number of choices received by each individual on each criterion, while the latter represents the number of people chosen by each individual.

## Procedure

*Instructional unit.* The instructional unit was designed according to a modified version of Dick and Carey's (1985) systems approach for designing instruction. Two additional steps were added – analysis of unit variables and a consideration of research and theories of learning. The steps in the models used in designing the instruction are as follows:

a) identify instructional goals;

b) conduct an instructional analysis;

c) identify entry behaviors and characteristics;

d) analyze unit variables;

e) write performance objectives;

f) develop criterion-referenced test items;

g) consider learning theory and research;

h) develop an instructional strategy;

i) develop and select instruction; and

j) design and conduct formative evaluation.

Formative evaluation was conducted in the following phases: a) expert review; b) one-to-one evaluation; and c) field evaluation.

The following section details the rationale for and steps employed in designing the combined learning treatments: cooperative learning, student classroom participation and advance organizers.

*Cooperative learning.* The specific cooperative learning method that was chosen for the purpose of the present study was Aronson's (1978) Jigsaw method. Originally designed to enhance performance of minority students in newly integrated, Texas public schools, the Jigsaw cooperative learning method involves the division of learning tasks among various groups of students (McDougall & Gimple, 1985).

Each group member was assigned a section of academic material to learn, and subsequently to teach group mates. Members from each group who were assigned, or chose, the same topic area, met in "expert groups," where they discussed and learned about their specific topic areas. Once they have became "experts" in their respective topics, students returned to their original groups, and took turns teaching their group mates what they had themselves learned. Individual students were then tested over the content for which they received individual grades.

Although mixed results have been reported for Jigsaw in terms of its effect
on academic performance, self-esteem and attitudes toward school
itz, et al., 1985), it was considered the most suitable cooperative learning
method for use in this study because of its match with the narrative, factual
information contained in the course, its ease of implementation by the course
instructors, and its lack of group reward structure. The instructors felt more
comfortable with individually-assigned, rather than group-assigned grades.

*Student classroom participation.* Participation during the classroom ses-
sions was not specially designed but came as a by-product of the cooperative
learning strategy. Since each student had the responsibility to both learn and
subsequently teach certain blocks of material, a primarily learner-directed
and discovery-oriented learning environment was created.

*Advance organizers.* At the beginning of each lesson, the instructor pro-
vided an advance organizer of the material to be covered in the lesson. Recall
of previously learned information, relevant to the lesson in question, was
stimulated at this point. Advance organizers were also provided to students
during the group-directed activities (i.e., as part of the cooperative learning
strategy) to pre-inform students of the tasks they were to accomplish and the
material to be learned.

*Administration of treatments.* In all, five units of instruction were designed and implemented with subjects in the two combined treatments. At the same time, identical content was being delivered, via lecture, to subjects in the control condition. Each instructional session lasted approximately two and one-halfhours during which students in the treatment conditions engaged in a variety of teacher-led and student-led activities, The following is a sample lesson plan:

Lesson 4

Topic: Information processing

Objectives: By the end of the lesson, the students were ex-
pected to be able to define information processing, as
well as name and explain the structures and
control processes in the brain involved in informa-
tion processing (according to the Atkinson-Shiffrin
model). In addition, the students were expected to
name and explain the various methods for improv-
ing learning and recall.

Summary of
Activities:

1. Instructor delivers short introductory lecture about information processing.
2. Students take part in two memory activities, led by the instructor.
3. Instructor informs students of the historical context of information processing.
4. Instructor provides advance organizer of topics to be covered in information processing.
5. Students take part in cooperative activity in which they learn various different aspects of the Atkinson-Shiffrin model of information processing, and subsequently teach what they have learned to rest of class.

6. Students take part in the Jigsaw learning activity, employing self-instructional packages; in expert groups, learn strategies for improving memory and recall, and subsequently teach what they have learned to other group members.

7. Instructor provides summary of content covered, and answers students' questions.

*Statistical analysis.* Cognitive achievement data were analyzed using analysis of covariance. The cognitive post-test served as the dependent variable and the prior achievement measure served as the covariate. Analysis of covariance assumes that the within cells slope ($b_{S/A}$) for all treatments is a reasonable approximation (not different within chance) of the slopes for individual treatments (i.e., $b_{S/Ai}$). As a result, homogeneity of regression was tested. The attitude data were analyzed by subjecting pre-test scores, for the 16-item inventory, to principle components analysis to locate blocks of items that were empirically related. Since PCA results in factors of correlated items that are orthogonal, related items by factor could be used as separate dependent variables. Multivariate analysis of covariance (with the pre-test serving as the covariate) was used to investigate the between-group hypothesis. The sociometric data were analyzed using the non-parametric sign test.

## RESULTS

The results of analyses of three measures are reported here: results associated with cognitive outcomes of the experiment, results associated with affective outcomes and results of the sociometric measure.

### *Cognitive Outcomes*

A test of prior achievement (henceforth referred to as pre-test) was used to measure learning up to the point of the administration of treatments and a post achievement test (post-test) measured achievement towards the course objectives after the treatments were administered. The post-test served as the dependent measure in the study and the pre-test was used as a covariate (See Table 2 for means and standard deviations), to remove otherwise unexplained variability in subjects' achievement before the start of the experiment. The cognitive post-test scores were analyzed by way of one-way analysis of covariance. The covariate was found to be a significant predictor of post-test scores $F = 53.44 (1, 127), p < .001$. However, there were no significant differences among the levels of treatment, $F = (2,127) = 2.92, p > .05$.
<table><caption>Table 2
Unadjusted Means, Standard Deviations, *r*, *r*² and *p* for the Prior Achievement and Post Achievement Measures</caption><thead><tr><th>Measures</th><th><em>M</em></th><th><em>SD</em></th><th><em>r</em></th><th><em>r</em><sup>2</sup></th><th><em>P</em></th></tr></thead><tbody><tr><td colspan="6"><strong>Cooperative Group 1 (Instructor 1)</strong></td></tr><tr><td>Prior Achievement<sup>1</sup></td><td>67.11</td><td>10.19</td><td></td><td></td><td></td></tr><tr><td>Post Achievement<sup>2</sup></td><td>28.13</td><td>4.06</td><td>.58</td><td>.33</td><td>.01</td></tr><tr><td colspan="6"><strong>Cooperative Group 2 (Instructor 2)</strong></td></tr><tr><td>Prior Achievement</td><td>63.80</td><td>11.60</td><td></td><td></td><td></td></tr><tr><td>Post Achievement</td><td>27.53</td><td>3.74</td><td>.33</td><td>.11</td><td>.02</td></tr><tr><td colspan="6"><strong>Control Group (Instructor 2)</strong></td></tr><tr><td>Prior Achievement</td><td>68.75</td><td>11.05</td><td></td><td></td><td></td></tr><tr><td>Post Achievement</td><td>29.88</td><td>3.23</td><td>.71</td><td>.50</td><td>.01</td></tr></tbody></table>

Note: Cooperative Group 1, *n* = 47; Cooperative Group 2, *n* = 43; Control Group, *n* = 41,

¹Prior achievement = % score based on combination of two exam scores plus a term paper.

²Post achievement = number of correct multiple choice items out of 38, administered at the end of the learning theories unit.

Further investigation of the interaction between the covariate and the individual treatments (i.e., sometimes called a test of homogeneity, a "slope test" or a test of parallelism) revealed a significant deviation from equal slopes for one of the three treatments, *F* (1, 126) = 4.54, *p* = .04. Since this test measures the divergence of individual treatment slopes from the other treatment slopes, a significant difference here indicates that for one group, the relationship between the pre-test and the post-test was different than for the other treatments. Figure 1 (see following page) shows this relationship among individual treatment regression lines. Correlation coefficients and associated statistics are shown in Table 2. These statistics indicate that for the Control Group, the pre-test and the post-test were highly correlated. For the Cooperative Group 1 a fairly high correlation was found and for Cooperative Group 2 a low correlation was found. A weak correlation between prior achievement and post achievement is predicted in several studies involving the combined instructional strategies (e.g., Burke, 1983; Tenenbaum, 1982). Under this condition, prior achievement becomes a less influential determinant of summative achievement than does the nature of the instruction. As a result, "lower" students are observed to perform better, while "higher" students continue to perform well.
<figure><img src="image_2.png" /><figcaption>Figure 1. Individual Treatment Regression Lines Illustrating Aptitude x Treatment Interaction.</figcaption></figure>

### Affective Outcomes: Cooperative Learning

Subjects' attitudes towards cooperative group work were analyzed in three steps – principle components analysis, multiple analysis of variance (MANCOVA) and discriminant functions analysis. The purpose was to uncover any between-group differences in students' attitudes that resulted from the various treatments. Since the instrument tapped different aspects of attitudes toward cooperative learning and group interaction, its items were not treated homogeneously, but reduced to separate sub-sets of like items. As a result of the possible presence of treatment effects in the post-test, the pre-test was used as the basis for establishing the presence of item sub-sets. Principle components analysis was conducted on the 16 pre-test items with subjects treated as a homogeneous sample.

The results of the principle components analysis revealed that 12 of the 16 items loaded highly on three factors (40% of the total variance was accounted for by these three factors). An interpretation of the three factors was carried out by attempting to associate conceptually homogeneous items in each separate factor. The first of the three factors appeared to address the pros and cons associated with group work (e.g., group grades are an unfair method of student evaluation), while the second concerned the practical aspects involved in the application of cooperative learning (e.g., group work enables more work to be accomplished in a short time). Finally, the third factor appeared to address, more generally, the notions of cooperation and competition (e.g., competition in the classroom motivates students to work harder).
Three attitudinal sub-tests were created by adding together post-test responses within each of the three factors (Factor 1 contained 5 items; Factor 2 contained 4 items; and Factor 3 contained 3 items) Between-group differences in post-test scores were then analyzed using MANCOVA, with the 12 items on the post-test broken into three sub-sets, serving as multiple dependent measures and the pre-test serving as the covariate. Amultivariate test of homogeneity of regression indicated that the assumption of parallel slopes was satisfied. Unadjusted means and standard deviations are provided in Table 3 (see following page).

The result of the MANCOVA revealed a significant main effect for treatments, F(6,174) = 5.12, *p* < .05. Inspection of the dimension reduction analysis in MANCOVA indicated that only one of the two potential vectors accounted for the majority of variance in group differences (99.78%), and so the three dependent measures were considered a multivariate set (i.e., univariate analysis was inappropriate).

Follow-up analyses of the multivariate treatment effects were conducted in discriminant functions analysis (Table 4 on the following page shows the results of these tests). Asignificant difference was found between the post-test multivariate means (i.e., group centroids) of Cooperative Group 2 and those of the Control Group. By examining the univariate post-test means in Table 3, one may conclude that, after treatment, subjects in the second cooperative group exhibited significantly higher attitudes towards cooperative group work than did their counterparts in the control condition. Significant differences were also found between the combined post-test means of the two cooperative groups and the Control Group.

## Sociometric Outcomes

Two sociometric measures were analyzed: changes in sociometric status (i.e., social acceptance), and changes in subjects' emotional expansion. Differences in pre- and post-test scores on both measures were analyzed by way of a sign test (i.e., dependent test of change in behavior for nominal data). Subjects in both Cooperative Group 1 (n = 48) and Cooperative Group 2 (n = 44) exhibited significant increase in sociometric status, C = 4.90, *p* < .05 and C = 6.40, *p* < .05, respectively The number of "isolates" (i.e., people unchosen) dropped from 11 on the pre-test, to six on the post-test for Cooperative Group 1, and from 13 to six for Cooperative Group 2.

A similar increasing trend was found for the emotional expansion measure. Subjects in both groups exhibited significant gains in emotional expansion, C = 13.76, *p* < .05 for Cooperative Group 1 (n = 33), and C = 19.60, *p* < .05 for Cooperative Group 2 (n = 32). From these results, it was concluded that exposure to the systematically designed instructional unit affected substantial gains in subjects' sociometric status and emotional expansion.
<table><caption>Table 3
Attitudes Towards Cooperative Group Work: Unadjusted Means and Standard Deviations</caption><thead><tr><th rowspan="2">Group</th><th colspan="2">Pre-test</th><th colspan="2">Post-test</th></tr><tr><th><em>M</em></th><th><em>S D</em></th><th><em>M</em></th><th><em>S D</em></th></tr></thead><tbody><tr><td colspan="5" style="text-align: center;">Factor 1</td></tr><tr><td>Cooperative Group 1</td><td>15.20</td><td>3.54</td><td>14.69</td><td>4.06</td></tr><tr><td>Cooperative Group 2</td><td>15.91</td><td>2.95</td><td>17.12</td><td>2.72</td></tr><tr><td>Control Group</td><td>14.00</td><td>4.34</td><td>12.38</td><td>3.79</td></tr><tr><td colspan="5" style="text-align: center;">Factor 2</td></tr><tr><td>Cooperative Group 1</td><td>14.63</td><td>2.43</td><td>14.77</td><td>2.47</td></tr><tr><td>Cooperative Group 2</td><td>15.30</td><td>2.51</td><td>15.09</td><td>2.34</td></tr><tr><td>Control Group</td><td>14.47</td><td>2.38</td><td>14.34</td><td>2.15</td></tr><tr><td colspan="5" style="text-align: center;">Factor 3</td></tr><tr><td>Cooperative Group 1</td><td>11.34</td><td>1.57</td><td>11.86</td><td>1.59</td></tr><tr><td>Cooperative Group 2</td><td>11.91</td><td>1.28</td><td>12.24</td><td>1.31</td></tr><tr><td>Control Group</td><td>11.69</td><td>1.28</td><td>11.34</td><td>1.31</td></tr></tbody></table>

Note: Cooperative Group 1, n = 35; Cooperative Group 2, n = 33; Control Group, n = 32.

<table><caption>Table 4
Results of Discriminant Functions Follow-up Analysis on Attitudinal Data</caption><thead><tr><th>Comparison</th><th><em>df</em></th><th><em>F</em></th><th><em>p</em></th></tr></thead><tbody><tr><td>Coop. 1 vs. Coop. 2</td><td>6.61</td><td>1.74</td><td>.13</td></tr><tr><td>Coop. 1 vs. Control</td><td>6.60</td><td>1.23</td><td>.30</td></tr><tr><td>Coop. 2 vs. Control</td><td>6.58</td><td>5.31</td><td>.01</td></tr><tr><td>Coop. 1 + 2 vs. Control</td><td>6.93</td><td>3.27</td><td>.01</td></tr></tbody></table>

DISCUSSION

Cognitive Outcomes

The primary question being asked in the present study was whether students, exposed to the combined cooperative learning/student participation/ advance organizer instructional strategy, would achieve significantly different cognitive post-test scores from those obtained by unexposed students and whether these scores would approach the two sigma effect of tutorial instruc-
tion. Clearly, this did not occur. That is not to say, however, that the combined instructional strategy did not influence cognitive learning whatsoever. While high, medium and low learners in Cooperative Group 1 and the Control Group performed at a level on the cognitive post-test that approximated their own performance on the prior achievement measure, lower ability subjects in the Cooperative Group 2 were observed to perform nearly as well as middle ability subjects. This means that the instructional treatment benefited the lower learners in the second treatment group. The weak correlation detected between prior achievement and post-test scores for this group, suggests that, in this group, the experimental treatment exerted a stronger influence on summative achievement than did student aptitude. There is evidence of this aptitude x treatment interaction in prior studies involving combined treatments (Burke, 1983; Tenenbaum, 1982), but it is not the most desirable outcome. By contrast, findings of aptitude x treatment interactions are fairly common in the literature of cooperative learning (e.g., Hulten & DeVries, 1986; Webb & Kenderski, 1982; Edwards, DeVries & Snyder, 1972; Slavin & Oickle, 1981; Peterson, Janicki & Swing, 1981; Webb, 1977). It is likely that the AT1 results achieved here with Cooperative Group 2 were influenced solely, or at least primarily, by the presence of the cooperative learning strategy.

The obvious question to ask, of course, is why the results obtained for the two combined treatments are not consistent, especially since these experimental groups were to be treated in essentially the same way. The answer to this question lies in the formative evaluation data (these data are not presented here). One of the outcomes of this analysis was that the unit pacing was inappropriate (i.e., there was too much material to cover), and as a result, the instructional strategy all but broke down in Cooperative Group 1. Cooperative Group 2, on the other hand, was not as affected by problems of pacing. It is not surprising, then, that the instructional strategy produced a more positive effect in the condition that provided the best test of it.

## Affective Outcomes

The affective measure included in this study appears to have detected differences that indicate a positive effect for the combined treatment conditions. From the results obtained from this measure, it is evident that the treatment attitudes of the two experimental groups combined were more positive than those of the unexposed (control) subjects. When examined individually, the post-treatment attitudes of Cooperative Group 2 exceeded those of the Control group, but those of Cooperative Group 1 did not. Here, as with the cognitive data, the differences between the two experimental treatments mirror the formative evaluation results.

## Sociometric Outcomes

In both treatments, the students' sociometric status and emotional expansion scores were found to have improved as result of exposure to the instructional treatment. However, because this analysis was performed within
treated groups only, it is impossible to determine if this occurred in the
untreated control group as well. As well, it is likely that the primary influence
in this result was the cooperative learning strategy. It is not the intention to
diminish the importance of these findings, since interaction among classmates
is clearly a desirable and somewhat unusual outcome in large undergraduate
courses. But it is unlikely that the combining strategy substantially influenced
these results (i.e., cooperative learning alone could have produced the same
outcomes). Likewise, it is unlikely that such results would accrue from any
combined treatment that did not involve enhanced student involvement, such
as cooperative learning, as an element.

*Issues in Designing Combined Instructional Treatments*

While it is difficult to discern the exact reasons for the failure of the
treatments to dramatically influence overall cognitive learning behavior, a
number of problems, with the experimental conditions and with the theory
itself, are candidates for consideration.

*Combining of treatments.* According to Bloom (1984), one of the major concerns of further research in combining instructional treatments is to uncover which two or three instructional variables can best be combined. He argues that to be successfully combined, variables must affect different objects of the instructional context, each contributed something unique to learning achievement. It is possible that some variables should not be combined because they compete with one another, they overlap one another or simply because one or more variables are not capable of producing the hypothesized effect on achievement.

*Pacing of instruction.* From the formative evaluation data, it is clear that the instructional potency of the design was severely minimized by the inappropriate pacing, especially in Cooperative Group 1. In almost every lesson, there was insufficient time provided to the student to allow them to integrate sufficiently the material they were learning. As a result, the grouped-based instructional activities became product-oriented rather than process-oriented. For example, in Cooperative Group 1 several instances were reported where students dictated answers to other students rather than teaching them. Consequently, one of the most important cognitive strengths of cooperative learning - namely, the clarification of concepts through oral review and explanation to others (Kohn, 1987) - was effectively nullified.

*Matching objectives and the instructional strategy.* Because of the introductory nature of the course in which this experiment was conducted, the objectives employed were relatively low-level (i.e., primarily knowledge and comprehension). While studies by Mevarech (1980, 1985), Tenenbaum (1986) and Slavin (1983) have shown that combining instructional treatments may influence both higher and lower mental processes, a greater benefit appears to accrue to higher forms of learning (Bloom, 1984). It is likely that skills such as problem solving are more amenable to combined treatments, especially when cooperative learning is among the variables being used.
*Overlap of instructional variables.* It is possible, even likely, that two of the three variables used in this experiment were too similar to have produced the combined effect proposed by Bloom. Cooperative learning is described by Bloom (1984) as affecting peer group relations, while enhanced student classroom participation is touted as an agent of influence affecting both the teacher and learner. In the present study, these designations are difficult to justify. The Jigsaw strategy not only alters the peer group structure of the classroom, it also changes the role of both student and teacher. Student becomes teacher, and teacher becomes facilitator and content expert. The same is true of student classroom participation. In fact, cooperative strategies represent on every structured means ofensuring studentparticipation. Hence, instead of these two strategies being complementary, they were in fact parallel, and highly similar in the influence they exerted on the learner, the teacher and the peer involvement. Since the rationale for choosing to combine specific strategies is guided by Bloom's classification system, the reliability of this system must surely be held in question.

*Appropriateness of cooperative learning.* Finally, it is conceivable that cooperative learning is an inappropriate strategy for combining. First, cooperative learning treatments have frequently been found to produce aptitude x treatment interactions, suggesting that it may not represent a reliable method of introducing general benefits across all kinds of students. Second, while the affective benefits of cooperative learning have been well established, its effect on cognitive learning has been less reliable. A meta-analysis by Michaels (1977), one of three that have appeared in the literature, asserts that competition is better than cooperation in fostering cognitive learning.

*Theoretical Uncertainties that Remain*

In the introduction, several theoretical uncertainties were addressed: a) the appropriateness of combining variables in higher education; b) the question of whether variables other than mastery can be additive; the issue of whether variables with lower effect sizes may be combined; and d) Bloom's classification of the instructional variables in question. This final section addresses, in speculative fashion, these issues.

*Combining variables in higher education.* It is possible that combining instructional strategies is inappropriate for higher education. Cooperative learning, in particular, requires greater amounts of in-class time to achieve learning objectives than does conventional lecture and lecture/discussion methods, and as a result may not be amenable to the greater quantities of information that are generally associated with college and university courses. It is also possible that some subject matters lend themselves better to alternative methods than do others. There is little evidence from prior studies or from the current one to shed light on this issue.

*Master learning as a necessary variable.* Since the intention of the combining strategy is to emulate one-to-one tutoring, then mastery learning -the single strategy which most closely resembles it-may well be a necessary condition for successfully achieving the two sigma effect. The only experiment
that has been performed to date which excludes mastery was one conducted by Nordin (1981) in Malaysia. This study was carefully designed so that it contained three essential factors: cues (stimulus); participation (response); and reward (reinforcement). It is unclear whether Nordin purposely aimed to approximate on-to-one tutoring, or if he was merely adhering to the instructional paradigm proposed by Dollard and Miller (1950) in which the three above features, plus motivation, are named as the essential features of human learning. What is clear is that all three factors used by Nordin are fundamental characteristics of one-to-one tutoring. It may be more important, then, to select variables that are components of tutoring, than to select variables that simply affect different objects in the instructional environment. The validity of this suggestion can only be determined in further research work.

*Variables with lower effect sizes.* Another uncertainty is the extent to which lower effect size variables, like advance organizers, contribute to the overall goal of substantially boosting cognitive achievement. Because of limitations beyond the control of the researchers, the variables in this study could not be effectively isolated to determine their relative affect on performance. Future research should include due consideration for these individual treatments in an attempt to determine the most parsimonious combination of alterable variables.

*Bloom's categories.* Finally, there is the issue of Bloom's so called "rule of thumb" for choosing variables to be combined. In the absence of mastery learning, a clear benefit to learning achievement, it is possible that Bloom's categories are far too general to perform effectively as prescriptions for success. It is not so much that the variables are mislabeled, but that depending on circumstances, different variables may perform similarly or may not exhibit the desired characteristics. For instance, the variable called feedback-corrective (mastery learning), may perform in essentially the same fashion as carefully graded homework assignments. Bloom describes the former as affecting the learner, and the latter as affecting the teacher. If the two overlap, the first nullifying the effects of the second, then surely the object being influenced is the same in both cases. In addition, if graded homework is not given sufficient in-class attention so that students become aware of their mistakes, it may have little effect than ungraded homework. Similarly, cooperative learning (object = peer group relations) may have overlapped student classroom participation (object = learner/teacher), each effectively cancelling the additive benefits of the other.

## CONCLUSION

Can instructional treatments be combined successfully in higher education courses to approximate the positive academic benefits of one-to-one tutoring? Clearly, this study has contributed little to answering this general question. It is only through studies like this one, however, that the limits and
potentials of any instructional theory can be sorted out. Future research may
well reveal that cooperative learning, in combination with the two other
variables tested here, or some yet untested combination, may yield the desired
effect. For now, however, we must suspend judgment and await additional tests
of Bloom's potentially important conception.

REFERENCES

Allen, D. I. (1969). Some effects of advance organizers and level of retention of written social studies material. (Doctoral dissertation, University of California, 1969). *Dissertation Abstracts International*, **30**, 4267A.

Anania, J. (1983). The infleunce ofinstructional conditions on student learning and achievement. *Evaluation in Education:An International Review Series*, 7(1), 1-92.

Aronson, J. (1978). *The jigsaw classroom*. Beverley Hills, CA: Sage.

Ausubel, D. P. (1960). In defense of advance organizers: A reply to the critics. *Review of Educational Research*, **48**(2), 251-257.

Ausubel, D. P., & Fitzgerald, D. (1961). The role of discriminability in meaningful verbal learning and retention. *Journal of Educational Psychology*, 25(5), 266-274.

Ausubel, D. P., & Youssef, M. (1963). Role of discriminability in meaningful parallel learning. *Journal of Educational Psychology*, **54**, 331-336.

Barron, R. F. (1971). *The effects of advance organizers upon the reception of learning and retention of general science content*. (Final report) (Project No. IB-030). Washington, DC: Department of Health, Education and Welfare. (ERIC Document Reproduction Service No. ED 061 554)

Bauman, D. J., Glass, G. V., & Harrington, S. A. (1969). *The effects of the position of an organizer on learning meaningful verbal materials* (Research Paper No. 24). Laboratory of Educational Research, University of Colorado.

Blaney, N. F., Stephan, C., Rosenfield, D., Aronson, E., & Sikes, J. (1977). Interdependence in the classroom: A field study. *Journal of Educational Psychology, 69*(2), 121-128.

Block, J. H. (Ed.) (1971). *Mastery learning*. New York: Holt, Rinehart & Winston.

Bloom, B. S. (1968). Learning for mastery. *UCLA Evaluation Comment*, *1*(2), 1-12.

Bloom, B. S. (1980). The new direction in educational research: Alterable variables. *Phi Delta Kappan*, **61**(6), 382-385.

Bloom, B. S. (1984). The two sigma problem: The search for methods of group instruction as effective as one-to-one tutoring. *Educational Researcher*, **13**(4), 4-16.

Bouton, C., & Grath, R. Y. (1983). Students in learning groups: Active learning through conversation. In C. Bouton & R. Y. Grath (Eds.), *Learning in New directions in teachingand learning* (pp. 73-81). San Francisco: Jossey Bass.
Brothen, T. F. (1986). Using active learning in large classes. In S. F. Schomberg (Ed.), *Strategies for active teaching and learning in university classrooms: A handbook of teaching strategies*. University of Minnesota. (ERIC Document Reproduction Service No. ED 276 356)

Burke, A. J. (1983). Student's potential for learning contrasted under tutorial and group approaches to instruction. (Doctoral dissertation, University of Chicago, 1983). *Dissertation Abstracts International*, 44(7), 2025A.

Campbell, D., & Stanley, J. (1963). *Experimental and quasi-experimental designs for research*. Chicago: Rand McNally.

Carroll, J. B. (1963). A model of school learning. *Teachers College Record*, 64(5), 723-733.

Cook, S. W. (1979). Interpersonal and attitudinal outcomes of cooperating interracial groups. *Journal of Research and Development in Education, 12*(1), 97-113.

Cox, J. (1984). *Cooperative learning: An innovative approach to teaching reading theory and practice*. Paper presented at the 28th meeting of the College Reading Association, Washington, DC (ERIC Document Reproduction Service No. ED 255 879)

DeVries, D. L., & Slavin, R. E. (1978). Teams-Games-Tournaments (TGT): Review of ten classroom experiments. *Journal of Research and Development in Education, 12*(1), 28-38.

Dick, W., & Carey, L. (1985). *The systematic design of instruction* (2nd ed.). Glenview, IL: Scott Foresman.

Dollard, J., & Miller, N. E. (1950). *Personality and psychotherapy*. New York: McGraw Hill.

Duin, A. H. (1984, May). *Implementing cooperative learning in the writing curriculum. What research shows and what you can do*. Paper presented at the annual meeting of the Minnesota Council of Teachers of English, Mankato, MN. (ERIC Document Reproduction Service No. ED 251 849).

Edwards, K J., DeVries, D. L., & Snyder, J. P. (1972). Games and teams: A winning combination. *Simulation and Gaming*, 3(3), 247-269.

Feller, W. A. (1973). The effects of two types of advance organizers and two types of spaced questions on the ability of a selected group of tenth grade biology students to recall, comprehend, and apply facts from written science material. (Doctoral dissertation, Temple University, 1973). *Dissertation Abstracts International*, 34,1766A.

Fitzgerald, D., & Ausubel, D. P. (1963). Cognitive versus affective factors in the learning and retention of controversial material. *Journal of Educational Psychology*, 54, 73-84.

Fraser, S. C., Beaman, A. L., Diener, E., & Kelem, R. T. (1977). Two, three or four heads are better than one: Modification of college performance by peer monitoring. *Journal of Educational Psychology*, 69(2), 101-108.

Garibaldi, A. M. (1979). Affective contributions of cooperative and group goal structures. *Journal of Educational Psychology*, 71(6), 788-794.

Green, T. G. (1986). The effect of structured worksheets on student performance. *Journal of Dental Education*, 50(10), 616-617.
Grotelueschen, A., & Sjogren, D. D. (1968). Effects of differentially structured introductory materials and learning tasks on learning and transfer. *American Educational Research Journal*, 5, 191-202.

Hamilton, J. D. (1976). The McMaster curriculum: A critique. *British Medical Journal*, 1, 1191-1196.

Hansell, S., & Slavin, R. E. (1981). Cooperative learning and the structure of inter-racial friendships. *Sociology of Education, 54*(2), 98-106.

Hulten, B. H., & DeVries, D. L. (1976). *Team competition and group practice: Effects of student achievement and attitudes* (Report No. 212). Baltimore, MD: Johns Hopkins University, Center for Social Organization of Schools. (ERIC Reproduction Service No. 154 021)

Johnson, D. W., & Johnson, R. T. (1974). Instructional goal structure: Cooperative, competitive or individualistic. *Review of Educational Research*, 44(2), 213-240.

Johnson, D. W., & Johnson, R. T. (1983). Effects of cooperative, competitive and individualistic learning experiences on social development. *Exceptional Children*, 49(4), 323-330.

Johnson, D. W., & Johnson, R. T., & Rynders, J. (1981). Effects of cooperative, competitive and individualistic experiences on self-esteem of handicapped and non-handicapped students. *Journal of Psychology, 108*(1), 31-34.

Johnson, D. W., Maruyama, Johnson, R. T., Nelson, D., & Skon, L. (1981). Effects of cooperative, competitive and individualistic goal structures on achievement: A meta-analysis. *Psychological Bulletin*, 89 (1), 47-62.

Kohn, A. (1987, October). Its hard to get left out of a pair. *Psychology Today*, 53-57.

Krahn, C. G., & Blanchaer, M. C. (1986). Using an advance organizer to improve knowledge application by medical students in computer-based clinical simulations. *Journal of Computer-Based Instruction*, 13(3), 71-74.

Levine, L. H., & Loerinc, B. M. (1985). Investigating the scope of an advance organizer for compiler concepts. *Journal of Educational Technology Systems*, 13(3), 175-183.

Leyton, F. S. (1983). The extent to which group instruction supplemented by mastery of the initial cognitive prerequisites approximates the learning effectiveness of tutorial methods. (Doctoral dissertation, University of Chicago, 1983). *Dissertation Abstracts International*, 44(4), 974A.

Lippitt, P., & Lippitt, R. (1968). The peer culture as a learning environment. *Childhood Education*, 47(3), 135-138.

Luiten, J., Ames, W., & Akerson, G. (1980). A meta-analysis of the effects of advance organizers on learning and retention. *American Educational Research Journal*, 17(2), 211-218.

Lysakowski, R. S., & Walberg, H. J. (1981). Instructional effects of cues, participation, and corrective feedback: A quantitative synthesis. *American Educational Research Journal*, 19(4), 559-578.

Madden, N., & Slavin, R. (1983). Mainstreaming students with mild handicaps: Academic and social outcomes. *Review of Educational Research*, 53(4), 519-569.
Mallor, J. P., Near, J. P., & Sorcinelli, M. D. (1981). Increasing student involvement in large lecture classes: Two case studies. *Journal of Experimental Learning and Simulation, 3* (3-4),191-196.

McClintock, E., & Sonquist, J. A. (1976). Cooperative task-oriented groups in a college classroom: A field application. *Journal of Educational Psychology, 68*(5), 588-596.

McDougall, K, & Gimple, D. (1985). *Cooperative learning strategies for teaching small group communication: Research and application*. Paper presented at the 71st annual meeting of the Speech Communication Association, Denver, CO. (ERIC Document Reproduction Service No. ED 271 800)

McKeachie, W. J. (1974). The decline and fall of the laws of learning. *Educational Researcher, 3*, 7-11.

Michaels, J. W. (1977). Classroom reward structures and academic performance. *Review of Educational Research, 47*(1), 87-98.

Moskowitz, J. M., Malvin, J. H., Schaeffer, G. A., & Schaps, E. (1985). Evaluation of Jigsaw, a cooperative learning technique. *Contemporary Educational Psychology, 10*, 104-112.

Nordin, A. B. (1981). Improving learning: An experiment in rural primary schools in Malaysia. *Evaluation in Education: An International Review Series, 4*(2), 143-263.

Northway, M. L. (1967). *A primer of sociometry* (2nd ed.). Toronto: University of Toronto Press.

Peterson, P. L., Janicki, T., & Swing, S. (1981). Ability x treatment interaction effects on children's learning in large group approaches. *American Educational Research Journal, 18*(4), 453-473.

Scandura, J. M., & Wells, H. N. (1967). Advance organizers in learning abstract mathematics. *American Educational Research Journal, 4*, 295-301.

Sharan, S. (1980). Cooperative learning in small groups: Recent methods and effects on achievement, attitudes and ethnic relations. *Review of Educational Research, 50*(2), 241-271.

Slavin, R. E. (1980). *Using student team learning*. Baltimore, MD: Center for Social Organization of Schools, Johns Hopkins University.

Slavin, R. E. (1983). *Cooperative learning*. New York: Long-man.

Slavin, R. E. (1985). Team-assisted individualization: Combined cooperative learning and individualized instruction in mathematics. In R. Slavin, S. Sharan, S. Kagan, R. Hertz-Lazarowitz, C. Webb, & B. Schmuck (Eds.), *Learning to cooperate: Cooperating to learn* (pp. 177-209). New York: Plenum Press.

Slavin, R. E., & Oickle, E. (1981). Effects of cooperative learning on teams on student achievement and race relations: Treatment by race relations. *Sociology of Education, 54*(3), 174-180.

Smith, K. A., Johnson, D. W., & Johnson, R. T. (1981). Structuring learning goals to meet the goals of engineering education. *Engineering Education, 72*(3), 221-226.
Tenenbaum, G. (1982). A method of group instruction which is as effective as one-to-one tutorial instruction. (Doctoral dissertation, University of Chicago, 1982). *Dissertation Abstracts International*, 43(6), 1822A.

Tenenbaum, G. (1986). The effect of quality of instruction on higher and lower mental processes, and on the prediction of summative achievement. *Journal of Educational Research*, 80(2), 105-114.

Walberg, H. J. (1984). Improving the productivity of American schools. *Educational Leadership*, 41(8), 19-27.

Webb, N. M. (1977). *Learning in individual and smallgroup settings* (Technical Report No. 7). Arlington, VA: Office of Naval Research, Personnel and Training Research Programs Office. (ERIC Document Reproduction Service No. ED 151 699)

Webb, N. M., & Kenderski, C. (1982, May). *Student interaction and learning in smallgroup and whole class settings*. Paper presented at the Conference on Student Diversity and the Organization, Processes and Uses of Instructional Groups in the Classroom, Madison, WI.

Weigel, R. H., Wiser, P. L., & Cook, S. W. (1975). Impact of cooperative learning experiences on cross-ethnic relations and attitudes. *Journal of Social Issues*, 31 (1),219-245.

## AUTHORS

Judith L. Amato is a graduate of the M. A. Program in Educational Technology at Concordia University.

Robert M. Bernard is an Associate Professor in Educational Technology at Concordia University, Department of Education, 1455 de Maisonneuve Blvd. W., Montreal, PQ H3G 1M8

Miranda D'Amico is a part-time Instructor in the Department of Education at Concordia University.

Bette DeBellefeuille is a Lecturer in the Early Childhood program at Concordia University.
Coming next:

**Special Issue on Interactive**
**Learning Technologies**

Volume 18, Number 3

Guest Edited by
Richard A. Schwier,
author of *Interactive Video*
Improving Idea Generation During
Decision Making in Small Group
Computer Conferences

Norman P. Archer

Abstract: Decision making in small group computer conferences tends to suffer from some of the same problems as face-to-face interactive groups. For example, there may be unequal participation, some group members may tend to contribute few ideas, and the group may focus too quickly on new ideas. This paper describes a technique for computer conferencing which should help to alleviate these problems. It is an adaptation and combination of the nominal group and rational decision making techniques, and provides a structure for the decision making process while at the same time it tends to improve group creativity by encouraging individual contribution. The technique has been used for simulated business decision making in student groups with sizes varying from 4 to 13 members.

INTRODUCTION

Computer conferencing and its close relative electronic mail have received considerable recognition as problem-solving and learning tools in educational institutions (Hiltz, 1986; Hiltz & Turoff, 1978; Kaye, 1987; Mason, 1987; McCreary & Van Duren, 1987; Rice & Case, 1983; Ujimoto & James, 1987; Welsch, 1982). An important aspect of using a computer mediated medium such as this is its impact on group behavior, and this has been investigated by, among others, Beckwith (1987), Kerr and Hiltz (1982), Kiesler, Siegel and McGuire (1984), McGuire, Kiesler and Siegel (1987), Stefik, Foster, Bobrow, Kahn, Lanning and Suchman (1987), Siegel, Dubrovsky, Kiesler and McGuire (1986), and Nunamaker, Applegate and Konsynski (1988).

The differences between group behavior within a computer conference as compared to face-to-face meetings have also required the development and adaptation of appropriate management techniques. In particular, the role of a human moderator in organizing, leading, and controlling computer conferences has been emphasized by a number of researchers (Hiltz & Turoff, 1978; Ujimoto & James, 1987; Stix, 1987). Stodolsky (1988) suggested a form of computer moderation for synchronous computer-mediated meetings, where time sequencing of participant contributions is a known problem (Hesse, Werner & Altman, 1988), but users tend to dislike such control mechanisms (Dubrovsky, Kiesler & Siegel, 1983).
Feenberg (1986) notes the major functions which characterize the computer conference moderating role as: setting context, norms, and agenda, recognition, prompting, weaving (summarizing and promoting unity), and meta-commenting. Kerr (1984) discusses the structured tasks and the roles of the computer conferencing moderator. Most are agreed that the moderator plays a key role in promoting a successful outcome of a decision-making conference, although a great deal of skill is required if the views of the group and not necessarily the moderator are to prevail in the final decision. One critical aspect is group motivation, but in the educational context as compared to the business or scientific environment and particularly for decision making, this is less of a concern to the moderator because individual contributions can be monitored, and individual achievement records provide the necessary motivation (Mason 1987).

Traditionally, the role of the moderator has been cast in the position of managing within the context of the computer conference equivalent of the interactive group meeting. Several meeting support methods have also been adapted to computer conferencing. The Delphi technique has been adapted by implementing a computer voting procedure (Turoff, 1972; Waggoner, 1987), and a modified Delphi technique has been used (Kerr & Hiltz, 1982) to collect date from expert panels. Nunamaker, Applegate and Konsynski (1988) have developed an electronic brainstorming tool for synchronous computer-mediated meetings.

A fairly recent development has seen experimentation with group decision support systems (GDSS) in business settings (Cook, Ellis, Graf, Rein & Smith, 1987; DeSanctis & Gallupe, 1985; Stefik, Foster, Bobrow, Kahn, Lanning & Suchman, 1987; Nunamaker, Applegate & Konsynski, 1988). Gallupe (1988) also reported on the application of a GDSS for teaching business cases. AGDSS consists of a set of software, hardware, and language components and procedures that support a group of people engaged in a decision-related meeting. While this is a fairly broad definition, GDSS implementation has usually been in a synchronous meeting environment with all the group members meeting in a well-equipped "decision room." A related system is the PCS system (Shaw, 1988) which could be used in a remote mode. PCS enables a number of individuals to interact through networked personal computers to develop mutual understanding of a problem domain through the use of repertory grid techniques. A survey by Gray (1986) indicates that GDSS may have a bright future, but technology requirements (for example further development in the use of artificial intelligence techniques), and the high cost of many of these systems are current inhibiting factors.

In this paper, we are concerned primarily with asynchronous decision-making by small groups, which includes the group product and community decision making classifications from the range of functions which McCreary and Van Duren (1987) have defined for computer conferencing in education. This involves group problem solving in such areas as the preparation of case reports, discussion papers, proposals, etc. A technique will be described which
improves group creativity through the adaptation of the nominal group technique (Van de Ven & Delbecq, 1974) to computer conferencing, within a particular decision making structure. This can be organized and managed in a straightforward manner by the conference moderator.

## PROBLEM SOLVING TECHNIQUES

Terry and Franklin (1982) describe five major approaches to problem solving: a) Routine; b) Scientific; c) Decisional; d) Creative; and e) Quantitative. The Routine approach is used when there is a known standard method to solve the problem. The Scientific approach (or Scientific Method) is well-known in academic research, and involves stating a proposition, investigating that proposition thoroughly using existing knowledge or by performing experiments, stating a tentative solution to the proposition, and cycling back to restate the proposition if necessary, The Decisional approach (often called Rational Decision-Making) appears in a variety of forms (Simon, 1960; Feldman & Arnold, 1983), and also is an iterative process. Activities which occur during the three phases of the Decisional (Rational Decision Making) approach appear in Table 1. Decision making may cycle several times through one or more of these phases before a final decision is reached.

<table><caption>Table 1
The Rational Decision Making Process</caption><tr><td>1) Intelligence</td><td>a) Assemble the known facts, and any facts which can be inferred from the problem context</td></tr><tr><td></td><td>b) Identify the problem(s) to be addressed</td></tr><tr><td>2) Design</td><td>c) List possible alternative solutions to the problem(s), along with their attributes</td></tr><tr><td>3) Choice</td><td>d) Select an appropriate decision based on the alternatives considered above</td></tr><tr><td></td><td>e) Set out an implementation procedure</td></tr></table>
The Creative problem solving approach is not necessarily a highly organized or structured approach to decision making. Its object, once the problem has been stated, is to encourage creativity and idea fluency, to verify and evaluate the proposed solution, and then to propose its application. The Quantitative approach is to formulate the problem, build a mathematical model to represent the system under study, and then to derive an answer from the model.

For the purpose of this discussion, let us consider a type of problem which occurs abundantly in the instructional environment, in the general class of non-recurrent and qualitative complex decision making, such as report preparation or case study assignments which are often done by small groups. Here, a quantitative measure of complexity which can be used is the number of potential alternative solutions (Payne, 1976). Problem solving in this instance cannot be handled by the Routine approach, since there are no standard procedures for such problems. Mathematical models typically cannot be used for complex qualitative problems, thus ruling out the Quantitative approach and leaving b), c), and d) as the most likely choices. Although there are similarities among these procedures, the Scientific approach lends itself well to the generation of new knowledge through experimentation, and the incorporation of that knowledge into the solution.

The Decisional or Creative approaches are more suitable for the group solution of complex problems to be considered here. The Decisional approach aids the group by imposing a structure on the problem-solving process, whereas the Creative approach mainly concentrates on improving the creativity of solutions proposed by group members. Group problem solving tools which fit into the class of Creative decision making include Brainstorming (Osborn, 1957), and the Nominal Group and Delphi techniques (Delbecq, Van de Ven, & Gustafson, 1975).

Several of these techniques have been adapted for use in asynchronous computer conferencing (CCA). The most widely used method for CCA is an adaptation of the common face-to-face Interactive Group (IG) method, where participants read messages that others have added, and then respond with their own messages. This is the most efficient and natural technique for simple information interchange, but it is not necessarily the most creative. Also, if decisions are to be made and reports are to be developed by deadline time, the process must also be managed properly through a moderator who is responsible for setting up a plan, organizing a procedure, and controlling the implementation of the process through to the final decision-making and report-generating stages.

The Nominal Group (NG) technique was originally developed as a face-to-face method for improving group creativity, by emphasizing individual contribution and avoiding unequal participation or group dominance by individuals. Delbecq, Van de Ven, and Gustafson (1975) describe the procedures for the standard NG technique, which is a structured group meeting in which the participants sit around a table. They initially do not speak to one another, but
write ideas on paper relating to the topic at hand. Then each individual, in round-robin fashion, presents one idea from his or her private list. A recorder writes the idea on a flip chart in full view of the group. When all ideas have been listed, discussion follows, to clarify ideas or to express support or non-support. Group decisions are by majority vote.

The main advantage claimed for the NG over the IG approach is the larger number of alternatives generated, enhancing the likelihood of a better decision. Burton (1987) notes that interacting groups are better at synthesizing and evaluating information, and achieving group consensus. Nominal groups are better at fact finding, idea generation, establishing objectives and priorities, and reduction of errors and estimation variability. He also suggests that a contingency approach might be appropriate, where the technique selected depends upon the nature of the problem, the group and the participants. We would expect some of these advantages and disadvantages to carry over to the computer conferencing forms of both these techniques, but this may be tempered by the fact that the literature on computer conferencing consistently indicates, for example, that there is more equal participation with computer-mediated conferencing than with face-to-face communication (Siegel, Dubrovsky, Kiesler & McGuire, 1988).

## THE NOMINAL GROUP ADAPTATION FOR COMPUTER CONFERENCING

There are many ways to structure a conference discussion, but if decisions are to be made, a good way to structure the agenda is through the Decisional approach. It is also possible to superimpose on the Decisional approach an adapted form of the nominal group technique, with the resulting structure being called the Computer Conferencing Nominal Asynchronous (CCNA) approach. This combines the creative advantages of the nominal group technique with the logical management structure of the rational decision making approach.

Using the CCNA structure, group members send their initial comments to the moderator by private electronic mail. The moderator summarizes the comments when all group members have contributed, and puts the summary into electronic conference form, to which group members can comment and add. When each phase (Intelligence, Design or Choice) of the conference is complete, the cycle of private mail and then conference discussion is repeated, until a consensus has been reached in the final decision and implementation phase. In this way, CCNA differs from the standard NG technique because, after the initial presentation of a summary of individual member opinions, the group works towards a consensus rather than using a majority vote. Consensus decision making is appropriate with the CCNA technique because interactive group problems such as unequal participation, which the nominal group approach is designed to overcome, tend to be less dominant in the computer
conference (Siegel, Dubrovsky, Kiesler & McGuire, 1986; Kerr & Hiltz, 1982). At the same time, CCNA exhibits an advantage over the CCA methodology, as does NG over IG (Burton, 1987), since it aids in the generation of ideas by encouraging more individual contributions.

CCNA also tends to inhibit motivation loss, coordination loss, and diffusion of responsibility which are adverse effects caused by large interactive groups (Feldman & Arnold, 1983). Coordination loss is lessened because students can interact at their own convenience while meeting deadlines imposed by the moderator. Overall responsibility is assigned to the moderator and, if the moderator is skilful, the group will be able to meet its deadlines. Group members are motivated because they are aware that their individual contributions are recorded, and those members who let the group down may be penalized.

The definite generation of more alternatives for the purpose of decision making does not necessarily result in better decisions. In fact, since we are limited in our cognitive ability to process information, only a limited number of alternatives can be assembled at one time, and as a result we tend to satisice (Simon, 1960), or select the best of the alternatives which can be considered rationally, rather than to optimize over all possible (or known) alternatives. In a complex situation, there may be hundreds or thousands of alternatives, and it is virtually impossible for an individual or group to search the problem space completely. However, the collective action of a group will normally allow the logical examination of more alternatives than a single individual, tending to lead to a better decision.

In the experiment reported here, measurements were performed on the impact of group size on the number of alternatives generated and the quality of the final decision, when using the CCNA method. At the same time, the general feasibility of the approach was examined. The educational objectives of the exercise included:

1) to expose participants to current computer-supported conferencing technology;

2) to enable participants to become familiar with the advantages and disadvantages of computer-supported decision-making; and

3) to develop an understanding among the participants of how to improve idea generation in a small group.

## METHOD

The study was carried out with four groups of MBA students taking information systems courses. Each group analyzed a business mini-case entitled "Quality Assurance Analyst Certification" (Senn, 1987). The groups, designated A, B, C and D, had 4, 4, 7 and 13 members, respectively. The conferencing system used was CoSy(R) (Van Duren, 1986) running on a VAX 11/
780. Students could access the system at their convenience using terminals or microcomputers running terminal emulation software. All of the students had some computer-related experience. Prior to the start of the experiment, the students were trained by working through an example computer conferencing case. The same conference moderator was responsible for all four conferences, and was experienced in the case material but did not participate in the discussions. The moderator's duties were to impose deadlines and to generate summaries at the end of each of the three phases of the decision-making process. To ensure that no moderator bias or distortion entered into the discussion, participants were required to enter their comments in point form (usually one line or less), and the moderator "summarized" (or rather, sorted) by grouping together related comments by the group members into a set of structured facts, alternatives or decisions (depending upon the decision-making phase underway) as the conference proceeded.

The groups were given two weeks to analyze the case, preparing their results with the help of the CCNA methodology. Each phase of each conference was recorded and then examined upon completion of the conference.

Research consistently suggests that groups make better judgments than individuals when the group members have varied skills and experiences (Shaw, 1981). To apply this in the current situation, the group assignment technique developed by Beheshtian-Ardekani and Mahmood (1986) was used. Each student filled out a questionnaire which contained a series of questions related to experience, background and grades in related courses in both computer use and general management (both areas related to the case to be studied). Weights were assigned to these questions and the total score on the questionnaires used as a ranking to assign students by cycling through the groups while working down the ranking. Because this project involved groups of unequal size, the objective was to set up groups with the same average weighted questionnaire scores. This assignment technique achieves the objective of high intra-group heterogeneity with minimum inter-group differences in average skill levels, and should be used whenever feasible in making group assignments.

## RESULTS

The number of alternatives (unique within a group) generated during Phase II of the Decisional process was significant ($p = .03$, R<sup>2</sup> (adj.) = .90) when regressed against group size. Thus the number of alternatives generated using this method is largely explained by group size, as one might expect since group members work alone in generating initial alternatives and do not see each other's alternatives until the summary has been prepared by the moderator. This encourages individual contributions to a broader set of alternatives.

In a complex problem such as the case studied here, there will be many possible alternatives. It is therefore expected that the duplication of alterna-
<figure><img src="image_3.png" /><figcaption>Figure 1. Alternatives Defined by Each Group</figcaption></figure>

tives selected by group members should not be high. This was borne out by the results shown in Figure 1, where the number of alternatives is plotted for each of the four groups against the number of times an alternative is mentioned. The number of distinct alternatives mentioned by each group was 9, 12, 18 and 25 for groups A, B, C and D, respectively. Of these, fully 64% were mentioned only once in any particular group (duplications between groups not excluded in the totals). The one alternative which usually showed the largest number of duplications in any group was "do nothing", which is always an alternative. Of further interest from the within-group alternatives generated is the fact that, except for the "do nothing" alternative, the proposed alternatives are typically not mutually exclusive in that there could be some overlap and complementarity, and the final group decision usually included a combination of alternatives rather than any single one. For groups A, B, C, and D, 3, 3, 3 and 4 alternatives respectively were used in the final proposed decisions and implementations.

The quality of the group decisions was also evaluated by three experienced judges, who rated the case reports on five quality-related questions (Archer, 1988) on an 11-point Likert-type scale. The average rating of these five questions was then used as a comprehensive decision quality measure. The
reliability of the judge data was .60. A regression of the comprehensive quality measure against group size was not significant (p = .06, R² (adj.) = .23).

DISCUSSION

In a controlled educational environment such as that offered by computer conferencing, it is important to encourage students both to be creative and to learn from their colleagues as well as from other sources. The CCNA technique appears to enhance the creative process, while the associated structures described for this experiment provide an organized and controllable environment which can be managed by a skilled moderator. The most difficult and time-consuming aspect of managing such a conference is the summarizing required, but this is true of any decision-making conference and will be a continuing problem until natural language understanding (NLU) software with the necessary capability is available to aid in this task. The time required for summarizing can and should be alleviated by requiring participants to enter their comments in abbreviated point form. Archer (1988) compared four types of meeting techniques (including CCNA) with equal group sizes. In that comparison, it was found that the amount of time spent by conference moderators was not significantly different among the four techniques, so there is no time penalty to the moderator who uses CCNA.

For larger groups, there is a litany of potential problems (Feldman & Arnold, 1983) which can militate against a quality solution in the normal face-to-face or asynchronous computer conferencing group meeting formats. This is normally true for group sizes larger than five, although Nunamaker, Applegate and Konsynski (1988) reported that participant satisfaction actually increased with group size in their GDSS experiments (this may be due to the fact that their participants met as groups, with computer-aided support). The CCNA approach also tends to alleviate this problem and as a result the potentially high collective creativity of the larger group can be used to advantage in preparing solutions which reflect the views of a larger number of participants.

Among the negative aspects of computer conferencing is that, on average, people prefer face-to-face meeting techniques rather than computer conferencing. This is not alleviated by the CCNA technique; in fact the original nominal group technique, although it has proven to be better in a number of ways for creative problem solving than interactive group techniques, has not achieved widespread use in the world of business (claims of its proponents aside). This is due to the same reason that computer conferencing has not been widely accepted in business (even though it has been well-received in academia). That is, people still prefer to interact directly if at all possible. The enhanced educational benefits of improved techniques may not be met with initial enthusiasm by students but, as indicated by Mason (1987), the two major elements to the effective use of learning groups are: active participation by students in the discussion, and faculty expertise and guidance provided
through structured tasks. These two elements are prominent features of the approach described in this paper, and it should also be noted that the CCNA technique greatly reduces the temporal problems of participants, comment sequencing which are usually attributed to asynchronous computer conferencing (Hesse, Werner, & Altman, 1988). This is because participants are required to focus on only one topic at any given time, so there is little likelihood that other sequences of interactions unrelated to that topic will occur. While the instructor did not intervene in the discussions described for this experiment in order to avoid biasing the results, it would enhance student learning if the instructor did take part in the discussions at the end of each of the three decision-making phases, in order to promote discussion in areas not already explored by the participants.

Those computer conferencing techniques which can be shown to improve the ability of participants to work together in business and academic environments will ultimately become more widely accepted if easy-to-use user interfaces can be developed for computer-mediated conferencing, and this is a thrust of current Group Decision Support System research (Nunamaker, Applegate, & Konsynski, 1988). In fact, CCNA could also be adapted to synchronous mode (CCNS) as an adjunct to GDSS, thus providing a technique for improving group creativity in synchronous computer-mediated environments.

The participants agreed that the educational objectives of the computer conferencing exercise (including, of course, learning the case material!) were met by the CCNA technique. These objectives are independent of spatial separation of group members, indicating obvious potential for the use of CCNA in distance education, although this was not explored in this study. Distance education is receiving more attention in North America, albeit perhaps not with the concentrated effort seen in the U.K. (Mason, 1988). The CCNA approach could aid in supporting computer-mediated distance learning through, for example, the process of assigning discussion group membership, the structured nature of the group decision-making process, and the defined manner in which group members contribute to the discussion independently of others. Each of these attributes of CCNA should encourage individuals to learn on their own, from other group members, and from the instructor, independent of spatial separation, and with temporal constraints which are not nearly as severe as in synchronous conferencing.

## REFERENCES

* Archer, N. P. (1988). A comparison of computer conferences with face-to-face meetings for small group business decisions. Faculty of Business Working Paper #298. Hamilton, ON: McMaster University.

* Beckwith, D. (1987). Group problem-solving via computer conferencing: The realizable potential. *Canadian Journal of Educational Communication, 16*(2), 89-106.
Beheshtian-Ardekani, M., & Mahmood, M.A. (1986). Development and validation of a tool for assigning students to groups for class projects. Decision Sciences, 17, 92-113.

Burton, G. E. (1987). The clustering effect: An idea-generation phenomenon during nominal grouping. *Small Group Behavior, 18*, 224-238.

Cook, P., Ellis, G., Graf, M., Rein, G., & Smith, T. (1987). Project Nick: Meetings augmentation and analysis. *ACM Trans. Office Information Systems*, 5, 132-146.

Delbecq, A. L., Van de Ven, A. H., & Gustafson, D. H. (1975). *Group techniques for program planning: A guide to nominal group and delphi processes*. Glenview, IL: Scott Foresman Company.

DeSanctis, G., & Gallupe, B. (1985, Winter). Group decision support systems: A new frontier. *Database*.

Dubrovsky, V., Kiesler, S., & Siegel, J. (1983, October). *Human factors in computer-mediated communication*. Paper presented at the Human Factors Society, Baltimore MD.

Feenberg, A. (1986). Network design: An operating manual for computer conferencing. *IEEE Transactions on Professional Communications*, PC-29, 2-18.

Feldman, D. C., & Arnold, H. J. (1983). *Managing individual and group behavior in organizations*. New York, NY: McGraw-Hill.

Gallupe, B. (1988). *Case analysis in the 1990's: Using a group decision support system for case analysis*. School of Business Working Paper #88-5. Kingston, ON: Queen's University.

Gray, P. (1986). Group decision support systems. In E. R. McLean and H. G. Sol (eds.) *Decision support systems: A decade in perspective* (157-171). Amsterdam: North-Holland.

Hesse, B. W., Werner, C. M., & Altman, I. (1988). Temporal aspects of computer-mediated communications. *Computers in Human Behavior*, 4, 147-165.

Hiltz, S. R. (1986). The virtual classroom: Using computer-mediated communication for university teaching. *Journal of Communications*, 36, 95-104.

Hiltz, S. R., & Turoff, M. (1978). The network nation: human communication via computer. Reading, MA: Addison-Wesley.

Kaye, T. (1987). Introducing computer-mediated communication into a distance education system. *Canadian Journal of Educational Communication*, 16(2), 153-166.

Kerr, E. B. (1986, March). Electronic leadership: A guide to moderating online conferences. *IEEE Transactions on Professional Communications*, PC-29, 12-18.

Kerr, E. B. & Hiltz, S. R. (1982). *Computer-mediated communication systems status and evaluation*. New York, NY: Academic Press.

Kiesler, S. S., McGuire, T. W. (1984). Social psychological effects of computer-mediated communication. *American Psychologist*, 39, 1123-1134.

Mason, R. (1987). Computer conferencing: Its contribution to self-directed learning. *Proceedings of the Second Guelph Symposium on Computer conferencing*, 33-39.
Mason, R. (1988, September). *The use of computer-mediated communication for distance education at The Open University*. Working Paper. Milton Keynes, England: The Open University.

McCreary, E. K. & Van Duren, J. (1987). Educational applications of computer conferencing. *Canadian Journal for Educational Communication, 16* (2), 107-115.

McGuire, T. W., Kiesler, S., & Siegel, J. (1987). Group and computer-mediated discussion effects in risk decision making: *Journal of Personality and Social Psychology, 52*, 917-930.

Nunamaker, J. F., Applegate, L. M., & Konsynski, B. R. (1988). Computer-aided deliberation: Model management and group decision support. *Operations Research, 36*, 826-848.

Osborn, A. F. (1987). *Applied imagination*. York, NY: Scribners.

Payne, J. W. (1976). Task complexity and contingent processing in decision making: An information search and protocol analysis. *Organizational Behavior and Human Performance, 16*, 366-387.

Rice, R. E., & Case, D. (1983). Electronic message systems in the university: A description of use and utility. *Journal of Communication, 33*, 131-152.

Senn, J. A. (1987). *Information systems in management (3rd ed.)*. Belmont, CA: Wadsworth.

Shaw, M. E. (1981). *Group dynamics (3rd ed.)*. New York, NY: McGraw-Hill.

Shaw, M. L.G. (1988). An interactive knowledge-based system for group problem solving. *IEEE Trans. Systems, Man and Cybernetics, 18*, 610-617.

Siegel, J., Dubrovsky, V., Kiesler, S., & McGuire, T. W. (1986). Group processes in computer-mediated communication. *Organizational Behavior and Human Decision-Making Processes, 37*, 157-187.

Simon, H. A. (1960). *The new science of management decision*. New York, NY: Harper & Row.

Stefik, M., Foster, G., Bobrow, D. G., Kahn, K., Lanning, S., & Suchman, L. (1987). Beyond the chalkboard: Computer support for collaboration and problem solving in meetings. *Communications of the ACM, 30*, 32-47.

Stix, A. H. (1987). Computer conferences: What makes them difficult, and for what types of conferencing situations are they most suited? *Proceedings of the Second Guelph Symposium on Computer Conferencing, 105-121*.

Stodolsky, D. (1984, December). Self-management of criticism in dialog: Dynamic regulation through automatic mediation. *Proceedings of Communication and Contracts Between People in the Computerized Society*. Göteborg, Sweden: University of Göteborg.

Terry, G. R, & Franklin, S. G. (1982). *Principles of management*. Homewood, IL: Irwin.

Turoff, M. (1972). Delphi conferencing: Computer-based conferencing with anonymity. *Technological Forecasting and Social Change, 3*, 159-204.

Ujimoto, K. V., & James, D. A. (1987). The adoption of information technology in academic settings. *Proceedings of the Second Guelph Symposium on Computer Conferencing, 123-144*.
Van De Ven, A. H., & Delbecq, A. L. (1974). The effectiveness of nominal, Delphi, and interacting group decision making processes. *Academy of Management Journal*, 17, 605-621.

Van Duren, J. C. (1986). *The development and early applications of CoSy at The University of Guelph*. Masters Thesis, Guelph, ON: University of Guelph, Dept. of Rural Extension Studies.

Waggoner, M. D. (1987). Explicating expert opinion through a computer conferencing delphi. *Proceedings of the Second Guelph Symposium on Computer Conferencing*, 167.

Welsch, L.A. (1932). Using electronic mail as a teaching tool. *Communications of the ACM*, 25, 105-108.

## AUTHOR

Norman P. Archer is an associate professor in the Faculty of Business at McMaster University, 1280 Main Street, West, Hamilton, ON L8S 4M4. This research was supported by a grant from the Natural Sciences and Engineering Research Council of Canada.
AMTEC '89

Conference Proceedings
Edmonton, Alberta, Canada 18-21 June 1989

a conference hosted by the
Society for Instructional Technology -
Edmonton (SITE)

To order copies send $25.00 to:

Society for Instructional
Technology · Edmonton

c/o Dr. Ray Schmidt

Strathcona County

2001 Sherwood Drive

Sherwood Park, Alberta

T8A

Transitions
Transitions
Transitions
Transitions
# The State of Educational Technology: Responses to Mitchell

Editor's note: In the first article of last issue (See "The Future of Educational Technology is Past") P. David Mitchell argued that educational technology has failed to change the landscape of educational practice dramatically. Further, the current manifestation of educational technology, he claims, will never propel us beyond the present. Hence, the future of educational technology, as we know it, practice it and teach it, is past – more accurately, our field has no future unless we alter the underlying premises that guide inquiry into and the development of learning technologies.

In an effort to spark dialogue on the range of issues surrounding Mitchell's article, we invited publishable responses from the general readership, and simultaneously sent copies of the article to some of the individuals in the U.S. and Canada who have expressed interest in the topic through their own writing. This section features the eight responses that were received. We extend our sincere appreciation for the time and energy that is evident in them.

## Mitchell's Wake

Andrew Agostino

Mitchell's article, "The Future of Educational Technology Is Past", should more appropriately be titled, "Mitchell's Wake."

In a most enlightening eulogy at the death of educational technology as a field of endeavour, the author has stood over the corpse and stated that the cause of death was an "incurable, terminal illness." His reasons for such a pessimistic view are ostensibly irrefutable. As a profession, the field has become inordinately disparate. Many of its practitioners have been absorbed by other organizations whose purposes are more bureaucratic than educational. University courses in this area of concentration have become reductionistic, espousing new technologies for the sole purpose of exploring what the author calls, "lower-order problems" of curriculum design and activities which are only capable at arriving at solutions to micro-educational problems without ever addressing larger, societal and even global concerns. Unequivocally,
Mitchell's discourse cuts deep enough to jolt even the most placid proponents if it were not for the fact that, after reading the article, it is difficult to discern whether the author has indeed come to bury this corpse or, approximating Shakespeare's Mark Anthony, praise it.

Mitchell stands behind a paradoxical podium. Being one of the field's strongest advocates (at least in the past) and not having yet resigned his position as a professor of educational technology, burying the corpse might also imply leaping into its casket. At this realization, the author seemingly relaxes his positions and WAKEs to optimism (although slight) by outlining the possibility that the corpse can indeed be resuscitated. This feat, the author states, can only transpire through what he delineates as a radical transformation, a paradigm shift towards systemic thinking. However, this is more of an affirmation of control system theory, already deeply cemented in the field, than a reorientation of focus. Graduate programmes in educational technology have (in the last few years) attempted to embrace systemic thinking even when engaging in such activities as media production or instructional design.

As Mitchell very knowledgeably explicates the ins and outs of Cybernetics Revisited, he seems to be falling in love all over again with the corpse. His refurbished viewpoint is further heightened when he conceives of educational technology as a metasystem which (although not viable in itself) comprises many viable systems, namely its proponents and practitioners who can offer it consistency. Moreover, his endorsement that the likelihood exists for arriving at some underlying principle, some overall schema that will organize the field of educational technology furthers the belief that the corpse can indeed be revived. Finally, Mitchell pays great homage to educational technology as a field of study capable of solving educational perplexities of global proportions. "Opportunities for educational technology seem endless." How can one morn at such a wake?

In the end, Mitchell's article is more of an impatient call for coherence (and rightly so) in the field, rather than a post mortem analysis. If not, it can only be a case of cerebral necrophilia.

## AUTHOR

Andrew Agostino is a high school teacher with Jerome Le Royer School Board in Montreal and a graduate of the M. A. Program in Educational Technology at Concordia University.
# Fashioning the Future Educational Technology: An Invitation

Don Beckwith

## INTRODUCTION

It is the intent of this paper to establish the next stepping stone toward fashioning the future educational technology. An outline of the long-standing inherent, unifying theory of educational technology and its past, present and future manifestations will be shared, followed by a discussion of the implications of a future-based theory for educational technology practice. Highlighted here will be a suggested shift of empowerment within educational technology. Finally, a beginning profile of desired learner outcomes of the future educational technology will be presented as a springboard for discussion and reader involvement in the further fashioning of our future. In short, this will be a proactive rather than reactive piece.

## THE THEORY OF EDUCATIONAL TECHNOLOGY

If there is some future of educational technology that is dead, it is not ours. Our future has been alive and well in the minds and words of our visionaries for a long time. The vision of the future educational technology, in terms of goals, has remained constant over the years, for it is based upon the unwritten, implicit, unifying theory of educational technology.

Yes, there is a unifying theory of educational technology. There cannot be a believed-in, viable vision without a supportive theory. Every educational technologist who has embraced our field as the way to positive change, every educational technologist who has been frustrated with our slow progress toward the ideal, every educational technologist who has been committed to the improvement of learning has shared this vision and its theory.

Simply stated (I will be fleshing this out for a future article), the principles of the theory of the future educational technology are these:

a) individuals are capable of learning, learning how to learn, and learning how to control the learning process;

b) all individuals are capable of becoming motivated;

c) individuals are capable of moving from poor or non-performance to excellent performance;

d) successive generations of individuals will be able to perform at higher and higher levels of excellence.

e) an individual who has learned, learned how to learn, or learned
how to control the learning process has done so as a result of
interactions within and/or between self and environment;

f) communication of clearly envisioned desired performance
facilitates performance attainment and evaluation; and

g) environments can be structured, in a systemic fashion, to effect
successful learning and motivation.

These principles suggest a causal-deterministic theory, if the conditions
are structured appropriately, any desired learning is possible. Providing
explanations for what could be, our future-based theory has remained constant
throughout the evolution of educational technology. It has provided the
rationale and motivation for everything we have done and continue to do and
has fueled our attempts toward dream realization.

While the vision and theory have remained the same, the means during our
evolutionary stages have changed. During the audio-visual stage of educa-
tional technology, such as multi-media learning environments, teaching
machines and educational television were touted as the means to our visionary
ends. Researchers and developers were intent on finding the "best" medium.
In the current stage of educational technology the predominant means has
become that of a systematic instructional design approach, one that incorpo-
rates the audio-visual component as but one variable within the instructional
system. In keeping, researchers and developers now pursue multi-variate
approaches rather than comparative, trial and error approaches. Today's
futurist educational technologists envision yet another means – a systemic
approach to developing learning environments and studying the learning
process, ultimately effecting a means-ends entwining wherein the learner is
the educational technologist.

IMPLICATIONS OF A FUTURE-BASED THEORY

Two aspects of the theory of educational technology give it strength, and
the ability to embrace the future as we create it: a) our theory is primarily a
set of proactive principles, a belief system to be effected; and b) our theory has
the potential to empower any component of the educational technology system.
Appropriate attention to these aspects can actualize the future educational
technology and its desired outcomes.

Effecting the Belief System

Our future-based theory has major implications for how we should practice
within the future educational technology. Rather than accept others' percep-
tions of desired learner outcomes, we must establish desired learner outcomes,
we must concentrate on our desired ends rather than appropriate means for
existing ends and existing constraints. Rather than try to predict what the
future might bring so that we may be prepared, we must create the future; we
must adopt a proactive stance. Rather than monitor the systems of the status quo in order to correct them, we must create ideal systems. Rather than design learning environments to facilitate the passing on of what is "known," we must design environments that facilitate the questioning of what is "known" and the creation of what is not known. Rather than foster learner dependence on the educational system, we must effect learner independence. Rather than assume that the societal suprasystems must be changed before those systems of interest to us can be changed, we must act within the awareness that a viable, newly formulated system will change its suprasystem. Rather than offer a legacy of accepted principles and practice, we must facilitate learner creation of new principles and practice; we must insist that learners go beyond our skills without taking on our baggage. Rather than concentrate on means, we must operate from our desired ends. Rather than hope that learners will someday be able to do what we can do, we must make it possible for learners to do what we can only dream of doing; we must pass on our dreams and the skills necessary to bring those dreams to realization. Rather than study the status quo and then design solutions for it, we must structure environments that will create ideal learners in spite of the status quo.

In order to create the future educational technology we must remove ourselves from the confining constraints of all that has gone before and all that is now. The vision of the desired outcomes of the future educational technology can be clear enough and enough coveted to be all we need. We should take care not to minimize or taint that vision with a redefinition based on experience, present conditions or what we imagine the future might hold.

*Empowerment*

Regardless of the means employed our visionary theory has remained a theory of empowerment for those who embrace it. Within each successive stage of our development a different component of the educational technology system is empowered. Whereas yesterday's educational technology empowers the instructional design team, tomorrow's educational technology will empower the learner. For the future educational technology to flourish, each and every learner must become an educational technologist, capable of transforming any environment into a fertile learning environment for meaningful and purposeful self-growth and fulfillment.

## FROM IMPLICATIONS TO REALIZATION

Let us assume that a learner is empowered if able to think and perform in the manner of a master future educational technologist. Let us further assume that we can describe and measure such thinking and performing ability. Finally, let us assume that we can structure learning environments that would facilitate learning of such abilities. By keeping our sites clearly focused on the desired learner outcomes of the future educational technology we can begin to
describe the type of future educational technology needed. Then we can
determine how to go about creating such a future educational technology. (How
many times have you heard this sermon – for every field but our own?)

To define the desired outcomes of the future educational technology, i.e.,
the outcome of learner empowerment, we must be able and willing to show that
educational technology is capable of objectively communicating the highest
level of sophisticated, problem-solving learner performance. If we can clearly
envision the desired future learner, we can clearly describe that learner's
excellent performance. To illustrate, let me share with you my vision of the
ideal, empowered learner of the future (the desired outcome of the future
educational technology). For me, the empowered learner can:

a) create plausible, alternative hypotheses to any current theory,
explanation, principle;

b) perceive all things from outside the accepted societal view;

c) bear in mind that which needs to be understood;

d) test own and others' conceptions/preconceptions;

e) apply what has been borne in mind;

f) distinguish system from asystem;

g) create systems;

h) communicate the systemic quality of a system;

i) look within, as well as without, for answers/truth;

j) see/analyze parts and whole simultaneously;

k) design systemic wholes;

l) implement and control systemic wholes;

m) visualize ideals;

n) maintain constant dissatisfaction with the status quo;

o) create alternative pathways as needed;

p) create new rules/principles, as needed for each new systemic
creation;

q) continuously transform self to next higher level on spiral (self-altering/enhanced regulatory capacity);

r) act independently of others' conclusions;

s) maintain lofty ideals;

t) see all constraints as temporary, status quo baggage;

u) define self in future terms;

v) analyze and synthesize simultaneously;

w) arrive at and share unique truths;

x) look beyond the needs of self; and

y) operate from the future.

Just as it took a lot of guts to move from the audio-visual approach to the
systematic approach, it is going to take a lot of guts to move to the systemic
approach, not to mention to the learner empowerment approach. My guts are
on these pages. Your guts are needed. No one else is going to do it for us -not
our clients, not our superiors, not our employers, not the administrative
leaders in our field. All it will take is for a handful of us to see it so clearly, want
it so badly to make it happen.

AUTHOR

Don Beckwith is the Director of The Learning Center, City University of New
York Medical School, 138th Street & Convent Avenue, New York, NY
10031, U.S.A.
# Total Educational Technology (TET): Challenging Current Limits

1vor. K. Davies

In a cable from London just over a hundred years ago, Mark Twain remarked "reports of my death are greatly exaggerated." A similar thought comes to mind when reading David Mitchell's important perspective "The future of Educational Technology is Past." Mitchell (1989) argues that "educational technology has no future -because it is dead (though not yet buried)," although he concludes his discussion more optimistically wondering whether resurrection is possible. To make three valid points about the present status of educational technology (a systematic rather than systemic orientation, a narrow view of worthwhile educational problems, and need for a new research paradigm), he exaggerates his case.

## MEANING IS IN THE USE

Mitchell's use of a "purple passage" as a literary device to gain attention for his position, although overstated, does little to diminish the substance of the main body of his thesis. Educational technology has, indeed, failed to realize its potential, and it has not emerged as "the central humane discipline of the future" (Richmond, 1967, p. 106). Probably it can never attain this goal. As Kenneth Richmond (1970) argued unceasingly, instrumentation changes means and ends.

Instrumentation alters orientation, techniques, and the learning situation itself, often in unique and sometimes imperceptible ways. Each new invention or development extends the range of what can be achieved. It is a cumulative, yet dynamic process, that has shifted the onus for learning from teachers and educational technologists to learners in association with parents and employers. Also, it has extended the art and the range of what is necessary and possible. In this sense, educational technology has died. But it has died many times, and has always been reborn with enhanced potential for becoming a humane discipline.

Unfortunately, the cult of efficiency, with the growing subordination of education to inappropriate business goals, has often eroded the effectiveness of what educational technology can offer. Quality, which Mitchell fails to address, has too often been a low priority, at least in terms of identifying and then taking steps to meet or exceed our client's expectations. Even our client's identity has sometimes been obscure, and the idea of internal and external clients (customers) as well as end-users is novel to educational technologists.

To use a tag of Wittgenstein's (1958, p. 139), the meaning is in the use, and in educational technology, as Mitchell says, we have often failed to deliver our promise. In the formal literature of the profession, technologists define for each
other the nature of the field; in use they have conveyed a more limited meaning. Because of this failure, less has been expected. We have become complacent and less threatening to our colleagues in other areas, and we have been sucked into the main stream of the education machine.

In seeking to become more professional, we have lost our way. Mitchell is right when he refers to the three traps (compromised integrity, adherence to the status quo and solidification) which Beckwith (1988, p. 8) postulated that we needed to avoid, as symptoms of our present woe. Educational technology does not have the richness of meaning that it ought to have. It has acquired mechanistic, systematic, engineering nuances, that seem to deny our wider educational responsibility. There is an unfortunate void between our words and deeds, ideas and practice.

## EDUCATIONAL TECHNOLOGISTS
### PART OF THE SOLUTION

Educational technologists have a bias for action, and this bias has, at times, caused us to lose track of our vision. Mitchell argues that "Educational technology must be dedicated to the efficiency (sic) of education as a whole and not simply to specific operations. . . The field of educational technology-in its concern for optimal organization of education – must not be limited to time-honored structures. Nor should it perpetuate failures." Laying aside Mitchell's use of the word "efficiency," this view, not unreasonably, widens the scope of educational technology.

No longer is educational technology a synonym for instructional development (with its sometimes narrow orientation towards the design, development and evaluation of instructional materials). Educational technology takes on the twin imperatives of enabling nations to enhance their collective human potential, while helping people realize their own.

This, as Mitchell realizes full well, revolutionizes the scope of the educational problems that educational technology can be expected to resolve. The challenges and opportunities are endless, but the traditions are not yet there. The perspective has been limited. Perhaps the view of instructional development, with its concern for often only instructional materials, has become an albatross around our necks. The perspective is too narrow. It limits the scope that on the one hand demands a world view, and on the other a concern with the whole range of human potential and performance – including not only fitting people to a task and environment, but also fitting the task and environment to people.

But do educational technologists currently have the competencies to undertake two such challenges? In a recent *ad hoc* study (Davies, 1988) of educational technologists in two large American corporations, strengths and weaknesses were assessed by people who were responsible for them.

Educational technologists were perceived as being highly able, technically competent in their professional activities, and having good people skills. However, they were also perceived as lacking a business orientation,
ment skills and experience, and even an issues or problem orientation. They were criticized for being reactive, rather than proactive, and for not being team players. While they had a well developed educational technology network, their education and business networks were poorly developed. If these characteristics are in any way typical, the profession can take pride in the ability of the people it has attracted. Their training, however, needs to be focussed and enriched to overcome these criticisms.

Educational technologists need a balanced mix of competencies in order to perform their role effectively. This mix changes in content and balance as technology or know-how develops. The physical component of the role is declining as a result of the increasing use of information technology, freeing up more time for front end analysis. Also, the judgement component, always a key constituent of professional activity, is becoming increasingly important.

The distinctly humane skills of perception and intuition, together with the crucial element of timing, are also becoming increasingly important in the mix. Two further competencies stand out in the performance of the more successful educational technologists (Davies, 1988). These are the twin skills of influence (which depends upon the "clout" which is developed in the organization), and facilitation (which relates to the ability to get things moving, as well as to sustaining the process). Influence is a personal skill while facilitation is an inter-personal one.

## A TOTAL APPROACH TO EDUCATIONAL OPPORTUNITIES

Educational technology has been dominated by a systematic perspective (see Davies, 1984, p. 9), even when the words claimed to be otherwise. Yet, as Mitchell remarks, central to the idea of educational technology is a systemic approach. Such a perspective considers the total system, with its interacting systems and sub-systems, which work together to achieve the system's goals and objectives for its total environment. But it is more than Mitchell suggests. It is a total socio-technical system, each part of which depends upon the configuration of the other - social and technical systems jointly optimized.

Mitchell fails to point out, however, that it is the total system that is optimized, while sub-systems and components are satisfied, i.e., are designed to do well enough (see Ackoff, 1970, p. 5-9). Educational technologists in the past have often unwittingly optimized sub-systems and components, and so created a cancer that ultimately threatens the "health" of the total system. This is one reason that educational technology projects have sometimes failed to realize their potential. Problems were conceived too narrowly, and designs developed in isolation from their environment. It is an example of what Adler calls the fallacy of reductionism - "assigning a greater reality to the parts of an organized whole than to the whole itself" (Adler, 1986, p. xix).

A total systems perspective is essential, if the field is to deal effectively with the wide range of global problems that Mitchell envisages. But total
implies more than Mitchell suggests, it has three inter-related meanings. Total describes not only the effectiveness of the system, but also the steps that are taken to maintain the system once it has been designed and implemented, and the total participation of all stake holders (learners, teachers, educational technologists, administrators, organization, parents, shareholders, community, etc.).

Mitchell proposes control theory, as the new paradigm for behavioral research, in order to escape from the dilemma of two models one for the controller (the instructional system), and one for the controlled (the learners). Central to this cybernetic approach is the attention that is given to feedback in the total system. In "the absence of universal reinforcers in educational settings," Mitchell points out that a "person's behavior controls their perception in relation to their intentions." He argues that this implies that educational technology has two options. Educational technologists can either "implement schemes that limit individual differences" or promote schemes that promote optimal enhancement of individual potential.

This is an important rational, but it avoids the importance of the feedback obtained from breakdowns in the total system after implementation. Ideally, instructional systems are designed so that failure is unlikely. But this is rarely the case, and educational technologists are becoming increasingly aware of the importance of taking time to analyze human error down to root causes (Davies, 1981). Five options are available to educational technologists to help eliminate or prevent system failures. The first two, taking steps to maintain a well regulated instructional system and adhering to the designer's operating procedures, improve the operation of the instructional system. The other three, restoring deterioration in the instructional system, removing weaknesses in its design, and dealing with human error as a critical source of information, involve enhancing the reliability of the system.

## CONCLUSION: THE FUTURE OF EDUCATIONAL TECHNOLOGY

Whether educational technology will become the central humane discipline of the future (Richmond, 1967), and whether it will help create "health, ideal space, and peace" (Beckwith, 1988) are matters of conjecture. What is more important is that from a consideration of both the possible and the probable futures of educational technology, we can take steps to ensure the desirable ones. David Mitchell, in his perspective "The Future of Educational Technology is Past" describes one, but there are other futures for us to consider as we seek continually to renew our field. The future is now.

## REFERENCES

Ackoff, R. L. (1970). *A concept of corporate planning*. New York, NY: Interscience.
Adler, M. J. (1985). *Ten philosophical mistakes*. New York, NY: Macmillan.

Beckwith, D. (1988). The future of educational technology. *Canadian Journal of Educational Communication, 17*(1), 3-20.

Davies, I. K. (1981). Task analysis for reliable human performance. *Performance & Instruction, 20*(2), 8-10 & 31.

Davies, I. K. (1984). Instructional development: Themata, archetypes, paradigms & models. In R.K Bass & C.R. Dills (eds). *Instructional development*. Dubuque, IA: Kendall Hunt.

Davies, I. K. (1988). *Competencies of educational technologists in two large American corporations*. Unpublished paper. Bloomington, IN: School of Education, Indiana University.

Mitchell, P. D. (1989). The future of educational technology is past. *Canadian Journal of Educational Communication, 18*(1), 3-27.

Richmond, W. K. (1967). *The educational industry*. London: Methuen.

Richmond, W. K. (1970). *The concept of educational technology*. London: Weidenfeld & Nicholson.

Wittgenstein, L. (1958). *Philosophical investigations*. Oxford: Blackwell.

## AUTHOR

Ivor K Davies is Professor of Curriculum Instructional Systems Technology, Indiana University, Bloomington, Indiana 47405.
# The Upcoming of Learning Technology

Philippe Duchastel

Mitchell's provocative essay leads me to believe that we should really rejoice at the demise of educational technology rather than attempt to shore it up with a strong injection of control systems theory (the proffered solution to the current crisis). I believe ET (Educational Technology) is problematical, but in a rather different sense than the one advanced by Mitchell, and, as announced by my title, the solution to the problem lies, I believe, not in educational cybernetics but rather in educational psychology.

## *ET As Ineffectual*

ET is ineffectual only in as much as we expect it to be a panacea for today's and tomorrow's educational ills. While it certainly may be true that near illiteracy and innumeracy characterize too many of our kids upon high school graduation, whose demise should that announce? ET is but one player in an organic educational and societal system that juggles constraints and goals to improve its situation. Here is where a systems view needs to be applied!

Mitchell is an idealist, however, and espouses (most mistakenly, I feel) a radical ET paradigm which views transformation of education as its goal (the clearing up of the messes, as he puts it). The educational technologist is seen as a change agent, one which attempts to change people's minds regarding the way to teach and to organize learning. It must be recognized that radical ET is an elitist perspective, which fortunately (or unfortunately?) often shoots itself in the foot with its latest enthusiasm (remember ETV or CAI?), thereby calling into question the short-sightedness of its missionary zeal. Over the years, I have come to believe that if the solutions we educational technologists propose to our clients are not readily appropriated by them, then the lack is not primarily in our client's understanding and receptiveness, but rather in our own proposed solutions.

This seems to be what Mitchell is advocating, that we need to broaden our problem space and bring to bear system tools to truly tackle the problems. Unless I have misread his point of view, however, his solution remains tied to an advocacy position in which we attempt to bring about change in a generally unwilling and unresponsive system. Only now, our efforts should be less blinkered by our largely non-theoretical and routine application of instructional design. Mitchell's idealism, a laudable prospect on its own terms, keeps him unfortunately well settled in the radical ET camp.

## *The Cybernetic Solution*

Mitchell is saddened by the oft-encountered shallowness of ET and calls for
a larger perspective to bring some measure of perspecuity to the situation. He forgets, however, how valuable even routine ET can be in instructional design (when not involved in foot-shooting, of course). My own personal experience in large-scale training programmes has pointed that out to me whenever I have contrasted the quality of training materials developed through an ISD process with the quality of materials developed without such a process. Even if it is limiting, the instructional systems design process can be beneficially applied to many educational and training problems.

I do agree with Mitchell that many aspects of ET are very much less sophisticated than one would ideally like. He rightly suggests, for instance, that CAL is simplistically geared to a trivial process of discourse which fails to build on anything like an appropriate model of student understanding. Let's keep in mind, however, that we are dealing here with advanced technology (the design of intelligent tutoring systems) that is still largely in the arena of R&D and therefore still in the future in any practical terms.

Sophisticated solutions to our educational ills, Mitchell contends, will require a cybernetic perspective that refines our notion of regulatory control theory so that it properly includes psychological frameworks (our own and that of others) into its workings. Despite stating that ET has unfortunately been traditionally concerned more with instruction than with learning, Mitchell is essentially proposing a refinement of control through instruction, even if that control is meant to be more responsive to student needs.

Here, in my view, is where the crux of the ET problem lies. Control is the single big issue that ET needs to address in order to invigorate its theoretical programme and prepare itself for the advent (onslaught?) of interactive technologies in the school and in the home. The issue is the following: given that we can control the flow of instruction along optimal lines (that, after all, is the ambition of any individualized system of instruction), should we do so? That is, to what extent should we hand over to the learner control over the instructional flow, as opposed to trusting our own models of instruction, as embedded in our instructional artifacts? The issue is a complex one and it will require our best efforts to make sense of its various aspects.

## *The Rise of Learning Technology*

The reason I raise this issue is that I believe Mitchell is misguided in proposing the need for a cybernetic regulatory model to guide ET. What is needed instead is a focus on how we can better involve the learner in education both formal and informal. We need to focus not on the design of systems, but rather on the design of learning environments (an orientation broached, but hardly adequately dealt with, in Duchastel, 1988).

Thus, there is little cause for concern over the demise of ET as long as we foresee its successor in what is becoming known as Learning Technology This new technology promises to focus to a much greater extent than its predecessor on learning processes and on how they tie into instructional settings.

In his essay, Mitchell has rightly pointed out that ET is headed towards
trouble. His proposed solution is a strong one, and one which is therefore contentious. I find that exciting, for I think that educational technologists are on the verge of redefining their field. That is cause for rejoicing, and for hope that our collective efforts may have some measure of impact in improving education. Let's bury ET and give birth to LT.

REFERENCE

Duchastel, P. (1988). Designing intelligent learning environments. In H. Mathias, R. Budgett, & N. Rushby (eds.) *Designing new systems and Technologies for learning*, (p. 93-98). London: Kogan Page.

AUTHOR

Philippe Duchastel is involved in the application of artificial intelligence to instruction and learning. He can be reached at the Learning Technology Institute, 2996 S. Moline Place, Aurora, CO 80014, USA.
# The Death of Educational Technology Has Been Greatly Exaggerated

Michael J. Hannafin

It is tempting to endorse the compelling arguments made by David Mitchell. Indeed, both Mitchell's and Beckwith's arguments are high in face validity and consistent with the views of an increasing number of academics. We rarely tackle the truly difficult educational problems with technology, preferring instead to merely "re-host" methods already developed. We sustain often fundamentally flawed educational and training systems. Collectively, we have evolved a dubious intellectual identity, relying more often on the advances of our so-called "root disciplines" (Clark, 1989) than the internal directiveness that characterizes related disciplines. In many instances, we have compromised our integrity by acceding to opportunism -financial, personal, and political.

Mitchell's analysis is certainly provocative, but is it also reasonable? His exasperation with afield so slow to mobilize its intellectual resources in needed directions yet so quick to respond to market-driven demands is understandable, but has he attacked the root or the symptoms? Consider the following issues, which represent only a sample of the relevant questions, clarifications, and issues which must also be considered. While it is impossible to provide the kind of depth and argumentation in a brief response, perhaps I can provide a somewhat different perspective to the topics addressed by Mitchell.

Which educational technology field has died? We are not a unitary field, but a meta-discipline of sorts. We are found across academic disciplines, in public and private educational settings, in R&D institutes, and across ages, grades and levels from preschool through adult. All subsets cannot be tarred with the same brush. Have all variants of the "educational technology" field failed equally? Should we conclude that educational technology has failed in areas such as flight and medical training? Or are we particularly distressed over the inability to address specific important problems, most notably the stagnation of public education? The problem is not simple overgeneralization, but the apparent disregard of notable successes in the face of frustration and failure.

Are the circumstances described by Mitchell really educational technology problems? Or has the field been tied to systemic circumstances which all education and society faces? Educational technologies offer both potentials and perils, but they are not, and have never been, panaceas to all that ails either education or society. We welcome all that educational technology can provide to address societal ills, but we cannot reasonably assess the potential or performance of educational technology by its lack of impact on problems for which it was neither intended nor implemented.
Perhaps the presumed context for educational technology, and the associated expectations for impact, are simply too grand. This is not toundermine the larger perspective advanced by Mitchell, but to recognize that it is precisely that: a perspective on the global and interactive nature of the factors that influence willingness and receptiveness to engage education. The socio-educational inequities described by Mitchell, ranging from rampant unemployment, to domestic illiteracy, to malnutrition and disease, and so on are parts of the context for understanding education. Can educational technology be rightly held accountable for either singularly contributing to this status or jeopardizing solutions? While I welcome increased attention of our technological resources toward the broad-based regional and global problems described by Mitchell, I cannot accept the condemnation of educational technology based upon the continued presence of world problems. The shortsightedness and lack of social impact chronicled by Mitchell is a fair assessment of some segments, but they are neither the focus nor responsibility of educational technology applications in others. To presume that, as a matter of course, educational technology must provide remedies to ills as widespread and systemic as those described is to condemn attempts preemptively.

Has educational technology failed? Failures are human, not technological. The failures are chronic shortsightedness, entrenchment in the “status quo” (Beckwith, 1988), the absence of effective champions of educational innovation, and the inability to initiate the systemic changes needed to ensure success. People are not technologically-mediated; technological success is people-mediated. When successful, we should applaud the vision and creativity of those responsible; when not successful, we should understand where culpability lies and act accordingly.

Does the field need to refocus? Yes, I think we do. Yet, while I share the commitment to refocus, I am alarmed by the potential implications. Is there a "new best way?" Mitchell wisely avoids the temptation to prescribe specifically what curriculum refocus should be implemented. Should all graduate programs and faculty unify efforts around a new enlightened philosophy, curriculum, or world view? In general, I believe that unification is neither needed nor possible. It is at best an intellectual argument of the 'What if...?' variety, and at worst an assault on the value of the individual perspectives evolved by intelligent people throughout the world. If, as I believe, we garner a measure of strength from the diversity of our views, the unification of focus would weaken the breadth of our foundation and limit our capacity to advance the next "new best way." The inherent diversity in training, emphases, and program views will continue to hamper the development of educational technology as a discipline, but the liabilities of adherence to a unitary view of the field more than outweigh the potential advantages.

The title of this paper is a loose translation of the often quoted statement, "The rumor of my death has been greatly exaggerated". This statement, attributed to among others Mark Twain, W. C. Fields, and Will Rogers, reflects measured surprise in that an individual so closely affected could have been
unaware of his own demise. Yet, we are surprised not so much that our epithets are written but that they cause us to verify vital signs that we know exist. We must identify our life signs, describe them sufficiently to convince others not to inter the corpse, and rally others to ensure that such mistakes will never again be repeated. We are indebted to F? David Mitchell not so much for his conclusions, but for his analysis. He has caused us to agree comfortably, become righteously indignant about others, aroused elements of defensiveness and outrage, and motivated others to comment, attempt to refute, and otherwise elaborate on several important ideas. In the final analysis, perhaps this is why Mitchell's perspective was articulated: to challenge and not merely to instruct.

## REFERENCES

Beckwith, D. (1988). The future of educational technology. *Canadian Journal of Educational Communication, 17*(1), 3-20

Clark, R. (1989). Current progress and future directions for research in instructional technology. *Educational Technology Research and Development,*

## AUTHOR

Michael J. Hannafin is Director of the Center for Instructional Development ans Services at Florida State University, Tallahassee, FL 32306.
# There Are No Ends, Only Means

Steven T. Kerr

In this ambitious article, Mitchell asks us to rethink in a fundamental way a number of assumptions about what educational technology is, and what it is good for. The principal suggestion is that systems analysis be rejuvenated as a conceptual model for further work in the field, with the addition of control system theory as a way of recognizing that interactions in social systems are rarely one-way. Adding to our armamentaria the notion of mutually determining control systems (instructor on learner and learner on instructor), Mitchell maintains, can provide a new, more useful conceptual framework for educational technologists.

Just as important as an improved model for work in our field, Mitchell believes, is the need to move away from the insignificant to the profound. We must abandon our present focus on tiny issues which have little overall impact on the educational well-being of learners (and which therefore exist comfortably within the current system of education), and work toward a deeper and more significant commitment to "real educational problems. . . illiteracy, numeracy, intolerance or lack of caring" (p. 23).

In Mitchell's analysis, the improvement in systems thinking to be achieved by the addition of control theory is the means to achieve the end of an enhanced ability to deal with meaningful problems. I agree on the ends; I disagree on the means. And, as the title suggests, means are what most of us deal with most of the time.

*The future of systems theory is past*. In the early 1970s educational technologists (like many others) became enamored of systems theory and systems analysis. This new approach, synthesized from such diverse fields as mathematical modeling, economics, and military operations research, seemed to offer a way around the intractable complexity of social problems. By seeing things whole, systems theorists argued, we might cope with the difficulties encountered in trying to solve a myriad of small, interrelated problems simultaneously. Educational technologists, searching desperately (and apologetically) for a way out of the blind created by years of NSD results from classical experimental research, saw "the systems approach" as an answer.

Unfortunately, we have kept our faith in systems theory while others all around us have been losing theirs. The critiques have been various, and have included objections to the philosophical and mathematical assumptions underlying the stance of the systems proponents (Berlinski, 1976; Lilienfield, 1978), the impossibility of predicting significant policy shifts that ramify across systems (Ayres, and the conceptual adequacy of the model of reciprocal determinism that underlies much of systems theory in general, and
control theory in particular (Phillips & Orton, 1983). Even systems theorists themselves have begun to discuss these problems openly, and have suggested there may not in fact be a single systems approach so much as a variety of such approaches (Carvajal, 1983; Tomlinson & Kiss, 1984).

The current interest among scientists in "fuzzy sets," "loosely coupled systems," and "chaos theory" does suggest a continuing attempt to understand how complex phenomena in the real world are related one to another. But these efforts also underline how different the task is when one confronts problems in a physical, rather than social, realm.

*System theory and educational reality.* Are the differences really so pro-
found? It is significant that education has witnessed a corresponding decline
over the past several years in the popularity of approaches predicated on the
discoverability of large-scale panaceas to problems of instruction. Perhaps the
best indication of this shift is seen in the new demand that more qualitative and
ethnographic approaches be used to study the peculiarities of instruction in
different settings. While the proponents of such methods certainly do not rule
out (and many would find desirable) trying to understand the interconnected-
ness of instructional approaches and the environment in which these take
place, most would probably say that it is far too early to suppose we can
accurately identify all the factors involved, much less describe how they effect
one another.

The problem of trying to apply systems theory to research and practice in education is that, unlike physical systems where laws may be assumed to underlie observed events, the variables affecting instruction are presently less amenable to a simple physical description. Conceptions of learning, definitions of curricular content and structure, models of instructional methods, approaches to monitoring and evaluating educational results, ways of delivering and administering educational services – all these are subject to multiple definition based only partly on purely scientific phenomena. Even the purposes of the educational system itself are subject to a greater share of political and social influence than educational researchers or educational technologists are often prepared to admit (cf: the “equity vs. excellence” debates that flared in the United States after the first round of educational reform reports in the early 1980s).

These educational realities appear to me to make several of Mitchell's assumptions quite dubious: that educational technology should rightly be devoted to a search for educational efficiency and "optimal organization" (What of those who see its purpose lying in other spheres entirely-the aesthetic, for example, or the enhancement of students' abilities to explore alternatives?); that the purpose of education is to "provide access to stored human experience" (What of the socializing functions of schools that many parents put on a par with acquisition of information? What of the descriptions of economists that stress the role of schools in controlling access to the labor market?); that educational technologists can easily find ways to deal with the "real problems" of "intolerance or lack of caring" (What of the difficulties of defining
ance for what?” and “lack of caring for *whom?*,” in a context where increasing numbers of parents are removing their children from the educational system because they disagree with the kinds of answers schools have often provided to these questions).

*The search for appropriate means.* I need to reiterate: I agree with the ends Mitchell identifies as being important for educators to work towards, but I disagree seriously about the means to get there from here. I doubt that we are as yet anywhere near the point where we can reasonably talk about formulat- ing what we know about education or educational technology in terms of a general “systems approach” to the important educational problems of our time. Educational technologists concerned to do something significant about those problems would be well advised to seek to deal in more complex ways with the meanings their craft has for teachers and students, and with the political and social diversity of the educational system. Technologists, in short, need to become educational practitioners, working in classrooms with teachers, as well as activists versed in the political and social context of the educational system. Working from within in this way will have more significance over the long term than seeking to reform from without via a broadly conceived systems approach to educational technology and instruction.

## REFERENCES

Ayres, R. U. (1984). *Limits and possibilities of large-scale long-range societal models*. *Technological Forecasting and Social Change*, *25*, 297-308.

Berlinski, D. (1976). *On systems analysis: An essay concerning some limitations of some mathematical methods in the social, political, and biological sciences*. Cambridge, MA: MIT Press.

Carvajal, P. (1983). *Systemic-netfields: The systems' paradigm crisis. Part I*. *Human Relations*, *36*, 227-246.

Lilienfeld, R. (1978). *The rise of systems theory: An ideological analysis*. New York, NY: Wiley-Interscience.

Phillips, D. C. & Orton, R. (1983). The new causal principle of cognitive learning theory: Perspectives on Bandura's "reciprocal determinism". *Psychological Review*, *90*(2), 158-165.

Tomlinson, R., & Kiss, I. (Eds.) (1984). *Rethinking the process of operational research and systems analysis*. Oxford: Pergamon.

## AUTHOR

Stephen T. Kerr is a Professor and Head of Curriculum and Instruction at the University of Washington, Seattle, WA 98195.
# The Future of Educational Technology is Past – A Reaction

Gerald M. Torkelson

I agree with Mitchell's analysis. It addresses a fundamental problem of most educational technologists who tend to focus upon the sophisticated dissection and creation of instructional processes and instrumentations, more than upon the actual accomplishments of learners in achieving personal/ societal goals and in acquiring enduring skills and knowledge. The consequence of this emphasis is that the partnership of the learner in identifying personal values, motivations, problems has often become obscured and lost. Frequently the learner's task has been to achieve short-sighted, immediate, transitory goals, rather than those which call for a matching of instructional techniques and materials with complicated long-term learner idiosyncracies. And another problem is that sometimes there is a tendency to judge technologists' competence more on the basis of models and complicated instructional systems than on the effects of those systems in producing learner satisfactions and intellectual, social, emotional growth.

My bias suggests that we must be, fundamentally and foremost, communication specialists whose goals, in general, are to understand and support interaction processes among learners and instructional systems with the hope of establishing learner "self-hood" and residual behaviors which support inquiry, introspection, adjustment, originality, psychological/philosophical balance, excitement in discovery, self-esteem, and success, (to name a few characteristics). We have developed sophisticated and useful paradigms for identifying desired educational objectives, learner behaviors, instructional instrumentations, and evaluation systems. We have not included in our systems, however, adequate provisions for learners to engage in extensive dialogue with that which is to be learned in order to meet their idiosyncratic drives to achieve in the frame of their immediate referents, intellectual, social and cultural heritage.

There is no doubt that creating an educational system which supports self-realization and a self commitment to broader societal goals will be very difficult to achieve. And it will also be difficult to achieve such ends through control system theory where the self-correcting feedback components inherent in all aspects of the system are critical to know and to control. I agree with Mitchell's criticism that our primary emphasis on one side of the equation (essentially the presentational system) is reaching a point of diminishing returns in trying to maximize learner and system potential.

To create an interactive, self-adjusting educational setting will demand a revised educational structure. It must permit personal attention to learner differences and dialogue techniques which support continuous adjustments in
both instruction and learner involvement, not only in mutual goal setting but also in identifying where instructional processes and instrumentation need complementary alteration. And it must go beyond systems of the past where individualization was characterized more by variations in time allotments and space than in significant divergence from stated goals and achievements which, upon careful analysis, often were of short-term value.

One obstacle to reform is an educational tradition which finances group instruction and maintains teacher-pupil ratios on an economic model, hardly conducive to learner/tutor interaction. Such a transformation of schooling will also demand new insights and roles for teachers/tutors as they become agents in the cybernetic system including a determination of how and why teachers control as they do and the implications of such control for other relevant components. Another difficult area will be to convince the general public to support such a system. On the other hand, it could be that many instructional tasks may be assumed through technological innovations, such as providing access to large bodies of information through instantaneous retrieval systems. Thus, human tutors will be free to interact more regularly with learners in continuous adjustment, the uniqueness of two intermeshed minds.

As to the training of educational technologists to the levels of sophistication demanded by cybernetic systems, graduate students and faculty could begin the process of change by recording and analyzing their own referent points and personalized goals as they become immersed in learning cybernetic interchange. This would require, also, the study of control systems in institutions which impinge on substantive educational achievement, from local to global.

I agree with Mitchell's implication that we must get away from expecting panaceas through our sophisticated one-way control systems. By building on what has already been accomplished in specifying educational goals and learner behaviors, it may be possible to avoid the conclusion that "the future of educational technology is already past" by focusing on those areas where societal systems and subsystems must collaborate with instructional designers in creating the utopian interactive, self adjusting system. To do less, even in a small way, is apt to result in Mitchell's prediction of the demise of educational technology, as we now know and practice it.

Instead of limiting learner alternatives, typical of much "engineered" instruction, the goal will be to open learners to a broad spectrum of choices. But even with more sophisticated technologies available, the ever present dilemma in "engineered" solutions will be to determine which alternatives are predictably those which may be required for a heterogenous collection of learners. Perhaps the magnitude of meeting the needs of learners in a dialoging, self-adjusting, tutorial system will make a comprehensive cybernetic system almost impossible to achieve. Considering the magnitude, also, of creating a control system sensitive to simultaneous feedback both within the instructional source and within the learner, the actual reality for a time may be a form of triage, treat only those learners most seriously ill educationally, depending
on other learners, better equipped, to utilize their own ingenuity in achieving personal goals through traditional avenues of instruction. Research in "programmed" instruction, for example, has shown that some learners progressed more rapidly by being left to their own devices in utilizing prose materials in the usual paragraph form, rather than being restricted to the slower, more plodding pace of programmed bits.

Mitchell should be commended for his analysis. I hope educational technologists will take his sobering judgements seriously.

## AUTHOR

Gerald M. Torkelson is Professor Emeritus at the University of Washington, Seattle, WA.
# Revitalizing Educational Technology:
A Response to Mitchell

William D. Winn

I greatly enjoyed David Mitchell's recent article on the state of Educational Technology (Mitchell, 1989). It raised a number of problems that we need to confront and proposed a solution that is appealing and potentially viable. Of the many telling points in the article, two stand out for me. The first is that technology is systematic in the way it goes about solving problems; yet the problems that technology is set to solve in Education are not amenable to systematic treatment because the cause-and-effect model that underlies our ways of making decisions are terribly unreliable when it comes to human activity. This unpredictability of human behavior is currently of considerable interest to a number of writers who question the validity of both our traditional methods of design and of research (Cziko, 1989; Streibel, 1989; Winn, 1989). The second important point is that Educational Technology lacks an organizing principle. If by this is meant a theoretical foundation, I thoroughly agree, in spite of AECT's (1977) claims to the contrary. The majority of people in our field have completely misunderstood system theory, interpreting a powerful conceptual tool as a series of mechanistic techniques. Our graduate programs have failed to provide students entering the field with anything like a conceptual basis for what they are being taught to do (Stewart, 1985). And our professional associations have failed miserably in the exercise of leadership in identifying and developing the theoretical framework, or organizing principle, the field so badly needs.

Although I therefore agree with a lot of what Mitchell has to say, there are two matters that I believe deserve some further thought. The first stems from the fact that I see Mitchell's criticism aimed more at scholars of Educational Technology than at those who practice it. There are a great many instructional designers at work in a variety of settings who are "doing" Educational Technology, and doing it well. I am thinking, for example, of those who develop computer-based training for the military or for industry. Of course, neither these people's conception of Educational Technology, nor of Education, nor therefore their assumptions about the means and ends of instructional design are the same as those Mitchell believes we should hold. Yet these folk show us that there are circumstances where it is possible, maybe even necessary, to ignore the constructive nature of learning, to concentrate upon improving instruction rather than facilitating learning, and even to infringe on students' freedom to choose, in order to get the job done. Such assumptions, and the instructional techniques that derive from them are anathema to educators and inappropriate for Education. Yet in the training setting a lot of what Mitchell implies is the misuse of technology is in fact working well. I therefore question
the conclusion that Educational Technology is dead. There are places where, in a different guise from a scholar's conception of what it should be, it is very much alive.

The second matter has to do with the manner in which Educational Technology might be revitalized. Mitchell's case for making cybernetics the organizing principle for the field is an appealing one, if for no other reason than it presents system theory, unattenuated, in the manner in which it was originally conceived. As such, it enables the educational technologist to look beyond the factors that, say, psychologists believe affect learning to a whole host of other influences about which we know very little and over which we have little control. However, the prospect of actually doing what Mitchell suggests is a daunting one, which I sense he realizes.

There are two alternatives that I believe are also worthy of pursuit. They are both more narrowly focused than Mitchell's proposal and may, in a sense, be counter to the purpose of Educational Technology as he sees it. However, both relate specifically to what it is educational technologists do and, more important, both would tie the field to a sound theoretical basis. (Both also have a psychological flavor, which reflects my biases.)

The first has to do with our idea of what good pedagogy is. It has always puzzled me that educational technologists set teaching by humans as a standard for judging the success of what they do. Thus, we find attempts to develop tutoring systems that are "intelligent", CAI software that interacts with students in a manner that attempts to imitate human discourse, distance education systems whose aim is to bring to students at remote sites instruction that is as good as what they would have received from a teacher in a classroom, and so on. Indeed, Mitchell himself sets up tutorial conversations as a standard which intelligent CAI might one day come close to attaining. Yet we have not asked whether what human teachers do is the best possible pedagogy. This standard has quite simply evolved as part of the traditions of practice of teachers. It has grown up unchallenged, and for all we know may not be the best way to help people learn. So why should we strive to develop instruction for delivery by non-human systems that attempts to emulate human teaching?

I propose a thought experiment in which we imagine that current pedagogy has evolved not within the traditions of practice of teachers but within the traditions of technology itself. In other words, if computers had been used in instruction rather than teachers, what would pedagogy look like today? It then becomes the task of the educational technologist to discover, or perhaps invent, the pedagogy that is most appropriate to instruction using non-human teachers (computers) and to use that as the standard against which we judge our success. Many will find such a proposal horrifying. But it has at least two things in its favor. The first is that it can allow the computer to become a true "tool for thought", as Salomon (1988) has suggested. Second it gives Educational Technology an independence from current practices and liberates it from the inertia of educational institutions that Heinich (1984) has seen as hampering our initiatives in public education.
My second alternative is to base our work on those aspects of human
behavior that are predictable. Here I am referring to mechanisms that vary
little from one person to another and that are rarely if at all influenced by
volition (that are, if you will, "cognitively impenetrable" (Pylyshyn, 1984). An
example would be preattentive perception. There is convincing evidence (Marr,
1982) that a great deal of organization occurs in the early stages of perception
before conscious effort is expended on interpreting what is seen. On the
reasonable assumption that this organization constrains attentive cognition,
we can see that an understanding of these mechanisms is valuable to whose
who design images for display by computer or by other devices. In terms of
scholarly activity in our field, this represents the complete antithesis of what
Mitchell has proposed -it is microscopic in the extreme, dealing with a small
number of processes, involving only the presentation of information and none
of the other activities that are necessary for learning. Yet the study of
preattentive perception, and the development of design principles from that
research, would provide a reasonably valid theoretical base on which to build
at least a small part of what we do.

One healthy sign for our field is that just about everyone working in it has
their opinion about what should be done. People are thinking and talking.
Mitchell has very eloquently argued a case for making cybernetics a theoretical
point of reference. I have added two more modest suggestions involving
pedagogy and perceptual psychology. I imagine that others will also share their
ideas. This kind of dialogue indicates to me that the field is neither dead nor
moribund. Being an optimist, I suspect that what Mitchell has seen as death
is in fact a mid-life crisis. Such crises are certainly not fatal; they are simply
part of growing up. Educational Technology has a lot of growing up to do, but
that does not require a miracle.

REFERENCES

Association for Educational Communication and Technology (1977). *Educa-
tional technology: Definition and glossary of terms*. Washington DC:
Association for Educational Communication and Technology.

Cziko, G. A. (1989). Unpredictability and indeterminism in human behavior: Arguments and implications for educational research. *Educational Researcher, 18*(3), 17-25.

Heinich, R. (1984). The proper study of educational technology *Educational Communication and Technology Journal*, *32*, 67-87.

Marr, D. (1982). *Vision*. New York, NY: Freeman.

Mitchell, P. D. (1989). The future of educational technology is past. *Canadian Journal of Educational Communication, 18*(1), 3-27.

Pylyshyn, Z. (1984). *Computation and cognition: Toward a foundation for cognitive science*. Cambridge, MA: Massachusetts Institute Press.
Salomon, G. (1988). AI in reverse: Computer tools that turn cognitive. *Journal of Educational Computing Research, 4*, 123-139.

Stewart, A. (1985). Appropriate educational technology: Does "appropriate-ness" have implications for the theoretical framework of educational technology? *Educational Communication and Technology Journal*, 65.

Streibel, M. (1989, March). *Instructional design and situated learning: Is a marriage possible?* Paper presented at the annual meeting of the American Educational Research Association, San Francisco.

Winn, W. D. (1989, March). *Rethinking cognitive approaches to instructional design*. Paper presented at the annual meeting of the American Educational Research Association, San Francisco.

## AUTHOR

William D. Winn is a Professor in Educational Communications at the University of Washington, 412 Miller Hall, Seattle, WA 98195.
Book Reviews

*Decoding Discrimination: A Student-based Approach* by R. Simon, John Brown, Enid Lee and Jon Young, London, ON: Althouse Press, 1988.

Reviewed by Denis Hlynka

*Decoding Discrimination* is an important book for Canadian educators and for Canadian educational technologists. It consists of detailed study guides to four Canadian films. The intent is that a high school teacher will spend approximately five lessons on each film. The four films are *Ravinder*, dealing with a Canadian Sikh; *Reflections: People of Ontario*, dealing with immigrant experiences of four ethnic groups (Black, German, Greek and Korean); *Enemy Alien*, about the internment of the Japanese in Canada during World War II; and *Maria*, about an Italian immigrant. All four films are united by the common theme of discrimination.

The book is built on the simple but powerful concept that teachers need instructional support material if they are to use film materials effectively in classroom teaching. I think the authors are right, and I think the guides they have produced are excellent. While the focus is on four specific films, the authorshave also produced a model which should guide teachers to using other films in a more effective and more efficient way Indeed they "encourage you to adapt and apply the ideas in this unit to other films that may be more relevant and accessible to you in your particular situation" (p. 3).

The basic model of film analysis is not stated explicitly, but needs to be
pulled out of the text by the reader. Nevertheless, this is probably the most
significant contribution the text makes, more so even than the explicit content
promised in the title "decoding discrimination." Each of the four films is
discussed under the following topics: description, general remarks, overview
of lesson plans, detailed notes for each lesson, and assignments for each lesson.
The authors suggest that each film needs several lessons, ranging from three
sessions for Ravinder, to six lessons for Reflections. In all cases the first lesson
introduces the film, then instructs the teacher to allow "an uninterrupted
viewing" of the entire film. Later lessons show and discuss specific segments
of the film. The assignments are extensive, approximately six per film.

Given the above overview of the text, it is necessary to focus critically on three issues. First, living as we do in an information society, it is almost essential that teachers do what this book suggests, that is, teach with film. It needs to be noted, however, that the term "film" is a problematic one which ultimately needs to be phased out and replaced with film/video (much like the term he/she). Most teachers now use video, and one assumes that the films discussed in this text must also be available in video format. This not merely a technical point, however. One does use video differently. At the very least, the nature of video is much more amenable to the kinds of analysis the authors desire. Indeed, given a video format, even more in-depth analyses as to how the medium works become possible. Only video permits easy access to any scene almost instantly. In addition, instant replay of any scene or even frame is a significant attribute of the video format.

Secondly, a good teacher can do wonderful things with any stimulus material, films included. Unfortunately, a poor teacher will simply miss the point and end up abusing and/or misusing the system. Thus the most obvious problem with which users of this book will be confronted is that the authors ask questions but supply no answers. This is fine if the teacher knows the subject, and is sensitive to the particular methodology being proposed. But these days, budget-stretching solutions no longer guarantee that every teacher can be an expert in all areas. The teacher needs help, and this text may or may not give the requisite information needed.

For example, how long will a teacher expect students to work on the question which says, "Do your own research on the garment industry in Canada. . ." (p. 106). It is simple to say that the teacher will decide. But more likely the teacher will not decide anything, merely assign the question, and let the student figure out what the author (or the teacher) is after. And a simple innocuous question suddenly becomes a weekend nightmare of impossible homework!

Or take a question like "How do we come to accept that a woman's place is in the home?" (p. 106) and "What changes do men have to make in order to expand the possibilities of a woman's place. . ?" Are these questions themselves inadvertently sexist? Is the first one asking that we do accept a woman's place? And why do men only have to make changes? Don't we all? Again, it is easy to say that we know what the authors have in mind. It is not so easy to communicate that clearly.

Thirdly, yet another set of activities proposed in this book is equally laudable in purpose but problematic in application. Throughout the units, students are expected to evaluate the cinematic techniques used in the films. On page two, students are told 'When viewing films. . . you will see that through such techniques as the choice of narration, dialogue, casting of actors, editing of scenes, and camera angles used to photograph the story, each maker has created a particular version of the nature and origins of social inequality.' The point is an important one. But later, when question 3.8 (page
SO) asks students to conduct a “film analysis,” no content information on cinematic technique is provided. It is a common error for textbook authors to treat certain topics as being intuitive. Very simple, one cannot intelligently react on an intuitive level alone to issues such as television intertextuality, gendered television, television modes of address or the semiotics of television. A content lesson on television/film literacy is clearly needed within this text, but is missing. It is not acceptable to shoulder the teacher with the task of becoming an expert in television analysis. It is not enough to request students to examine, as on page 81, the contributions of a film score. It is not that simple.

As a parallel example, I recall studying a complex philosophic technique called deconstruction, then some time later reading a junior high school English curriculum which stated naively, “students will deconstruct a text…” A second example and perhaps the most common such reductive statement is by those teachers and librarians who use the work “research” as something every Grade 2 student does. I also recall my ten year old son coming home to tell me about his “thesis” on the lemming! Such a use only tends to simplify and trivialize the process of “research.”

At this point let us return to the title of the book. The authors are trying to teach about discrimination. The subtitle suggests that the approach is “student based;” that this is “anti-racist education,” and that it is “using film.” Each of these terms seems problematic. “Student-based” can mean assigning homework for which the teacher must work out answers. “Anti-racist education” seems to be both a negative and narrow term. And “film,” as has already been noted, needs to be extended to include video.

The above comments notwithstanding, it needs to be re-iterated that “I like the book.” The model of using media to explore a significant issue within the classroom is laudable, perhaps even essential, as we enter the last decade of the twentieth century. Students need to know how to work with the simulation provided by media which goes under the name of “information.” The book, used with care, can be an important first step.

REVIEWER

Denis Hlynka is a Professor in the Faculty of Education at the University of Manitoba.
L'éducation pour la santé par Alain Rochon, Montréal, PQ: Agence d'Arc Inc, 1988.

Reviewed by Cécile Michaud

L'éducation sanitaire est perçue comme un moyen de modifier collective-ment les comportements de santé tout en respectant les choix individuels; c'est donc une voie de l'avenir (Palsley, 1981) et elle intéresse plusieurs disci-plines. Ce récent livre d'Alain Rochon nous vient de la médecine (éducation sanitaire) et s'adresse "aux intervenants, aux étudiants et aux dirigeants (1)" de la sante communautaire ou de l'éducation à la santé tout aussi bien qu'à "tout organisme bénévole ou communautaire, à tout formateur, à toute personne s'intéressant à l'apprentissage'.'

Il s'agit d'un document qui rassemble différents principes et modèles théoriques, plusieurs exemples et un guide "f.a.c.i.1.e." Cet acronyme suggère six étapes pour la réalisation d'un projet d'éducation à la santé: façonner le scénario du comportement principal, analyser les écrits, consulter les gens, identifier ce qui sera réalisé, lancer le projet, évaluer pour s'ajuster. Le livre se révèle une source intéressante de renseignements et d'exemples; les exer-cices aident le lecteur à vérifier ses acquis et des lectures sont suggérées pour approfondir les thèmes.

Le technologue de l'éducation pourrait cependant ressentir un certain malaise à la lecture du livre. Ce malaise ne se situerait pas tant au niveau du contenu du document (qui est pertinent, complet, exact et varié), mais bien au niveau de sa structure. Habitué à l'ingénierie éducative (Dick et Carey, Romizowski), le technologue recommanderait une définition préalable du problème éducatif à résoudre et de la population à cibler plutôt que le faconnement d'un scénario et l'élaboration de la chaîne comportementale tel que proposé par Rochon à la première étape. De plus, le technologue suggére-rait un processus d'évaluation formative (Baggaley, Caron, LeRose, Palmer) qui servirait à améliorer le projet éducatif en cours de production et qui garantirait sa validité "écologique". Finalement, le technologue se méfierait d'une définition de l'éducation à la santé qui n'aurait que la modification du comportement comme but (Hirst et Petr, Masee...).

Cette structure, contestable du point de vue de l'éducation s'explique, en fait, par le large éventail de personnes à qui le livre s'adresse et par le pré-requis mentionné à la page 68. Le livre s'inscrit en fait à la suite d'un document fort populaire dans le domaine de la santé, Planification de la santé (Pinault et Daveluy, 1985). Il est, par ailleurs, possible de l'utiliser pour y rechercher des informations en éducation à la santé et l'introduction facilite cette recherche. Il s'agit donc d'un livre de référence de langue française unique en son genre.
L'éducation à la santé est un champ vaste qui peut bénéficier de l'apport de plusieurs disciplines: le monde de la santé communautaire vient, grâce à ce livre, de faire une percée significative au Québec. Souhaitons que des conseillers en éducation, en communication ou autre contribuent à compléter cette perception de l'éducation à la santé.

REVIEWER

Cécile Michaud is an instructor at CEGEP du Vieux Montreal and a doctoral student in Educational Technology at Concordia University.

*Developing Competent Health Workers: A Handbook for Designing Education and Training Programs* by Lori Vanderschmidt, Thomas Frostman, John McCollum, and Ascher Segall. Boston, MA: Boston University Center for Educational Development in Health, 1985,227 pages. Available in English and French (*French text follows on page 159*).

*Reviewed by Gordon Trueblood*

The handbook is truly an international volume. It is the product of a research and development project supported by the United States Agency for International Development. The preparation and field-testing of this book represent the collaboration and participation of eight different nations on four continents. This is an important point as the models used in the text have been tested across socio-cultural situations.

In evaluating this book it is important to consider the audience for which it is primarily intended. The book is directed to the health professions and the training of primary and public health workers in third world countries. The target audience are people who probably have had little or no formal preparation in instructional systems design, but who must get on with the important task of developing courses, curricula, and training programs. These people, and others similarly situated would derive considerable benefits from the simplicity of the developmental model used in this book.

The book is well written and logically organized, with a liberal use of guidelines, headings, charts, and examples. This facilitates its use as a "handbook," to the extent that information is easy to find and easy to follow. The book is organized into three major parts: the process on how to develop courses (4 chapters, 63 pages); a methods section for a) completing steps in Part 1 and b) teaching methods (36 pages); and examples of courses developed following the handbook process (4 examples, 118 pages). The page allocation

<footnote>(1) Extrait tiré de la couverture du livre.</footnote>
gives a rough idea of where the emphasis of the book lies. But to get a better conceptual map of the book, some explanation of what each section covers is in order.

The first section is more than one might expect at first blush. It deals with the general problems of what students should learn and, instead of starting with what to teach, it starts with defining the instructional situation which orientates the designer to the training environment and input variables (number of students, setting, resources and constraints). The section then moves on to skill development in the preparation and verification of job descriptions, task analysis, evaluation plan, writing educational objectives, selecting and sequencing course content, tests and evaluation. Each component is clearly explained in terms of its relationship to other components and reinforced with practical exercises and self-check and review.

The second section concerns methods and provides elaborations on methods of job analysis and verification, student assessment methods, and teaching methods. The elaboration of methods for job analysis and verification considers a variety of ways of "checking-out" the job description with administrators, teachers, clients and those already doing the job. Student assessment is described from the point of view of a variety of methods that are appropriate for assessing skills and/or knowledge relevant to the tasks of the job description. The teaching methods section describes a variety of methods that can be used in individual and small group learning situations. Methods are matched with teaching goals and to the degree and level of learner performance. Given the international perspective of this book, the teaching methods are limited to eight or nine methods which are feasible across cultural and economic differences. More could have been said about the advantages and disadvantages of certain teaching methods and the advantages and limitations of different media. The section is also short on specific training techniques and strategies and learning activities, but to avoid transgression of socio-cultural norms, this would best be articulated at the country and local level.

The third and largest section represents four actual (detailed) examples of courses, of varying duration for varying levels of workers, developed following the handbook process. This section is the largest because each example is a thoroughly articulated program planning document which provides a concrete model to the users of the book on how the final document should appear. The book under-represents the utility of the final document. In addition to serving as a guide to program staff as implementation and evaluation are carried out, such a detailed record is also used by others to justify funds needed for program implementation, in which case a budget section would be required before the plan is considered complete.

The greatest shortcoming of the book is the total lack of references for the user who may seek more depth information about any particular topic.

One final point: the volume has earned a reputation as a useful textbook. In that context it is used in a graduate course at Harvard University School of Public Health to teach health professionals knowledge and skills in the design
and development of competency based training programs for health workers.

*L'élaboration systématique d'un plan d'enseignement*, par Lori Vanderschmidt, Thomas Frostman, John McCollum et Ascher Segall. Boston, MA: Boston University Center for Educational Development in Health, 1985,227 pages. Disponible en anglais et en français.

Le manuel constitue véritablement un volume international. Il est le fruit d'un projet de recherche et de développement appuyé par l'Agence des états-Unis pour le développement international. La préparation et l'essai sur le terrain de cet ouvrage représentent la collaboration et la participation de huit nations différentes sur quatre continents. Ce point est important pisque les modèles utilisés dans le manuel ont été mis à l'épreuve dans plusieurs situations socio-culturelles.

En évaluant le livre, il importe de tenir compte des principaux publics cibles. Le livre s'addresse aux professions de la santé et vise la formation des travailleurs du secteur des soins de santé primaires et de la santé publique des pays du tiers monde. Les lecteurs cibles sont des personnes qui ont probablement peu ou point de préparation officielle pour la conception de systèmes didactiques, mais qui doivent s'attaquer à l'importante tâche de préparer des cours, des programmes d'études et des programmes de formation. Ces personnes, ainsi que d'autres qui se trouvent dans des situations semblables, puisque le livre représente une méthode ou technique de mise au point de programmes de formation, tireront grandement profit de la simplicité du modèle de développement utilisé dans le livre.

Le livre est bien écrit et logiquement organisé, et fait un usage profus de lignes directrices, de rubriques, de graphiques et d'exemples. C'est d'ailleurs ce qui facilite l'utilisation en tant que "manuel," et ce, à un point tel que les renseignements sont faciles à trouver et à saisir. Le livre comprend trois parties principales: le processus de préparation de cours (4 chapitres); une section de méthodes pour les étapes figurant dans les quatre premiers chapitres ainsi que des méthodes d'enseignement (3 chapitres); et des examples de cours qui ont été préparés selon le processus présenté dans le manuel (4 exemples).

La première section traite des problèmes généraux quant à ce que les étudiants devraient apprendre. Cette section commence par la définition de la situation d'apprentissage, qui oriente le concepteur par rapport au milieu de formation. La section porte sur le perfectionnement des aptitudes dans la préparation et la vérification des descriptions de tâches, l'analyse des tâches, le plan d'évaluation, la rédaction des objectifs d'enseignement, le choix et l'enchaénement du contenu du cours, les tests et l'évaluation.

La deuxième section porte sur les méthodes et explique en détail les méthodes d'analyse et de vérification des tâches, les méthodes d'appréciation des étudiants et les méthodes
d'enseignement décrit une variété de méthodes qui peuvent être utilisées dans les situations pédagogiques où il n'y a qu'un étudiant ou un petit groupe d'étudiants. Les méthodes sont assorties aux buts pédagogiques ainsi qu'au degré et au niveau de rendement de l'apprenant. tant donné la perspective internationale de cet ouvrage, les méthodes d'enseignement sont limitées à huit ou neuf méthodes qui sont applicables peu importe les différences culturelles et économiques.

La troisième et la plus importante section représente quatre exemples réels de cours préparés selon les méthodes prescrites dans le manuel. Il s'agit de la section la plus volumineuse parce que chaque example est un document minutieusement articulé de planification de programme. En plus de servir de guide au programme et au personnel au fur et à mesure que se déroulent la mise en oeuvre et l'évaluation, le document de planification de programme peut également être utilisé par d'autres personnes pour justifier les fonds requis pour la mise en oeuvre du programme, dans quel cas une section budgétaire serait requise.

La plus grand lacune du livre est l'absence totale de références pour l'usager qui voudrait peut-être obtenir des renseignement plus approfondis sur un projet particulier.

Un dernier point à souligner: le manuel est utilisé à la Harvard University School of Public Health dans un cours du niveau supérieur pour enseigner aux professionnels de la santé les connaissances et les aptitudes en matière de conception et de préparation de programmes de formation axée sur les compétences qui sont destinés aux travailleurs du secteur de la santé.

REVIEWER

Cordon Trueblood is Director of Health Education for the Medical Services Branch, Health and Welfare Canada and a doctoral student in Educational Technology at Concordia University.
**Information for Authors**

CJEC welcomes papers on all aspects of educational communication and technology. Topics include, but are not limited to: media and computer applications in education, learning resource centers, communication and instructional theory, instructional design, simulation, gaming and other aspects of the use of technology in the learning process. These may take the form of reviews of literature, descriptions of approaches or procedures, descriptions of new applications, theoretical discussions and reports of research,

**Manuscript Categories**

Manuscripts may fall into one of two classes: General, dealing with a topic or issue at a general level (although reference to specific instances or examples may be included), and Profiles, dealing with or describing only a specific instance of an approach, technique, program, project, etc. A Profile may be thought of as a descriptive case study.

Most manuscripts dealing with a topic in general should include reference to supportive literature, while manuscripts submitted to the Profile category may or may not. The Editor reserves the right to change the designation of a manuscript or to make a designation, if none has been made previously by the author. Authors interested in determining the suitability of materials should consult past issues of CJEC or contact the Editor.

All manuscripts received by the Editor (either general or profile) will be judged for suitability, contribution, accuracy, etc. by a panel of anonymous reviewers designated at the time of submission. Normally, the review process requires about eight weeks. There are no deadlines for the submission of manuscripts.

**Manuscript Preparation**

Manuscripts should be typed on 8 1/2 x 11-inch ordinary white paper. All materials must be double-spaced, including quotations and references. Include a title page on which appears the title of the manuscript, the full name of the author(s) along with position and institutional affiliation, mailing address and telephone number of the contact author. An abstract of 75-150 words should be placed on a separate sheet following the title page. While the title should appear at the top of the first manuscript page, no reference to the author(s) should appear there or any other place In the manuscript. Elements of style, including headings, tables, figures and references should be prepared according to the *Publication Manual of the American Psychological Association*, 3rd Edition, 1983. Figures *must* be camera-ready.

**Submission of Manuscripts**

Send four copies of the manuscript to the Editor along with a letter stating that the manuscript is original material that has not been published and is not currently being considered for publication elsewhere. If the manuscript contains copyright materials, the author should note this in the cover letter and indicate when letters of permission will be forwarded to the Editor. Manuscripts and editorial correspondence should be sent to: Richard A. Schwier, Canadian Journal of Educational Communication, College of Education, University of Saskatchewan, Saskatoon. SK, S7N OWO.