CANADIAN JOURNAL OF EDUCATIONAL COMMUNICATION

Volume 24, Number 2, Summer 1995
ISSN 0710 - 4340

Special Issue On Distance Education
Guest Editor Dan Coldeway

Learner Involvement: A Review of the Elements of More Effective Distance Education
Tom Jones
E. Schieman

Motion Curricula and Non-Motion Curricula in Distance Education
Technology Selection Reconsidered
Darcy Walsh Hardy
Judy C. Ashcroft
Michael D. Abbiatti

Learning Together at a Distance
Michelle Savard
Phillip C. Abrami
Sidney N. Mitchell

Successful Electronic Distance Collaboration: The Importance of Social Negotiation
Phil Black

The Perceptions and Needs of Faculty in Distance Education Courses in a Conventional University
Margaret Landstrom

Using a Transactionist Model in Evaluating Distance Education Programs
Mary F. Kennedy
Blair W. Kettle

An AMTEC Publication
EDITORIAL BOARD

Farough Abed
Central Connecticut State University

Cheryl Amundsen
McGill University

Jon Baggaley
Concordia University

Robert C. Branch
Syracuse University

Robert M. Bernard
Concordia University

Katy Campbell-Bonar
Keewatin Community College

W. Bruce Clark
University of Calgary

Dan Coldeway
Athabasca University

Thomas M. Duffy
Indiana University

Rodney S. Earle
Brigham Young University

D. J. Engel
University of Alberta

George L. Geis
OISE

Edward S. Halpern
AT&T Bell Laboratories

Michael Hannafin
University of Georgia

Denis Hlynka
University of Manitoba

W. J. Hunter
University of Calgary

Robert Jones
Sheridan College of Applied
Arts and Technology

Lorne Koroluk
University of British Columbia

Richard Kenny
University of Ottawa

James J. LaFollette
University of Alberta

Richard F. Lewis
University of Windsor

William Milheim
Penn State Great Valley

Earl R. Misanchuk
University of Saskatchewan

Ronald Owston
York University

Beverley A. Park
Avalon Consolidated School
Bd. of
St. John's

Lauran Sandals
University of Calgary

Louise Sauvé
Télé-Université

Richard F. Schmid
Concordia University

R.J. Schmidt
Strathcona County Board of
Education

Richard A. Schwier
University of Saskatchewan

Steven Shaw
Concordia University

Mariela Tovar
Concordia University

Rolland Viau
Université de Sherbrooke

Clayton R. Wright
Grant MacEwan Community
College

AMTEC BOARD OF
DIRECTORS

**President**
Gary Karlsen
Magic Lantern Films

**Past President**
Allen LeBlanc
Sault St. Marie School
Board

**President Elect**
Richard A. Schwier
University of Saskatchewan

**Secretary / Treasurer**
Mary Anne Epp
Langara College

**Director**
Danielle Fortosky
University of Saskatchewan

**Membership**
Genevieve Gallant
Cabot College
Canadian Journal
of Educational
Communication

Volume 24, Number 2
Summer 1995

Editor

David A. Mappin

Guest Editor

Dan Coldeway

MEDIAWARE REVIEW EDITOR

Len F. Proctor

BOOK REVIEW EDITOR

Diane P. Janes

Translation

Nadia Rousseau

Production Manager

Pat Wong

Patrician Desktop Publishing

ISSN 0710-4340

The Canadian Journal of Educational Commu-
nication is published by the Association for Media and Technology in Education in Canada; 3-1750 The Queensway, Suite 1318, Etobicoke, Ontario M9C 5H5; Attention: Ms. Mary Anne Epp, Secretary/Treasurer. Notification of address change should be sent to the above. All articles are copyright by AMETEC and may be reproduced for non-profit use without permission provided credit is given to CJEC. Back Issues of CJEC are $16.05 Canadian and may be obtained by contacting the Editor. CJEC is indexed in the Canadian Education Index and ERIC.

Second Class Mail Registration No. 6956

GUEST EDITORIAL

Special Issue on Distance Education
Dan Coldeway

95

ARTICLES

Learner Involvement: A Review
of the Elements of More
Effective Distance Education
T. Jones
E. Schieman

97

Motion Curricula and Non-Motion
Curricula in Distance Education:
Technology Selection Reconsidered
Darcy Walsh Hardy
Michael D. Abbiatti
Judy C. Ashcroft

105

Learning Together at a Distance
Michelle Savard
Sidney N. Mitchell
Philip C. Abrami
Maria Corso

117

Successful Electronic Distance
Collaboration: The Importance of
Social Negotiation
Phil Black

133

The Perceptions and Needs of
Faculty in Distance Education
Courses in a Conventional University
Margaret Landstrom

149

Using a Transactionist Model in
Evaluating Distance Education
Programs
Mary F. Kennedy
Blair W. Kettle

159

MEDIAWARE REVIEW

171

BOOK REVIEWS

181

All correspondence should be
addressed to:

DAVID A. MAPPIN
Division of Technology in Education
3-102 Education North
University of Alberta
Edmonton, Alberta, T6G 2G5
e-mail: David.Mappin@Ualberta.ca
1995-96

Subscription

Membership Form

AMTEC c/o Genevieve Gallant

3-1750 The Queensway, suite 1818

Etobicoke, Ontario, Canada M9C 5H5

Name

Title

Address

City

Prov/Postal code

Phone(w)

Phone(h)

Fax

Netmail

<figure><img src="image_1.png" /></figure>

I am willing to have pertinent information from the above included in the next Members Handbook

ALL RENEWALS COME DUE ON EITHER JANUARY 1ST OR JUNE 1ST

If this is a renewal then please indicate your Membership Number

☐ Student of Educational Technology (includes GST) $32.10

☐ Retiree (includes GST) $32.10

☐ Individual (includes GST) $80.25

☐ Commercial (includes GST) $80.25

☐ CJEC Subscription Only - Canada $45.00

☐ CJEC Subscription Only - U.S.A. $45.00 U.S.

☐ CJEC Subscription Only - Outside North America $45.00 U.S.

*Students must have this form signed by your Department Head:

GST # 122939853

Special Interest Groups:

Please duplicate this form

Members should choose one free special interest group.

Additional SIG memberships are $10 each.

- K-12

- Multimedia

- Research & Theory

CJEC is typeset in
Ventura 5.0™

Printed by:

QuebecorJasper Printing Inc. Edmonton, AB

Acknowledgment

The Canadian Journal of Educational Communication
is supported by a grant from the Social Sciences and
Humanities Research Council of Canada.
SPECIAL ISSUE ON DISTANCE EDUCATION

Dan O. Coldeway
Guest Editor

Welcome to the special issue on distance education. I hope you find the articles and information interesting and useful.

This special edition presents work that resulted from the standard edito-
rial process of CJEC. The result is six articles that demonstrate the range and
diversity within the literature on distance education. The first two articles by
Jones and Schieman and by Hardy. Abbiattii and Ashcroft are discussion pa-
pers using an analytical approach to issues facing distance educators. Both of
these articles serve as an introduction to the topic of this special edition. Ana-
lytical methods (e.g. philosophical, conceptual, and historical approaches) are
frequently found in the distance educational literature and are important con-
tributions at many conferences.

The third and fourth articles report on work resulting from an empirical approach to distance education. The Savard. Mitchell, Abrami and Corso article reports research targeted toward a better understanding of students working together at a distance. The Black article also looks at long distance collaboration using a different approach. Empirical studies are an important contribution to distance education and often difficult to plan and complete. Both the methodology and the results of these two studies should be of interest to many readers.

The final two articles represent an evaluation/case study methodology.
The article by Landstrom focuses on faculty perceptions of distance teaching.
This topic is important as conventional faculty are encouraged to participate
in distance educational design and delivery. The article by Kennedy and Ket-
tle reports the findings of a distance education program evaluation. Again,
the findings and methodologies used in both evaluation/case study articles
should be of interest to many readers.

I was pleased to receive two articles from outside of Canada for this spe-
cial edition. Although Canadians have been very instrumental in distance
education development and scholarship, distance education is truly of world-
wide interest and importance.
The remainder of the edition contains book and media reviews. I would
like to thank the previous editor of CJEC, Mary Kennedy for her assistance
in arranging for this contribution to this edition.

The debate over distance education as a discipline, its place in education
and the social sciences, and its utility for all levels and types of education
and training will no doubt continue. I hope this edition contributes and
moves that important discussion further along.

Finally, I would like to thank the authors for their hard work on this edi-
tion. I would also like to thank the editorial board and many outside review-
ers for their comments and insight.

GUEST EDITOR

Dan O. Coldeway
Learner Involvement: A Review of the
Elements of More Effective Distance
Education

T. Jones

E. Schieman

Abstract: Three factors have particular relevance for the designers of instruction for adult distance learners: 1) means and resources by which independence (Le., learner control) is supported and nurtured; 2) access to different interaction levels and 3) availability and ease of use of different technology platforms.

To promote a high level of learner involvement in distance instruction/learning, instructional developers and designers should strongly consider what weight will be given these factors. Systematic program and course design which attends to independence. interaction and technology should result in more effective distance education.

Résumé: Trois facteurs ont une importance particulière pour les concepteurs de programme d'enseignement à distance pour adultes: 1) les moyens et les ressources par lesquels l'indépendance (c.-a.-d. le contrôle de l'étudiant) est supportée et favorisée; 2) l'accès à des différents niveaux d'interaction et 3) la disponibilité et la facilité d'utilisation de différentes technologies.

Afin de promouvoir un niveau élevé d'implication de la part de l'apprenant à distance, les concepteurs en éducation devraient considérer l'importance accordée à ces trois facteurs. Les programmes et les cours systématiques qui mênent à l'indépendance, à l'interaction et à la technologie devraient résulter en des programmes d'éducation à distance plus efficaces.

The applications for media and technology in our post-industrial society, our “electronic cottage” society and our “electronic highway” culture must be re-thought, re-organized and re-configured to address better the changing needs of the individuals who comprise the sub-groups of this society. The historical utilization of educational media - e.g., film, video and computer - based learning materials, and their variants - can no longer serve as the model for education in the future. This appears to be especially true for distance learning environments where interaction, asynchronous uses and heterogeneous audiences are being served. There currently exists a plethora of educational opportunities at the post-secondary level which attests to the
needs of adult and young learners. For prospective learners, there exists a
wide range of diverse choices, far beyond the restricted application of tradi-
tional correspondence-like courses and programs which were the norm only
a short time ago. This diversity and range of opportunity has resulted in the
search for new, novel and innovative technological delivery mechanisms by
those with the responsibility for creating, delivering and evaluating these edu-
cational opportunities.

MOTIVATION FOR CHANGE

Innovation in the ways and means of instructing and learning, as it applies
to individuals who are forced by circumstance to take their instruction at the
workplace or the home, the vacation cottage or the office in the high rise, can
be dealt with in a number of perspectives. One commonly cited view is that
learning can be enhanced by via the introduction of new forms of instruc-
tional media. For educators, this can dictate a focus on technologies which ac-
commodates the needs and the concerns of the learner. However, new appli-
cations of technology must do more than make the instruction more glamor-
ous, result in a faster transmission rate, provide a cleaner electronic signal or
require the instructor to become more of a techno-whiz. Rather, the technolo-
gies of instruction and the developers of the substance of instruction the in-
structional designers, must facilitate the learners in the taking control of their
learning. This generation of interactive technologies must make it possible
for distance education learners to monitor the process of learning and thereby
construct and reconstruct knowledge. Further, these technologies must make
it possible for distance education learners to be self-reflective and self-correc-
tive during learning. Any instructional tools used in instruction, and particu-
larly in distance learning, must be designed and constructed to be responsive
to the needs of the individual and the iterative nature of thinking and learn-
ing. It can be anticipated that the technologies of delivery and instruction of
the next decade will allow learners to pace, sequence, assess and negotiate
their strategies for completing assignments and for locating, accessing and
manipulating information pertinent to their situation. These new technologies
(new at least in the ways in which they are employed in instruction) can of-
fer increasingly more convenient and more effective channels of educational
opportunities to a society which has become more mobile, more sophisti-
cated and more demanding in gaining access to education and training. The
clients of education clearly have become more diverse and this alone sug-
gests that if current and future educational missions are to be fulfilled, a sys-
tem that is at once flexible, individualistic and comprehensive must be
adopted. This scenario suggests a just-in-time education/training model be in-
corporated into what educational institutions do.
INDEPENDENCE, INTERACTIVITY AND TECHNOLOGY

Distance education has been confounded by the dilemma of how to con-
struct and to communicate messages which are germane to the course or pro-
gram under consideration and at the same time of how to deal with the issue
of accommodating the needs of the learners when it is now recognized more
than ever before that learners learn in different ways and at various times and
locations. The aspects most often missing from distance education course de-
velopment models are the provisions for activities which can be conceptualized as being appropriate for learners who are independent learners by nature
and where the content can be delivered in forms which allow the learner to
manipulate the materials in an independent fashion.

INDEPENDENCE

Historically, the independent learner has been viewed as someone who
was working in isolation with little or no involvement with either the instruc-
tor, tutor or other learners. More recently, the independent learner may be de-
scribed as one who may choose to be involved or not to be involved in inter-
active instruction elements in the context of a formal lesson or program. The
instructional design solution is to provide alternatives for the independent
learner - for example, (a) watching or listening to broadcast programs (or
audiocassette/videocassette configurations) and responding in written form
such as term papers, reviews, journals; (b) reflective, personal musings of
some kind; (c) computer-based learning, where the learner is working alone
at a computer terminal (without connections such as would be the case as
with e-mail). The inclusion however, of the aforementioned electronic con-
nection, quite suddenly places the scenario in the domain of interactive in-
struction.

What has been traditionally been considered good course design has now
been identified as being inadequate for many of the clients of the distance-de-
livering institutions. That is, the concern in course development remains how
the designer provides for learners who range from dependent to independent.
As well, there is the concern for integrating the opportunities for interaction
deemed so essential by most designers. Superimposed on the above two is-
sues in distance education is the question of how the technologies of instruc-
tion are most appropriately employed and which technologies are suitable in
which instructional situations.
INTERACTION

Interaction has been defined in various ways with various purposes in mind by distance education authors. Interaction itself can take many forms all based on the level of involvement by the participants of the instructional experience. For this discussion, the definition used by Daniel and Marquis (1983) is cited in order to make the point that the issue at hand, as far as the debate over interaction needs is concerned, is indeed a most complex one. Daniel and Marquis use a somewhat restricted definition for explaining the activities in distance education as taking place when "the student is in two-way contact with another person (or persons) in such a way as to elicit from them reactions and responses which are specific to his own requests or contributions". In this definition, there is a technological reference made to teaching activities involving telecommunication systems. In education generally and in distance education in particular, the emergence of the newer technologies have increasingly featured greater opportunities for interaction. These innovative technologies of distance education are defined by Rice (1984) as technologies "that allow or facilitate interactivity among users or between users and information". In a similar vein, Lundin (1989) has suggested six levels of interaction which are identifiable when telecommunication systems are used for the distance delivery of instruction. He describes these levels as:

Level 1: 'reaction' as a form of interaction with prepared audio (radio) and video (television) broadcast. This is a voluntary, usually passive and, therefore an ineffective and often unproductive kind of interaction;

Level 2: 'parallel participation' in which the program shows activities and asks listeners or viewers to carry out the same activities. For example, 'Play School' and yoga lessons on television;

Level 3: 'limited interaction' in which the participant has choices regarding the exploration of a fixed data base. For example, viewdata (Viatel Ltdidon) is claimed to be interactive in this way, as are most data bases and programmed learning;

Level 4: 'responses' requested as a form of interaction built into the program software. For example, a 30 minute audio or videotape can be produced in such a way as to keep a student involved for up to a week or two to study by requesting certain activities and investigations to be carried out, then returning to the tape, and so on;
Level 5: 'simulated' interaction in which the program acts as a catalyst for local, real, live interaction among participants;

Level 6: 'live' transactional interaction at a distance- i.e., interaction by which participants can, by comments and questions, contribute to the creation of the unique content or data base which becomes the product of the program or event. This interaction can be both synchronous (e.g., audio and video teleconferencing) or asynchronous (e.g., computer conferencing).

Education, regardless of format or environment, is a social process and therefore some form of interaction can be assumed to be desirable. This can be considered even though there exists some research evidence indicating that the level of interaction may not have an overall impact on performance, but does affect learners in matters of being at ease with the methods and enjoying the instruction (Richie and Newby, 1989). For the professional distance educator, the concerns of interaction can have several pervasive implications. Quality instructional materials should include in the process, the considerations of learning styles, of teaching styles, of course planning and the type of distance education delivery methods to be employed. Clearly to disregard or to be unaware of these concerns is to directly affect the quality of the programs and courses being developed.

TECHNOLOGY

The role of technology in distance learning, and indeed in instruction and training generally, has been debated for some time in the professional literature. The relative merits and limitations of various formats has discussed by Wilkinson (1980), Clark (1983), Carlson and Ross, Sullivan and Tennynson (1992). For the distance education arena, this can especially pertinent as the technologies of distance education are so integral to the process. Authors such as Clark (1983) have made a strong case for debunking the notion that, in cases in which more sophisticated technologies exist, more effective instruction results. Edling and Paulson (1972) on the other hand have pointed out that technology can do the following with certainty, accuracy and speed: (a) make information permanent, (b) make information more accessible, and (c) make information different. Recent developments in the technologies of distance education have seen a movement away from those systems which merely deliver the pre-programmed material of the instructor (that is, distribute the notes, the overhead transparencies, the video clips, etc.) to strategies which allow for increased flexibility and hence independence of the learner. The appearance of such innovations as local-area networks, optical storage (including CD-I), telecommunications and collaborative group
software hold much promise for distance education instructors and learners. As learners gain access to remote local-area networks (LANs) and as distance educators makes more and better use of such technologies as recordable CD-ROM, audiographic systems, videoconferencing and specialized software designed for real-time group interaction, the enhancements of distance education delivery will likely result in more effective learning. What is needed is for a model of course development which provides for an appropriate balance between pedagogy and technology and, if done successfully, will result in benefits to the expanding community of adult learners.

How to best utilize the technologies of distance education? Romiszowski (1988) pointed out a major deficiency when he noted that the means to overcome some apparent limitations of distance education were urgently required. He cited several difficulties, namely; the impersonality, the inflexibility and limitations of centralized systems of teaching and learning as being obstacles to the expansion of meaningful instruction to learners at a distance. He noted that "distance education can be interactive, can involve groups as well as individuals, can be totally private and/or one-to-one when required and can be learner-controlled." Barnard (1992) echoes these sentiments in his observation that the merging of computer and video technology, which has given rise to multi-media, will open new avenues of communication and "remove the barriers of time and distance" between learner/learner and instructor/learner. Microcomputer-based systems which allow for the integration of text, high-resolution graphics, digital sound, still- and full-motion video and which can access external sites (server or peer-to-peer LANs, bulletin boards, individuals) via modem will give all members of a distance-delivered course the tools to interact with any one or all of the group at virtually any time they wish. Other smaller-scale technologies (e.g., CD-ROM reader, VCR) will lend strong support to the "stand-alone" learner.

## OTHER CONCERNS

Distance educators have long carried on the discussion of the relationship between learning, interaction and the role of the learner. References to "active learners" and "interactive learning" abound in the literature as do the references to the importance of feedback to the learner. These terms are loaded with multiple meanings that depend on situational factors. One can conceive of the meaning of "active learning" as being as simple a manoeuvre as pushing the "play" button on a VCR and of "interactive learning" occurring when the learner is required to perform simple manipulations of learning materials such as changing the tapes in the audio cassette player. Clearly these interpretations of "active learning" and "interactive learning" are restrictive and limited and do not convey the true meanings of the terms as described in the dis-
tance education literature. What is required in these instances is deeper mean-
ing so the researchers can go about the business of hypothesis-generation.
There must be encouraged in the literature a greater sense of which techno-
logical strategies are reactive and which are proactive. It is suggested that in-
structional designers identify ways that distance education technologies can
be used in proactive ways. In the example of computers being used in dis-
tance instruction, designers might avoid using the devices as tutors but rather
as a tool to solve problems of assisting the learner in formulating novel ap-
proaches to dealing with new and complex situations and of putting into the
hands of learners a set of tools which can help them become active rather
than passive learners i.e., inquiry learners rather than plodders in the library
or the classroom. In this way, the device can become another way in which
learners can learn rather than another way in which instructors can teach.
This can become important to the instructional designer where the tool (the
schema where instruction is viewed from the perspective of the role of the
learner) is the focus for the design of the course or program and the computer
(or any other technological device). In this way, the designer can conceptual-
ize and implement alternative ways to think about and design technology-
based instruction.

CONCLUSIONS

It is now more apparent than ever before that no one medium can facili-
tate learning better than any other medium or that any one medium can suf-
fice for all instructional applications. What does appear to matter are the
methods and models employed in the systematic design and development of
the courseware. Clearly, what distance educators must attend to in the design
process are the elements of effective design (Dick and Carey, 1992). Consid-
erations such as the analysis of the learning outcomes and the media charac-
teristics required to achieve the desired performances or knowledge need to
be addressed. Those technology systems that accommodate learner-controll-
ed pacing and encourage learner independence should be considered for the
courseware design. Technologies which are flexible by nature and allow high
participatory intervention or interaction by individual learners are ones to be
considered for use. By focussing on the design of the learning materials
rather than on the technologies, the designer can move to ensure that critical
knowledge or performance concerns are identified and accommodated. As
well, accurate and manageable goals are to be included, with instructional
strategies which are suitable for the content and the clients. With these con-
siderations in mind the learning materials aimed at the distance learner can
be made to be more effective.
REFERENCES

Barnard, J. (1992). Multimedia and the future of distance learning technology, *Educational Media International*, 29(5), 129-144.

Carlson, H., (1991), Learning style and program design in interactive multimedia, Educational Technology: *Research and Technology* 39(3), 41-48.

Clark, R. E. (1983). Reconsidering research on learning from media, *Review of Educational Research*, 53, 445-460.

Daniel, J. & Marquis, C. (1979). Independence and interaction: getting the mixture right, *Teaching at a Distance*, 15, 29-44.

Dick, W. & Carey, L. (1990). *The systematic design of instruction*. Glenview, Illinois: Scott, Foresman, Little & Brown (3rd edition).

DiVesta, F.J. & Reiber, L.P. (1988). Characteristics of cognitive instructional design: The next generation. In *Association for Educational Communications and Technology, Proceedings*, Ames, Iowa: Iowa State University, 220-221.

Edling, J. V. & Paulson, C.F. (1972) Understanding instructional media. In *National Special Media Institute, Contributions of behavioral science to instructional technology, Vol. 2: The cognitive domain*, Washington, D.C.: Grypon House, 161-199.

Lundin, R. (1989). Interaction and participation: The key elements in the design of educational videoconferencing. Paper delivered at the *Pacific Telecommunications Eleventh Annual Conference*, Honolulu, Hawaii.

Rice, R. (1984). New media technology: growth and integration. In R. Rice & Associates (Eds.), *The new media*. Beverley Hills: Sage, 33-54.

Richie, H. & T. J. Newby. (1989). Classroom lecture/discussion vs. live televised instruction: A comparison of effects on student performance, attitude and interaction, *American Journal of Distance Education*, 3(3), 36-44.

Romiszowski, A. (1988). *The selection and use of instructional media*. London: Kogan Page.

Ross, S., H. Sullivan & Tennyson, R. (1992). Educational technology: four decades of research and theory, *Educational Technology: Research and Development*, 5-22.

Wilkinson, G.L. (1980). *Media in instruction: 60 years of research*. Washington, DC: Association for Educational Communications and Technology.

AUTHORS

T. Jones is an Associate Professor at Athabasca University.

E. Schieman is an Associate Professor at the University of Calgary.
Motion Curricula and Non-Motion
Curricula in Distance Education:
Technology Selection Reconsidered

Darcy Walsh Hardy
Michael D. Abbiatti
Judy C. Ashcroft

Resume: Distance education programs are delivered through a variety of media, from print to full-motion video. However, the current trend appears to be leading toward the use of video-based systems for most courses, irrespective of the course content and its educational delivery requirements. In other words, more attention should be paid to what a course requires in its delivery to be effective. This article looks closely at why a delivery technology is selected and proposes a method for selecting the medium based on course curricula. A listing of delivery methods is included to assist the reader in determining an appropriate technology.

Résumé: Les programmes d'éducation à distance sont offerts à travers une variété de médias, de l'écrit au vidéo. Toutefois, les présents courants semblent pencher vers l'utilisation d'un système basé sur les vidéos dans la plupart des cours sans tenir compte du contenu des cours et de ses besoins particuliers de livraison. En d'autre mots, plus d'attention devrait être portée sur les exigences particulières d'un cours afin que sa livraison soit efficace. Cette article se penche sur les raisons pour lesquelles on choisit la livraison de la technologie en éducation, et propose une méthode de sélection de médium d'enseignement basée sur le contenu du cours. Une liste de méthodes de livraison est incluse afin d'aider le lecteur à déterminer une technologie d'enseignement appropriée.

This paper is the result of several discussions related to appropriate use of
technology for the delivery of distance education. It is the authors' hope that
readers will benefit from this somewhat different viewpoint of where we are
and where we should be in technology selection today. The current chronol-
ogy for selecting delivery systems is discussed, as well as a proposed method
to identify appropriate technologies for distance delivery based on the need
for motion in the curriculum.
*Present Chronology for Selecting Technology to Deliver Distance Education*

The current trend to invest in compressed video is an example of the need for the development and application of a sound planning sequence that ties curriculum and technology together, based upon actual need and realistic budgets (Guilder 1993; State of Louisiana 1993; Williams et. al. 1990).

Enchanted by the possibilities of distance education delivery systems, the immediate reactions of educators often center more on the technology being used to deliver instruction than on the instruction itself. The interest in distance education, especially in its "newness" and "magic," centers on the technology. The vendors show top-of-the-line wares at conferences and on-site demonstrations, where educators see the possibilities and consequently desire to provide the best for their students and to be on the forefront. The first introduction to distance education is seeing action shown through the hardware of the delivery system.

Some distance education providers feel that the ideal distance learning classroom is one where instruction is delivered to a small number of students via live two-way video and audio using fiber optic technology. Certainly, compressed digital systems are becoming more the technology of choice as the services improve and the costs decline. However, many educators and administrators are inappropriately disappointed if they cannot afford the compression technology at this moment. Many times large expenditures are rationalized; using the "if you build it they will come" mindset. In fact, enrollments do increase in some cases as a result of adding a high-end system (Schriftgliesser 1994; Weiss 1994).

However, distance learning classrooms that are two-way video and audio are not easily obtainable and are often out of reach financially. Unfortunately, many of the discussions today involving distance education focus only upon video-based delivery. In comparison to print, audio or computer-based delivery systems, a video-based delivery program demands the highest initial investment, upkeep, and on-going cost; the most time in preparation and coordination for delivery; and the greatest number of skilled persons for course delivery. Due to these cost barriers, providers strive to develop systems that come as close as possible to the ideal without actually establishing this type of classroom. Granted, the fully interactive classroom may become more available for most aspects of distance education, but it is not now affordable for most institutions. And *now* is what many distance education providers must be concerned with at this point.

As long as the National Science Foundation, the National Telecommunications and Information Administration, and the National Information Infrastructure Initiative serve as major funding sources, high-end technology will flourish. The recent investments in education by telephone companies like Bell Atlantic in New Jersey or U.S. West in Colorado and Wyoming will
ther the cooperative buildout that will eventually result in the true digital In-
formation Highway. What about the educators who have a demonstrated
need to deliver courses, but do not have the clout, politically or otherwise, to
obtain compression technology on a grand (or even a small) scale? (Office of
Technology Assessment 1989; State of Louisiana 1993).

The answer is to decide which technology is actually appropriate to de-
liver a course or courses based upon proven requirements for the curriculum.
Excellent examples of such reality-based decision making can be found at all
levels of the academic continuum (Office of Technology Assessment 1989).
One very useful tool in a search for such examples is the United States De-
partment of Education. Office of Educational Research and Improvement
(OERI) Distance Learning Database (Garnette 1994). In addition, the abso-
lute explosion of interest in distance education has prompted such mainstays
as the *Chronicle of Higher Education* to deal regularly with the innovations
that abound.

The fact that school districts, colleges, and universities are spending liter-
ally hundreds of thousands of dollars on high-end technology is reason
enough to address these issues. Why are distance education providers spend-
ing such large amounts of money when it may not be necessary? This is not
to say that if they have the money they shouldn't use it. After all, as stated
previously, in the ideal distance education classroom, the teacher and stu-
dents can see and hear each other simultaneously. When facing the reality of
having less than two-way video and audio. however, this is a question that
educators should ask regularly: Are we, as distance education providers, pro-
moting delivery technologies based on our own desires to be on the current
edge, or on the instruction to be delivered? The choice does not have to be all
or nothing; but the answer is not a compromise with parts of a video system.
This partial response is often the worst alternative.

Distance education does solve many problems, but any provider must pos-
sess the resource base to minimize the barriers associated with distance learn-
ing: distance, time, and money. The barrier of cost in itself associated with
video-based delivery often limits access to distance learning programs. If the
resources are found and video system installed, the same excitement de-
mands to see immediate results for the large investment. There is expectation
that a course will be delivered soon, and in the experimental stage (which is
also the first impression that persons have of distance education) insufficient
*time and expertise is spent on formulating the instructional design of the*
*course*. When instruction is delivered over distance education, time must be
spent in preparation -the site must be prepared, the teacher must be trained,
and the content must be converted from a traditional format to a distance
learning format.

Evaluation of the effectiveness of the system, for which the school paid
so much and expected even more, is often difficult and inconclusive because
the needs for the distance education delivery system were not clearly set forth at the beginning. The system might work perfectly, the instructor might teach brilliantly, but the course selected might not meet the needs of the students or of the school, so the experience becomes a very expensive "extra." Attempting to force all instruction into a single delivery mode will end up in an information "dead end" for a large portion of our learner population. We are currently in the exponential period of the instructional telecommunications growth curve. There are numerous technology camps, each touting a particular methodology or system of delivery. Many times an inappropriate technology is chosen to deliver a particular course, and a great deal of disappointment ensues. The fact remains that the technology did not fail; rather, the decision process used to determine the technology was flawed.

The concept of distance learning occupies a unique position in the continuum of delivery options in distance education. Schools and universities do not need full-motion video in every teaching/learning environment. On the contrary, providers must maximize the available bandwidth for video in order to assure that the appropriate system has been utilized to reach the desired learning outcome. The overall goal must be to improve student performance.

## Recommended Chronology for Selecting Technology for Delivery of Distance Education

Successful distance education programs can be built, even within budget restraints, when the planning process begins with thoughtful attention to these three steps:

1. determining the need for a program or course, with consideration of program level and the institution's distance learning infrastructure;

2. formulating the instructional design of that course; and

3. selecting the appropriate technology to deliver the course based on the instructional content and design.

Currently, some school districts and universities are placing the third step before the second, inevitably increasing the barriers to the successful delivery of a course or program. Institutions have also been guilty of selecting a technology for delivery without consideration of the course content (i.e., without determining if a particular technology is required to send the instructional message). This paper acknowledges each of these issues, and offers a new approach to selecting an appropriate delivery system. Another way to categorize curricula is provided.
DETERMINING THE NEED FOR A PROGRAM OR COURSE

The first step in any distance education program is to determine the need for the course or courses. Surveys, questionnaires, and interviews can assist a school or organization with this effort. Once an organization determines that — in order to meet the needs of its students — distance education technology must be used, the emphasis must then be placed on the instruction itself, beginning with the design of the course. In addition, a decision must be made regarding the appropriateness of distance delivery for the program level (i.e., undergraduate or graduate). This decision and many that follow in the design and delivery of a course may depend heavily on the distance learning infrastructure that exists at the sponsoring institution.

FORMULATING THE INSTRUCTIONAL DESIGN OF THE COURSE

When designing a distance learning course, there are many questions that must be answered by those involved in the program. What is to be taught? Is teaching students to build a home the primary objective? Is it how to successfully complete complex algebra problems? Or is teaching a second language the goal of the program? Whatever the goal is, questions like these must be asked in order to systematically design the curriculum for the course. Although this paper will not address ways to design a course, this process should be completed prior to selecting the appropriate delivery system.

SELECTING THE TECHNOLOGY TO DELIVER THE COURSE

How does one determine which technology is most appropriate for the content being delivered? A general analysis of the instruction will give the provided valuable information, such as desired learning outcomes and ideas on how to achieve these goals instructionally. But the type of delivery itself can often be determined by placing the curriculum in one of two categories: *motion* or *non-motion*. This is a dramatic departure from the traditional way of selecting the technology which was to choose between video-based, audio-based, traditional, or "other."

Here the terms *motion* and *non-motion* refer to the curriculum, or the type of instruction, not to the delivery. This analysis of the curriculum should be done before the delivery technology is selected. A course contains *motion* curriculum if the instruction requires motion in its presentation to students. In other words, if motion is a mandatory part of the delivery in order for the student to understand the concept(s) being presented, then that is a motion curriculum. If a course is designed to teach students how to complete a scientific experiment that involves measuring and pouring activities sensitive to error, that would probably require motion sequences to teach the course suc-
cessfully. Thinking in these terms, providers may gain insight as to the types
of curriculum that require motion and those that do not require motion.

*Non-motion* curricula are those that can be taught without motion in the delivery. Most high school and college level courses easily fall into this category. Courses such as English, mathematics, history, and other social sciences are taught on a regular basis by lecture and via traditional correspondence study. The reader who is not familiar with correspondence study may be surprised to see mathematics in this list; however, one must remember that, according to the above definition, mathematics is not a motion-based curriculum. Therefore, it should not require a motion-based delivery system to meet its distance education goal (Schmidt, Sullivan, & Hardy 1994).

VIDEO IS NOT THE ONLY WAY TO DELIVER DISTANCE EDUCATION

Audiographics systems allow teaching to be done from any site in the sys-
tem. As a matter of fact, any site with the basic equipment and standard
phone line automatically becomes a virtual classroom. Faculty do not have to
be on the campus of the academic institution to teach a class.

One such innovative approach to delivery at a distance not requiring full-motion video is the simple analog phone-based system deployed in 1990 by the University of Wisconsin (Weiss 1994). Weiss describes an audiographic system used by the College of Engineering to teach a variety of technical courses to students spread over the state of Wisconsin. Classes are small (32 in this semester's Technical Japanese course), as are expenditures. All that is required is a 386 class PC, a VGA monitor, Vis-a-Vis software, a graphics tablet, and a microphone. Standard analog telephone lines are linked to 9600 baud modems to create a virtual whiteboard environment. James Davis, Professor of Technical Japanese, has fully interactive audio and shared graphics communications with his students. Davis prepares graphics ahead of time and transmits them to the students as needed. This type of modified document conferencing has many virtues when applied to the appropriate classes. Davis does not require full-motion video.

A second example, this time in the pre-college environment, is the nation-
ally recognized telelearning program known as Project Outreach (Loftin
1990). This very effective application of non-motion graphics is located in
the small town of Natchitoches, Louisiana, on the campus of the Louisiana
School for Math, Science and the Arts. The facility is a boarding school for
gifted and talented students. Project Outreach was initiated in 1986 as a
means of providing courses necessary to attend in-state colleges and universi-
ties to rural high school students in Louisiana. The program started with de-
livery of courses to over 1200 students in 116 high schools across the state of
Louisiana during academic year 1992-93. Costs are significantly lower than any technology involving full-motion video. Once again, Project Outreach uses an audiographic system closely related to the University of Wisconsin program. Formal evaluations reveal that students learn as well as in a conventional classroom, grades are as high (higher in some cases), and faculty members are revitalized as they learn new teaching techniques (McElveen 1992).

A third program to mention is one offered through the UT TeleLearning Center at The University of Texas at Austin. Although the majority of distance education programs serving schools are delivered by satellite, TeleLearning courses at the high school, college, and continuing education level are more cost-effective because they are delivered by telephone and supplemented with print materials, videotapes, and computer-assisted instruction. During live audioconference lectures, the teacher and students talk with each other using telephone equipment that allows multiple-source input and reception, bridging all participants together in a manner similar to a typical conference call.

The response to the UT program continues to be overwhelmingly favorable. The TeleLearning Center's distance education programs offer a cost-effective method for providing students with course options that would not otherwise be available. In addition, because the Center utilizes economical low-end technology, class size at the high school level is capped at 30 students per section, while the college level classes do not exceed 50 students per section. In addition to the vocational program in Health Science Technology and Spanish I and II for high school students, the Center offers a special program, *Algebra Across the Wire*, for Texas migrant students. Algebra I and II is offered each summer to students as they move around the country, and class averages for the past three years have not dipped below 90% (Schmidt, Sullivan, & Hardy 1994). Obviously, video has not been required for the successful completion of these courses.

The allure and functionality of non-motion graphics in the delivery of distance education came about quite unexpectedly. While directing one of the largest public school distance education programs in Texas, one of the authors taught seven teachers to use the popular Persuasion presentation software as a tool to enhance the fifteen hours per day of full-motion video (ITFS-based) classes taught by the faculty. As time progressed, the classes became more graphically oriented and less dependent upon the full-motion video. Student progress was not adversely effected; teachers were enjoying the activity.
THE INTERNET PROVIDES OTHER OPTIONS FOR DELIVERY

A final fact to support the case for stopping to think through the decision to invest in full-motion video is the tremendous growth in the Internet (Rheingold 1993). Accurate figures on exactly how many people are on the Internet and associated on-line services are very hard to obtain. It is safe to say, however, that the number of students on-line is growing daily. Classes at the elementary, middle school, high school, undergraduate, and graduate levels are easily documented. For example, the phenomenal success of the America OnLine (AOL) service as a teaching tool is worth mentioning. AOL has classes and workshops offered in at least 30 different interest areas on-line. Students receive everything from homework help to college degrees without the involvement of full-motion video (AOL 1995). A member of the AOL faculty has had some excellent experiences as both a teacher and student on-line.

Certainly the ease of building multimedia presentations into classes on-line is a definite plus for computer-based delivery of distance education. In addition, students are not tied to a particular location in order to access the classes. The evolution of Mosaic and other World Wide Web tools will put the power of all types of presentation styles and technologies at the fingertips of anyone with access to the Internet (Rheingold 1993). In fact, some of the "virtual faculty" will choose full-motion video — some will not. *The key will be the question of appropriateness of the technology to the task at hand.*

<table><caption>Table 1
Hierarchy of Technology</caption><tr><td>Compressed voice, video, data (fiber, T-1) (di;al-up)</td></tr><tr><td>Compressed voice, video, data (analog phone)</td></tr><tr><td>Full-motion analog video, voice, data (uplink, ITFS, cable)</td></tr><tr><td>Audiographics + fax</td></tr><tr><td>Audiographics</td></tr><tr><td>Phone/Fax/Downlink</td></tr><tr><td>Phone/Fax/Videotape</td></tr><tr><td>Phone/Fax/Digitizer (scanning "videophone")</td></tr><tr><td>Phone/Slides (telelecture)</td></tr><tr><td>Phone/Fax</td></tr><tr><td>Standard videotape/on-line computer</td></tr><tr><td>On-line computer (Tenet, Internet, BBS)</td></tr><tr><td>Standard videotape/off-line computer/CD ROM</td></tr><tr><td>Off-line computer (CAD/CD ROM)</td></tr><tr><td>Standard videotape + "datatrack"</td></tr><tr><td>Standard</td></tr></table>
If it is determined that the curriculum is a motion curriculum, then the selection of a delivery system is made. Costly mistakes in selecting a technology can be avoided through the use of a well-constructed and constantly updated hierarchy of technology (Table 1) which functions as a decision-making tool along with a carefully thought out distance learning plan.

**Please note that the hierarchy is built upon technologies that are currently available. Each organization should begin with a "basic capability package," consisting of image transfer, video delivery, and on-line communications capabilities. The evolution of a full-fiber backbone will merely require consolidation of categories and the creation of more fully interactive delivery options.

In order to determine which type of delivery is best for a particular course, organizations can use the hierarchy in Table 1 as a flexible menu of options. In doing so, responsibility for the teaching/learning environment is at the building/department level. By applying the hierarchical concept, instructors tailor a system in an effective and economically sound manner. Providers can choose the level of technology actually needed to reach the levels of learning desired. Theoretically, an organization can choose multiple inputs as required.

School districts and universities just venturing into distance education have frequently fallen victim to organizing new programs without giving enough consideration to the type of curriculum being offered. In many cases, the typing teacher or the business teacher inherited the computer classes and is in line for the distance education course, or the data processing department has inherited the distance delivery lines. In other instances, the curriculum is chosen according to which teacher is willing to participate. In other cases the choice correlates with the class where the members of the school board have children. Similarly, many colleges began by putting the technology in the president's conference room, only to find that they had lost access to the equipment.

Given the variety of delivery systems available today, there should never be a situation where a school cannot receive distance education services based upon a lack of means to deliver the services to the school's location. Not every school can afford a satellite hook-up or compressed video technologies, but almost every school has a cassette player, a VCR, and/or a telephone. The suggestion here is not necessarily to avoid video-based or electronic instruction, but to *consider what the curriculum absolutely requires before selecting a delivery system*.

Instructional designers take great care to determine learning outcomes of instruction in order to design a course. Should distance providers not do at least the same to determine how courses should be delivered? The bottom line is this: if a course requires motion within the instruction in order to effectively present the curriculum to the learner, then that course should be
fered either in person or via a video-based delivery system. If, on the other hand, the course to be delivered does not require motion in order to present the necessary information to the learner, then our organizations may be spending a great deal of money just to be on the high-end technology bandwagon.

In summary, there is no doubt that educators want to provide quality education programs for our students at a distance. Neither time, distance, nor funding should prevent a student from receiving the educational opportunities that he/she deserves. If educators cannot reach students via wireless technologies, they can certainly do so by using the ubiquitous telephone or mail networks that literally circle the globe. The time has come to spend more time on the instruction itself, not the delivery system.

Providers must consider what is absolutely necessary to deliver a course or program *before* acquiring expensive studio equipment and large grants to cover expenses. With instructional telecommunications, no student will be denied an opportunity to learn because of technical limitations. The only reason for lack of delivery is the lack of willingness or persistence on the part of the human elements found within the system. Distance education providers cannot afford to be "trapped in the future," and believe that all distance instruction requires full-motion video. Educators should give strong consideration to all forms of distance delivery in order to overcome the barriers of distance, time, and money, and to create high-quality distance learning programs that can be adapted to an ever-changing technological world.

## REFERENCES

* Garnette, C. P. (1994). *Office of Educational Research and Improvement Distance Learning Database*. U. S. Department of Education.

* Guilder, G. (1993). TeleLearning in the electronic school-The information revolution. *The Executive Educator*, pp. 15-18.

* Loftin, R. (1990). TeleLearning project outreach. Louisiana School for Math, Science, and the Arts. Program Description/Promotional Information.

* McElveen, L. (1992). Statewide distance learning program - Telelearning project outreach final evaluation report. Louisiana Evaluation Enterprise, Inc., Louisiana Department of Education.

* Office of Technology Assessment. (1989). *Technology Links: Choices for Distance Learning Systems*.

* Rheingold, H. (1993). *The Virtual Community*. Addison-Wesley.

* Schmidt, K., Sullivan, M., & Hardy, D. (1994). Teaching migrant students algebra by audioconference. *The American Journal of Distance Education* 8(3), pp. 51-63.

* Schriftgliesser, D. (1994). Key trends in broadband communications: The next five years. *Telecommunications Magazine*, pp. 111-108.
State of Louisiana, Board of Regents. (1993). Reaching Louisiana learners: Report of the regents task force on distance learning in Louisiana higher education.

Weiss, J. (1994). Distance learning. *Syllabus Magazine*. Syllabus Press, 8(3), pp. 41-44.

Williams, V., Estes, N., & Bergin, V. (1990). Educating Texans for the information age. The Center for Research on Communication. College of Communication, The University of Texas at Austin.

## AUTHORS

Darcy Walsh Hardy is the Director of the UT TeleLearning Center at the University of Texas at Austin. P.O. Box 7700, Austin, TX. 78713-7700.

Michael, D. Abbiatti is a Distance Learning Consultant at the Louisiana State University in Shreveport, Louisiana. 10019 Raintree Drive, Shreveport, LA 71115.

Judy C. Ashcroft is the Director of EIMC at the University of Texas at Austin. P.O. Box 770, Austin, TX 78713-7700.
116 CJEC SUMMER 1995
Learning Together at a Distance

Michelle Savard
Sidney N. Mitchell
Philip C. Abrami
Maria Corso

**Abstract:** Two studies explored the effects of cooperative and individualistic structures on student learning and attitudes in a simulated distance education environment. In Study 1, 117 male and female college students were randomly assigned to complete a case study assignment online for a business strategies course either with a partner or alone. In Study 2, 107 male and female students were randomly assigned to complete two case studies online either with a partner or alone. In addition, to explore the effects of familiarity, students in the pairs condition were stratified into those who had worked together previously and those who had not. There were few differences between students in their attitudes and achievement due to working cooperatively with a partner or individually. The differences which were found suggested that students in the pairs condition took longer to complete learning tasks and expressed slightly more frustration than students in the individual condition. Nevertheless, students expressed a desire to work with a partner again. The studies also explained gender differences. For female students those in the cooperative condition were more positive about their performance and the opportunity to exchange ideas than those working alone. These results are discussed in terms of both the differences between distance education and traditional instruction and the methodological and design limitations of the studies which limit generalizability. Prescriptions are offered for the design of future research and the implementation of cooperative learning in a distance education environment.

**Résumé:** Deux études ont exploré les effets des structures individuelles et coopératives dans l'apprentissage et les attitudes de l'étudiant dans un environnement qui simulait l'éducation à distance. Dans la première étude, on a demandé à 117 étudiants de niveau collégial, hommes et femmes, de compléter individuellement ou en équipe de deux une étude de cas à l'intérieur d'un cours par ordinateur portant sur les stratégies en affaires, Dans la deuxième étude, on a demandé à 107 hommes et femmes aussi aux études de compléter, individuellement ou avec un partenaire, deux études de cas. De plus, afin d'explorer les effets de la familiarité, les étudiants travaillant en équipe de deux ont été devisés en deux groupes; ceux qui avaient déjà travaillé ensemble, et ceux qui travailleraient ensemble pour la première fois, Peu de différences au plan des attitudes et des réussites ont été notées entre les étudiants travaillant en équipe et ceux travaillant seuls, Les différences trouvées suggèrent que les étudiants travaillant en équipe ont pris plus de temps pour compléter la tâche d'apprentissage et ils
ont exprimé un peu plus de frustration que les étudiants travaillant seuls. Néan-
moins, les étudiants ont tout de même exprimé le désir de travailler en équipe
dans le futur. Les études ont aussi démontré une différence quant aux sexes des
étudiants. Pour les femmes étudiantes, celles travaillant en équipe étaient plus
positives envers leur performance et sur l'opportunité d'échanger des idées que
celles travaillant seules, Ces résultats sont discutés au plan des différences entre
l'éducation à distance et l'éducation traditionnelle ainsi qu'au plan des limites
apportées par le plan de recherche et la méthodologie employée qui limitent
la généralisation des résultats. Des suggestions sont apportées pour des plans de
recherches ultérieurs et pour l'implantation d'apprentissage coopératif dans une
situation d'éducation à distance.

Keegan (1986) described distance education as a learning environment where the learner and teacher are geographically separated. It is also an environment where students are often physically separated from one another. Distance education offers the learner a degree of flexibility seldom encountered in traditional instruction by allowing students to determine where they will receive instruction and when they will learn. Thus, distance education appears to lend itself to individualized learning structures where students learn for themselves and by themselves: there is independence between the means to learning among students and the learning goals of individual students.

Although distance education differs from traditional classroom education,
numerous studies have demonstrated that there is seldom a significant differ-
ence between the attitudes toward learning and the achievement of distance
and traditional learners (Bissell, Coombs, Medvedeff & Rogers, 1987; Black-
wood & Trent, 1968; Boswell, Mocker, & Hamlin, 1968; Cheng, Lehman &
Armstrong, 1991; Hoyt & Frye, 1972; Puzzuoli, 1970). However, distance
education is not problem free; student attrition appears to be a major diffi-
culty (Haile, 1986; Kember, 1989. 1990; Mason, 1989; Sweet, 1986).

The problems in distance education which result in increased dropout
arise from two sources: student related and course related. Student related
factors include home and work environments which interfere with a student's
ability to study (Haile, 1986; Kember, 1989, 1990; Naidu, 1989-1990; Pe-
runiak; 1983). Haile (1986) and Naidu (1989-1990) claim that social and in-
tellectual isolation are two of the course-related factors that contribute to the
decision of distance learners to dropout. That is, distance learners experience
fewer opportunities to interact with course instructors and other students in
order to discuss course content, assignments, learning strategies, and con-
cerns about their learning. This isolation may amplify student perceptions of
external control-the belief that success and achievement are generally a
function of factors beyond individual control. Kember (1990) and Peruniak
(1983) found that external locus of control was a factor related to the drop
out rate. One way to ameliorate the deleterious effects of social and intellec-
tual isolation is to increase the level of learner interaction with other
and the instructor. A medium which lends itself to such interaction among learners is computer-mediated communication (CMC).

## Computer Mediated Communication

CMC can increase student contact with their institution and with their peers (Bissell, 1987; Davie and Wells 1991: Kaye, 1989a; Hiltz, 1988; Henri, 1988). Moreover, Bissell (1987), Davie and Wells (1991) and Kaye (1989b) claimed that the immediacy of contact and feedback made possible by CMC may lead to higher completion rates for distance learners. Kaye (1989) and Mason (1990) stated that CMC facilitates group discussions, encourages learner autonomy, enables frequent feedback and gives learners greater access to experts. Dede (1990) surveyed students using CMC and found that students: were able to benefit from peer teaching; obtained extrinsic motivation from peer approval; and felt that they had more of an equal opportunity to participate in discussions. Schriner (1989) found that computer conferencing appealed to both shy and disadvantaged students. Finally, this medium allows for a permanent record of interaction which can later be used for analysis (Davie and Wells, 1991; Davies, 1988; Kaye, 1989).

Although CMC has the greatest potential for increasing teacher-to-student, student-to-student, and student-to-institution interaction, it does have some disadvantages (Carrier & Schofield, 1991; Cheng, Lehman, & Armstrong, 1991; Davie & Wells, 1991; Harasim, 1987; Henri, 1988; Howard, 1987; Naidu, 1989-1990; Wild & Winniford, 1993). The weaknesses are: software is not "user friendly"; hardware is often unreliable; students frequently do not receive sufficient training in the use of technology; student and teacher workload is sometimes increased; time constraints are amplified by a longer communication cycle; weaker writers are more reticent to participate; and student progress may not be sustained over time.

To take fuller advantage of the strengths of CMC may require restructuring the learning environment to further encourage both the participation and interaction of students. Techniques for group learning used in traditional classroom settings may be adaptable to distance education. Cooperative learning strategies, in particular, may facilitate learning together at a distance. Cooperative Learning

According to Abrami et al. (1995) cooperative learning is an instructional strategy in which students work together in groups that are carefully designed to promote positive interdependence among students. This positive interdependence is coupled with individual accountability so that students are responsible for their own learning as well as a contribution to the group task. Developing positive interdependence can be facilitated in several ways including: sharing resources, working toward a common learning goal, depending on one another for obtaining a reward given equally to all team members, and so on. Demonstrating each individual's responsibility for helping the
group meet its goal successfully is individual accountability. Individual accountability can be encouraged in several ways including: having group members sign projects with their names and area of responsibility, summing individual improvement scores to create a group score, and so on.

The results of a review conducted by Johnson and Johnson (1989) suggested that cooperative goal structures have moderately large positive effects on productivity and achievement compared to either competitive or individualistic goal structures. The average effect sizes exceeded +.60; the percentile advantage for cooperation was about 24 percent. Furthermore, very few of the studies (eight percent or less) showed results where either competitive or individualistic goal structures were superior to cooperative goal structures. Finally, large positive effects were found for cooperative structures on social and affective outcomes.

The results of the more selective review conducted by Slavin (1989) also revealed positive effects on achievement for cooperative learning methods compared to control conditions. The average effect size (+0.21) was modest in size resulting in a percentile rank advantage of almost nine percent. Furthermore, the majority of the studies (72 percent) were in the direction favouring cooperative learning methods. Only a fraction (15 percent) had evidence favouring the control methods.

## CMC and Cooperative Learning

A review of the research revealed two studies utilizing cooperative learning in a CMC environment (Harasim, 1987; Hiltz, 1988). Harasim (1987) analyzed user patterns and rates of participation of students enrolled in online seminars and workshops. Goals and rewards were interdependent among students working in groups. Students reported that the interaction made possible by the online system, promoted learning. They enjoyed the lack of competition for air space and perceived computer conferencing as an "equalizing force". Harasim (1987) reported that students found that on-line courses promoted more equal participation than face-to-face classes. A student reported "I learned much more than in a regular three hour course because of the action of all the students in the course. It is much more interesting this way" (Harasim, 1987, p. 181). Harasim concluded that cooperative learning can be used to provide a highly active, interactive and effective distance learning environment. However, although high rates of participation were found, learning in this study was not directly measured.

In order to determine if cooperative learning was more effective and supportive than the traditional classroom approach, Hiltz (1988) compared the achievement and attitudes of live classes of students who were exposed to frontal teaching, computer mediated communication, and cooperative techniques with CMC. Hiltz (1988) found that course grades were significantly higher in one of the classes using cooperative techniques on-line. However,
there were no significant differences between the midterm or final examina-
tion scores in any of the five courses surveyed. Finally, Hiltz (1988) found
that the group utilizing cooperative learning online collaborated the most out
of the three groups. Forty-nine percent of the students perceived that they
had more interaction with the other students online than they had in the past
in a traditional class.

Together, these studies offer some support for the use of cooperative
learning in distance education. These studies argue against the notion that the
conveniences of distance education-learning alone where and when one
wishes-outweigh the benefits of collaboration with peers. Consequently, we
wanted to explore further the effects of learning structure (individualized ver-
sus cooperative) in a computer-mediated environment on a range of student
variables, including perceived effectiveness, involvement, affect, and
achievement.

In addition, we wondered whether the effects of cooperative or individual-
istic structures would be uniform for all students. In particular, we wondered
whether student gender would interact with structure, particularly in view of
the use of computers as the communication medium.

There is evidence of gender differences in user patterns and attitudes to-
wards computer usage among students. For example, Chen (1986) found that
males had more interest, confidence and respect for computers than females
and that computer anxiety was lower in males than females. Males reported
using the computer 6.1 hours per week while the females used the computer
3.6 hours per week. When the amount of experience and access was control-
lled, significant differences were not found in computer interest between
males and females.

To explore the effects of structure and gender in a distance education envi-
ronment, two studies were completed with groups of university undergradu-
ates working on-line. It was hypothesized that a significant difference would
be found in the achievement and attitudes of males and females working on-
line either alone or in cooperatively structured pairs. It was further predicted
that gender and structure would interact; the effects of structure would be
greatest for female students.

STUDY 1

Method

*Participants.* The sample consisted of two intact classes of male and fe-
male undergraduate business students enrolled at a university in the North-
eastern United States. All students were registered for a business strategies
course. Each class was taught by the same professor during the Spring, 1993.
The classes met twice a week for discussions; in addition, students were
couraged to use email to correspond with their professor and students at other universities, thereby simulating the likely future of business communications. Of the 117 students who participated, 45 were randomly assigned to work individually and 72 students were randomly assigned to work in cooperative pairs. Random assignment insured that experimental groups were equivalent in technical abilities save for the operation of chance.

Students were free to discontinue their participation in the data collection without penalty. Students could also elect not to work in the assigned condition. However, the instructor reserved the right to assign alternative material.

*Design.* Study 1 was a gender (male, female) by structure (cooperative pairs, individuals) between groups factorial design. The unit of analysis was the individual participant.

*Procedure.* Students were informed that they would be receiving an assignment online, which was a case study requiring a solution to a marketing problem, and which was to be completed within a week either individually or in pairs for homework. Those students assigned to the pairs condition were sent their partner's email address and were told to contact their partner in order to complete the assignment. With the assignment, students were given an outline of procedures to follow. They were informed that the assignment would be graded and worth three percent of their final mark. This grade would be either an individual grade or group grade for a collective effort, depending on their individual or pairs status. Students were directed not to speak face-to-face with their assigned partner in order to simulate a distance education environment. The dependent measures were collected in class the week following the completion of the assignment.

*Measures.* An attitudinal questionnaire consisting of 35 Likert scale items was administered. The questionnaire measured: a) the level of cooperative or competitive orientation (6 items); b) the feelings generated as a result of completing an assignment entirely online (16). Also included in this category was a manipulation check that was used to determine if the students actually did complete the task online; c) the effectiveness of the simulation of distance education (3); d) the amount of effort needed to complete the assignment (1); e) the degree of involvement (3); f) the perceived locus of control (5); and g) the time-on-task (1). Lower scores indicated higher levels of agreement.

The second measure was an achievement test. A second case study (the posttest) measured the skill transfer of strategies developed while completing the online case study. For the posttest, students were required to analyze, in-class, a case study which was similar in nature to the one that they had been required to be completed online. The second case study was completed individually.

*Results.* Two sets of analyses were conducted, the first on all students in the study, and the second on students in the individual condition plus students who were in 'pure' groups. Pure groups were defined as a pair where
both individuals had input on the assignment (N= 36). Students (N=36) were eliminated from the second analyses based on the following criteria: a) if they contacted their partners by means other than email; b) if they did not contact their partners; c) if they did not work on the assignment with their partners: or d) if they strongly agreed with the manipulation check. The results for 'pure' groups and individuals are summarized below.

The results revealed only two significant differences between the individualistic structure and the cooperative pairs structure in attitudes. First, students were less comfortable completing the online assignment cooperatively (M = 2.69) compared to individually (M = 1.95), F (1, 71) = 5.34, p .05. Second, students expressed more frustration completing the online assignment cooperatively (M = 3.23) compared to individually (M = 3.69), F (1, 71) = 4.75, p <05. There was no significant difference between the groups in achievement (p > .05).

There were only two significant interactions of gender and structure. The Gender X Structure interaction effect was significant regarding the amount of effort students extended in completing the online assignment compared to other assignments, F (1, 71) = 5.79, p < .05. In particular, females in the cooperative condition reported working harder (M = 2.71) than females in the individual condition (M = 3.19). There was also a significant interaction effect for perception of course performance, F (1, 71) = 5.16, p < .05. Females in the cooperative condition were less likely to denigrate their performance (M = 3.77) than females in the individualistic condition (M = 3.19).

*Discussion.* Study 1 attempted to simulate a distance education environment, with male and female students who were required to complete an assignment by email either working alone or working cooperatively with a partner. The study found many similarities between the cooperative and individualistic conditions in promoting student attitudes and achievement; differences were limited to students feeling less comfortable and more frustrated working in cooperative pairs. One possible implication of these experimental results is that students will continue to show no strong preference for either individual work or cooperative work online under practical, field conditions of longer duration and greater isolation from the instructor and other students. Furthermore, these data are mildly encouraging for the use of cooperative structures for female students learning online.

A second interpretation of these results considers methodological and design factors such as the duration and strength of the treatment, the fidelity of the treatment, and the quality of the measures. For example, the treatment was brief, the task may not have been challenging enough (all students earned relatively high grades), despite student reports that they had insufficient time to complete it. The assignment was not worth a significant percentage of the grade for the course (3%) which may have affected student motivation. Also, the assignment came at a time when the students were occupied
preparing for midterm examinations. Finally, about half of the students were eliminated from the cooperative pairs condition for reasons ranging from not working with their assigned partners to working with others in ways not prescribed by the experimenters.

O'Malley and Scanlon (1990) conducted a study to determine student preferences for working collaboratively or individually. They found that: a) although students preferred working alone, online collaboration was perceived as helpful; b) appropriateness of group activities may be task dependent; and c) there is value in synchronous online group activities. Furthermore, Wild and Winniford (1993) reported positive effects of CMC when students were given the choice of selecting partners with whom to work.

Consequently, we decided to conduct a second study to explore further the effects of cooperative and individualistic structures on male and female students learning online while taking into account some of the shortcomings of Study 1. Several changes were made. First, to limit differential mortality, students provided informed consent prior to being assigned to experimental conditions. To further strengthen the treatment, the assignment was worth more of the course grade than in Study 1, it was more challenging, and in the cooperative condition, promoted goal interdependence among the pairs. To further encourage interdependence, communication among pairs was synchronized as opposed to the asynchronous communication used in Study 1. The length of the treatment was also doubled. Finally, to explore the effects of familiarity and cohesiveness, the pairs were divided into two groups: those who had worked together previously and those who had not. We expected that the attitudes and achievement of the cooperative pairs in the familiar condition would be higher than the attitudes and achievement of the cooperative pairs in the unfamiliar condition.

## STUDY 2

### *Method*

*Participants.* The sample consisted of two intact classes of male and female undergraduate business students registered for a business strategies course. Each class was taught by the same professor during the Fall, 1993. Of the 107 students who participated, 33 were assigned to work individually and 74 students were assigned to work in pairs. Those assigned to the pairs condition were stratified into pairs who had worked together previously (N=47) and pairs who had not (N=27). For various reasons, 6 students assigned to the pairs condition worked alone and their data were withheld from analysis.
Design. Study 2 was a gender (male, female) by structure (familiar cooperative pairs, unfamiliar cooperative pairs, individuals) between groups factorial design. The unit of analysis was the individual participant.

**Procedure.** Students were initially informed that a study was being conducted which would require them to complete two case study assignments online in the university's computer lab either individually or in pairs. Students were required to complete the assignments as part of the course requirements. However, the students could choose not to have their data used for research purposes.

The students were randomly assigned to either the individual or cooperative pairs condition. Those students assigned to the pairs condition were sent their partner's email address and were told to contact that partner in order to complete the assignments. All students were informed that this exercise was to simulate distance education, therefore, they should not speak directly to their partners or professor all queries were to be sent via email.

Along with the first assignment, students were given an outline of procedures to follow. They were informed that the two assignments would be graded and together worth five percent of their final mark. This grade would be either an individual grade or group grade, depending on their individual or pairs status.

Feedback on the first assignment was sent to each student prior to the second class. The second assignment was completed exactly as the first. That is, students resumed with the same partner they had during the first assignment or they continued working alone. The assignment was completed online, during class time at the university's computer laboratory. Finally, students were administered the attitudinal questionnaire and the posttest during the class following the completion of the second assignment.

**Measures.** The attitudinal measure was a refined version of the questionnaire used in Study 1. Several item stems were eliminated or clarified and some of the response alternatives were slightly modified. In addition, the response anchors were reversed such that a high score indicated increased agreement. The case study posttest was thematically similar to the one used in Study 1 but was more challenging and graded somewhat more stringently. Results: The results revealed no significant achievement effects ($p > 0.05$) and few attitudinal effects attributable to structure. Students in the individual condition reported needing less time to complete the assignments ($M = 1.61$) than students in either the familiar pairs ($M = 2.57$) or unfamiliar pairs conditions ($M = 2.37$), $F(2, 96) = 4.74, p < 0.05$. However, compared to students working individually ($M = 2.79$), students who had the opportunity to work with a partner, whether familiar ($M = 3.70$) or unfamiliar ($M = 3.59$), expressed a greater desire to work with a partner in the future, $F(2, 96) = 3.42, p < 0.05$. Finally, there was a single significant Gender X Structure interaction effect. Female students reported the greatest satisfaction with idea exchange
in the familiar pairs condition (M = 4.59), followed by the unfamiliar pairs condition (M = 4.20), and the least satisfaction with the alone condition (M = 3.57), F (2,96) = 3.46, p < 0.5.

*Discussion.* As in Study 1, Study 2 also found many similarities between the cooperative and individualistic conditions in promoting student attitudes and achievement. The differences found suggested that the work took longer with a partner yet students in the pairs conditions wanted to work with another student again in the future. Furthermore, female students liked the idea exchange possible when working in cooperative pairs.

Methodological and design factors may also have compromised treatment efficacy. The treatment, while longer in Study 2, was still only a fraction of a complete course. The value of the assignments was also greater in Study 2, but still was only a small part of the grade for the course. Students in both studies only simulated distance education; the degree of isolation and separation from teachers and peers was apparent only during the brief treatment phase of one business course. Which is incongruent with the constant separation from teachers and peers that most distance learners experience. Furthermore, these students registered for an in-class course and may have held a set of expectations that may differ from those of “free choice” distance learners, aware that they will forfeit contact with teachers and peers for scheduling flexibility.

Finally, there were unanticipated difficulties with computer use. Some students lacked enough email experience to fully communicate with their partners. A small number did not try.

GENERAL DISCUSSION

The growing importance of education coupled with large worldwide enrolments in courses taught at a distance argue for increasing efforts to develop technological and pedagogical tools to enhance learner success. These developments depend on educational research for their validation.

The two studies reported here attempted to use techniques of field experimentation (e.g., random assignment) to study the effects of cooperative versus individualistic structures on male and female college students. In general, the data suggest few differences between students in their attitudes and achievement due to working cooperatively with a partner or not in a computer mediated environment. The differences which were found suggested that students took longer to complete learning tasks and expressed slightly more frustration when working with a peer. On the plus side, students expressed a desire to work with a partner again, while female students in the cooperative condition were more positive about their performance and the op-
portunity to exchange ideas than those who worked in the individual condition.

There are two interpretations of these findings. The first interpretation focuses on the unique features of distance education. It suggests that the effects of cooperative versus individualistic structures in a simulated distance education environment are smaller than those which occur in traditional classrooms where students have the opportunity to interact face-to-face in real time. For example, the attraction of distance education is that students are free to learn where and when they chose. Using cooperative learning may not support the extent to which these benefits are realized.

There are at least two interrelated features of distance education which may ameliorate the effectiveness of cooperative learning: communication effectiveness and social loafing. Group problem solving depends on the ability of group members to communicate with one another when assistance is necessary. The asynchronous communication in Study 1 may have limited communication effectiveness while the synchronous communication in Study 2 may have slightly facilitated it, explaining those students' preferences for collaborative work in the future. Nevertheless, the time and effort required of written communication may not suit many learning tasks as well as the efficiency and immediacy of oral communication.

In contrast, Carroll (1990) found that students who participated in asynchronous conferences with structured decision support (the use of topic headings or the use of topic headings and moderator support) demonstrated high quality decisions. Carroll also claimed that when CMC is asynchronous, it has the potential to promote communication patterns that are more effective than those that occur face-to-face. However, she recognized that CMC can lose its focus without structure and a moderator. Carroll concluded that the structure in the conference provided students with a means of coordinating their discussion.

One of the attractive features of distance education and an asynchronous environment, is that it allows students to work when it is most convenient for them. Nevertheless, if placing students in cooperatively structured groups is desired, then it may be necessary to ensure that they are on-line at the same time or that they understand that others (their partners) are depending upon them to do their share of the work.

A second distinguishing feature of distance education is the relative anonymity of each partner's learning effort. Research on group performance and productivity suggests that several factors contribute to the tendency for individuals to minimize their efforts on a collaborative task, such as equality of efforts, personal responsibility, and involvement or whether the individual perceives the work as worth it, too costly, or not necessary (Shepperd, 1993). In particular, Harkin (1987) showed that individual efforts decreased when others could not identify individual contributions. If such is the case, coopera-
tive learning structures which promote individual accountability may be es-
sential for effective distance learning among peers.

Further research is necessary comparing cooperative versus individualis-
tic structures in distance learning and traditional classroom settings and
which explores the possible mediating effects of communication effective-
ness and social loafing. However, these problems may be overcome when
fourth generation distance education technology permits visual, oral, and
written communication in real time.

The second interpretation focuses on the design and methodological con-
straints of the studies-the duration and strength of the treatment, the fidelity
of the treatment, and the quality of the measures-which limit their gener-
alizability. Together these limitations also offer certain prescriptions for the
design of future research and the implementation of cooperative learning us-
ing CMC in a distance education environment.

1. Students should be equipped with the appropriate technical skills to use CMC effectively prior to working with peers.

2. The learning task should lend itself to collaboration by requiring input from all members of the group.

3. Sufficient time should be available for meaningful dialogue and for the treatment to "take hold".

4. Students should have the requisite collaborative and communication skills to work effectively with others; this may be facilitated through the use of online teambuilding activities.

5. Students should be motivated to learn cooperatively; this may be accomplished by the use of outcome, means, or interpersonal structures which promote interdependence.

6. Students should believe that learning is important, that working together facilitates performance, and that they are responsible for their own learning and the learning of others. The latter may be accomplished by the use of structures which promote individual accountability.

Finally, more research in needed to determine the cognitive gains made
possible through the use of CMC. For example, does the use of CMC pro-
mote or inhibit deep processing skills? Using an instructional strategy which
promotes deep processing is particularly important in distance education.
Kember (1989) found that distance education students who consistently used
a surface approach were more likely to dropout. That is, those students who
habitually used rote learning were less likely to finish a course by distance
than those who used higher order learning skills. One learning strategy that
can overcome the problems of CMC and promote deep processing skills is
cooperative learning. According to et al (1995) higher order process-
ing skills are required when interacting for a group goal.
REFERENCES

Abrami, P. C., Chambers, B., Poulsen, C., De Simone, C., d'Apollonia, S., & Howden, J. (1995). Classroom connections: *Understanding and using co-operative learning*. Toronto, Ont.: Harcourt Brace.

Bissell, S.A., Coombs, N., Medvedeff, D., & Rogers, S. (1987). *Distance education technology using electronic conferencing*. In Proceedings of the Second Guelph Symposium on Computer Conferencing. Guelph, Ontario: University of Guelph, 3-9.

Blackwood, H. and Trent, C. (1968). *A comparison of the effectiveness of face to face and remote teaching in communicating educational information to adults*. Extension Study 4. 16p, ED028324.

Carroll, B. (1990). *The effectiveness of structured decision support in computer conferences*. Proceedings of the third symposium on computer-mediated conferencing, Guelph, Ontario, Canada. 142-145.

Carrier, G. and Schofield, M. (1991) Student support and computer mediated communication in distance education. *Canadian Journal of Educational Communication, 20*(1), 40-54.

Chen, M. (1986). Gender and computers: The beneficial effects of experience on attitudes. *Journal of Educational Computing Research, 13*(3/4), 265-281.

Cheng, C., Lehman, J. & Armstrong, P. (1991). Comparison of performance and attitude in traditional and computer conferencing class. *The American Journal of Distance Education, 5*(3), 5-1-64.

Davie, L. & Wells, R. (1991). Empowering the learner through computer-mediated communication. *The American Journal of Distance Education, 5*(1), 15-23.

Davies, D. (1988). Computer-supported co-operative learning systems: Interactive group technologies, and open learning. *Programmed Learning and Educational Technology, 25*(3), 205-215.

Dede, C. (1989) "The evolution of distance learning; technology mediated interactive learning." *Journal of Distance Learning on Computing in Education, 22*(3), 1-23.

Haile, P. (1986) *An analysis of computer conferences supporting the distance learner*. Paper presented at the Annual Conference of the American Educational Research Association.

Harasim, L. & Johnson, E. (1986) *Educational applications of computer networks for teachers/trainer in Ontario*. Teacher Trainer Series. OISE.

Harasim, L. (1987). *Computer-mediated cooperation in education: Group learning networks*. Second Symposium on Computer Conferencing and Allied Technologies. University of Guelph: Ontario. pp. 171-186.

Harkins, S.G. (1987). Social loafing and social facilitation. *Journal of Experimental Social Psychology, 23*, 1-18.
Henri, F. (1988). Distance education and computer assisted communication. *Prospects, 18* (1), 85-90.

Hiltz, R. (1988). *Collaborative learning in a virtual classroom: Highlights of findings computerized conferencing and communication centre*. New Jersey Institute of Technology.

Howard, D. (1987). Designing learner feedback in distance education. *The American Journal of Distance Education, 1*(3), 24-40.

Hoyt, D. and Frye, D. (1972). *The effectiveness of telecommunications as an educational delivery system*. Final Report. 73p. ED0703 18.

Kaye, T. (1989a). *Computer-supported collaborative learning in a multi-media distance education environment*. Paper presented at the NATO Advanced Research Workshop, Maratea, Italy, September, 24-28.

Kaye, A. (1989b). CMC and Distance Education. In R. Mason and A. Kaye (Eds). *Mindweave*. Oxford, England: Pergamon Press.

Keegan, D. (1986). *The Foundations of Distance Education*. Croom Helm. Kember, D. (1990). The use of a model to derive intervention which might reduce drop-out from distance education courses. *Higher Education, 20*, 11-24.

Kember, D. (1989). An illustration, with case studies, of a linear process model of drop-out from distance education. *Distance Education, 10*(2), 196-211.

Kirby, D. (1993). Is it time to rid universities of distance education? *Journal of Distance Education, 8*(2), 69-72.

Johnson, D.W., & Johnson, R.T. (1989). *Cooperation and Competition: Theory and research*. Edina, MN: Interaction Book Co.

Mason, R. (1990). *Conferencing for mass distance education*. In *Proceedings of the Third Symposium on Computer Mediated Communication*, 56-64. University of Guelph. Guelph, Ontario.

Mason, R. and Kaye, A. (1989). *Mindweave*. Oxford, England: Pergamon Press.

Naidu, S. (1989-1990). Computer conferencing in distance education. *Bulletin: International Council for Distance Education, 20* (19-24), 39-46.

O'Malley, C. and Scanlon, E. (1990). Computer supported collaborative learning: Problem solving and distance education. *Computers Education, 15*(1-3), 127-136.

Peruniak, G. (1983). Interactive perspectives in distance education: A case study. *Distance Education, 4*(1), 63-79.

Puzzuoli, D. (1970). *A study of teaching university extension classes by tele- lecture*. ERIC DOC. No. ED042961.

Shepperd, J. (1993). Productivity loss in performance groups: A motivation analysis. *Psychological Bulletin, 113*, 67-81.
Schriner, D. (1989). Computer Conferencing and Collaborative Learning: A discourse community at work. *College Composition and Communication*, 40, 472-478.

Slavin, R.E. (1989). Cooperative learning and student achievement: Six theoretical perspectives. In C. Ames and M.L. Maehr (Eds.), *Advances in motivation and achievement*. Greenwich, Conn.: JAI Press, pp. 16 1 - 177.

Sweet, R. (1986). Student dropout in distance education: An application of Tinto's model. *Distance Education, 7*(2), 201-2 13.

Wild, R. & Winniford, M. (1993). Remote collaboration among students using electronic mail. *Computers in Education, 21*(3), 193-203.

AUTHOR NOTE

Reprint requests and inquiries concerning this article should be addressed to: Dr. Philip C. Abrami, Centre for the Study of Classroom Processes, Concordia University, 1455 de Maisonneuve Blvd. W., Montreal, Quebec, Canada H3G 1M8. This research was facilitated by grants awarded to the third author by the Social Sciences and Humanities Research Council, Government of Canada and FCAR, Government of Quebec. An earlier version of this paper was presented at the annual meeting of the Canadian Association of Distance Education, Vancouver, British Columbia, May, 1994.

AUTHORS

Michelle Y. Savard is a Research Assistant in Educational Technology at Concordia University, 1455 de Maisonneuve Blvd. W., Montreal, Quebec H3G 1M8.

Sidney N. Mitchell is a Research Assistant in Educational Technology at Concordia University, 1455 de Maisonneuve Blvd. W., Montreal, Quebec H3G 1M8.

Philip C. Abrami is a Professor, Director of the Centre for the Study of Classroom Processes at Concordia University, 1455 de Maisonneuve Blvd. W., Montreal, Quebec H3G 1M8.

Maria is a Director at the Business Administration Student Advisement Center and a Lecturer at the Syracuse University. She is affiliated with the State University of New York at Oswego.
CALL FOR PAPERS

CJEC SPECIAL
25th ANNIVERSARY ISSUE

Papers are requested on topics related to effective methodology for researching media and technology, instructional technology generally, or the process of instructional design. Papers should adopt either an analytical, historical/retrospective, or a futuristic, "beyond the millennium" viewpoint.

Since papers will be subjected to the editorial review process, it is suggested that they be submitted no later than:

JANURARY 12, 1996

Submit directly to:

David A. Mappin

Division of Technology in Education

3-102 Education North

University of Alberta

Edmonton, Alberta

T6G 2G5
# Successful Electronic Distance Collaboration: The Importance of Social Negotiation

Phil Black

**Abstract:** The emergence of the Internet has allowed us to initiate electronic distance collaborations with all parts of the world. However, our experience and understanding of what factors bring success to such collaborations is limited. This article describes an electronic distance collaboration between students at three universities, as well as the introductory research conducted by this article's author. Results suggest that success in such an academic exercise might be best determined by problems found and corrected. That predicting success for an electronic distance collaboration cannot be reduced to formulae but should instead be considered highly complicated intersubjective communicative interactions.

**Résumé:** L'arrivée de l'Internet nous a permis d'établir des relations de collaboration dans tous les coins du monde. Toutefois, notre expérience et nos connaissances sont limitées quant aux facteurs qui peuvent influencer la réussite de ses relations. Le présent article décrit une relation de collaboration par le biais du courrier électronique entre des étudiants de trois universités. L'article présente aussi les résultats d'une recherche exploratrice faite par l'auteur. Ces résultats suggèrent que la réussite d'un tel exercice académique aurait plus de chance d'être établie si il y avait une remédiation des problèmes identifiés. Prédire la réussite d'une relation de collaboration par le courrier électronique ne peut être réduit à de simple formule. La prédiction d'une réussite de ce type de communication devrait plutôt considérer la complexité de l'interaction intersubjective.

Where the hell are You? Today is Wednesday and we have to turn something in on Friday. haven't heard s-t from you since the very first time you sent me something. You're f-king smoking if you think that I'm going to do it by myself. My teacher said that I should kick your a-s. (From one student to another student in an electronic distance collaboration.)

No matter what I say to students as they enter into electronic distance collaborations, I am always surprised, often unpleasantly. The student who sent the above message was not from my class. Nonetheless, his instructor and I, involving our students in an inter-university electronic distance collaboration, worked closely together in an attempt to teach students to think before
they press the key that sends off electronic-mail messages. Obviously our in-
struction in this regard was somewhat lacking.

For the purposes of this article I am defining Electronic Distance Collabo-
ration, or EDC, as any kind of collaborative effort done over distance, using
computer networks such as the Internet. Though electronic mail (e-mail) is
just one method of communication employing computers and computer net-
works, it is the primary method used in the collaborations I discuss in this ar-
ticle.

After some four years of conducting electronic distance collaborations be-
tween my students and students in other parts of the world, as well as stu-
dents in the same university, I have learned to expect the unexpected. I have
also learned that rather than concentrating on teaching how to be successful
in these electronic distance collaborations, I am further ahead if I approach
these projects as opportunities for the students (and, of course, the instruc-
tors) to experience EDC, complete with successes and failures, and, hope-
fully, for the participants to become better collaborators as social negotiators
in computer mediated communication in general, and specifically is using an
apparently simple medium — electronic mail.

In this article I ask:

1. how success in electronic distance collaborations should be defined

2. what factors are related to success?

In answering these questions I rely upon my recent research which in-
cludes a text analysis done on the collaborative work of my students and
their collaborators at the University of Missouri at Kansas City and the Uni-
versity of Nebraska at Lincoln in the fall of 1994, as well as a survey I con-
ducted of my students of the same period.

THE PROBLEMS INHERENT IN ELECTRONIC DISTANCE
COLLABORATION (EDC)

Since the beginning of the use of language we have amassed significant
common communicative experience. However, even in the best of times
when we share many of the same experiences with our interlocutors we can
never absolutely predict the outcome. Wittgenstein suggests "If language is
to be a means of communication there must be agreement not only in defini-
tions but also (queer as this may sound) in judgments" (Wittgenstein 88e).
Factors such as motivation, intention, mood, experience, and so on, are diffi-
cult to predict. Our communication is at best a guess, or a really, really good
guess. Kent maintains that "we put language to use - language does not use
us” (Kent 16). When we have known someone for years we may feel that the words we use are perfectly clear to both parties when in fact we are not only guessing, but are probably making more incorrect guesses than we realize. On the other hand, when we don’t know that person, we may be more conscious of our guessing and we adjust, or “shift” in the words of Kent, as we predict the other person’s communicative strategy.

In an electronic distance collaboration students often assume that just because they send an electronic mail message that the message is completely clear and will invoke the intended response. Students fail to realize that collaborating with another student face-to-face requires a certain amount of shifting, or what I am calling social negotiation, and that using e-mail to communicate with participants makes the need for social negotiation even more imperative. This is all further complicated when participants, new or experienced, to electronic mail, or any computer mediated communication medium for that matter, tend to apply the wrong experiences to it (Willis 17); that is to say, they are applying experiences of face-to-face communication or letter writing or something else which results in behaviors of respondents that often surprise the initiators (Goode 61). To use Kent's terms, the students are making bad guesses.

Porter argues that we should think of electronic mail “from a broad perspective: as an environment in which a diverse range of writing and research practices can be exercised and studied.” His argument suggests that e-mail should not be considered simply as a medium for writing memos (Porter 41). Too often, the advice we receive for using electronic mail tends to follow the lines of etiquette (Updegrove 37) which, though useful, still tends to ignore the rhetoric of electronic mail (or more broadly, computer mediated communication) and pay attention to more stylistic issues.

I argue here that electronic distance collaboration, specifically where electronic mail is used, requires attendance to social negotiation in a new and different way. As a means of communication electronic mail is distinct and separate from other media, such as print or oral media. I further believe that it is a mistake to assume that theories which apply to face-to-face communication are applicable in an electronic distance collaboration.

## THE RESEARCH

I conducted the research in this project in the hope of locating some direction for future research rather than to gain conclusive evidence. For example, the number of students I use for my statistical population (24) is simply too low to be statistically revealing. However, the research does give me some direction for future research.
## Settings and Participants

In the fall of 1994 Julie Bukovich, a technical writing instructor at the University of Missouri at Kansas City, and I set up an electronic distance collaboration between our students. We brought in a third class, taught by Jerry Parsons, Assoc. Professor at the University of Nebraska in Lincoln. I was teaching Business Communication; Julie and Jerry were both teaching Technical Writing. The plan was to have students at all three campuses jointly write some sort of report. We set up eight groups in each class, with at least two and at most three students from each university in each group. Group One in my class would work with the corresponding Groups One in each of the other two classes, and so on.

We decided that a good topic for students to report on would be native American tribes, specifically the Shawnee, Pawnee, and Iowa. These tribes were generally of the geographic locations corresponding to the three participating universities. Each student group was assigned one of eight categories ranging from religion to art to economics. For example, Group One in my class was to research the religion of the Iowa Indians while Jerry's students researched the religion of the Pawnee tribe and Julie's students worked on the religion of the Shawnee people.

Native American culture worked well for this project because the students really had no knowledge of the native American peoples. We felt that this lack of experience of native American cultures would provide a more even playing field for all. As to computer literacy, the students, and the teachers, were all over the scale. Some students were completely new at electronic mail while others were computer science majors and had extensive experience.

## Research Questions

My goal in conducting this research has been to better understand how success in an electronic distance collaboration can be achieved. Is there a relationship between computer literacy and EDC success? At the time I felt I had some intuitive understanding but I needed some confirmation which would then lead me in a more clear direction. I first needed to decide on a working definition of "success."

# HOW SHOULD SUCCESS IN ELECTRONIC DISTANCE COLLABORATIONS BE DEFINED?

Success in an electronic distance collaboration is not easy to define. Oh, we could say, for example, that success is the generation of a jointly written
proposal. Or one that is accepted. But what about future collaboration? Does the mere fact that the two parties are able to generate jointly written proposal that is accepted suggest that participants will be able to work together in the future?

In our collaboration one particular group experienced significant difficulties and in an attempt to reduce confusion sent the following message to the group members at the other two universities.

Following are some suggestions... for the tri-campus...collaboration:

1. When communicating via e-mail, try not to use personal pronouns (i.e. we, us, you, it, etc.). Attempting to be exact and distinct in what one communicates will hopefully avoid miscommunication and further delays.

2. Rapport has been established; please do not use sarcasm or unnecessary humor when discussing the collaboration project via e-mail. Approaching the collaboration in an unemotional, technical, and business-like manner will aid in reducing wasted time due to miscommunication.

With time running out for this project to be completed, all three campuses need to communicate effectively and efficiently.
Unless UMKC or UNL writes ISU and expresses that UMKC or UNL is dissatisfied with the suggestions that ISU has made, it is to be assumed that the previous announcements and guidelines are acceptable and understood.

Thank you for your cooperation.

This communication caused a great deal of upset among the receiving group members at one university. Someone had originally tried to interject some humor in an e-mail message (as in a face-to-face communication) and when the authors of the above message rejected that humor, the rejection was read over and over and over until the recipients were, to say the least, piqued. In the end, the two groups of students cleared up the mess. (The third group remained strangely silent throughout this particular exchange.)

The students involved in the misunderstanding spent enough time straightening out the confusion that their end product suffered somewhat. Yet I would prefer to have them involved in a future collaboration because they know just how misunderstood things can be. The group making the communication errors, I believe, will be most prepared for future electronic distance collaborations because they were forced to negotiate with their collaborators.
METHOD

Text Analysis of Electronic Mail Exchanges between Students

Students were all asked to send copies to their respective instructors of
any e-mail they sent. In our Ultrix operating system at Iowa State, students
are asked for "carbon copy" addresses before they write the message. I am
guessing that this contributes to students "addressing before writing before
thinking" in that, I received copies of several electronic messages wherein
students actually discussed me with their partners, seemingly oblivious to me
as a reader. For this reason, and because I could follow these electronic dia-
logues throughout the semester, I believe I have a fairly complete set of
mail exchanges. It is possible that students forgot to include me in the carbon
copy section but quite often the e-mail was addressed to a small distribution
list students created for their groups. In those, I was a list member, so when
mail was addressed to the list, I automatically received a copy.

In examining the e-mail exchanges, I first sorted the exchanges by group
then by date, thus enabling me to follow the conversations within each
group. (I had copies of all electronic mail sent from one of the other two uni-
versities and some e-mail from the other university. I used e-mail from the
other universities to help me make sense of the messages my students sent.) I
then looked for two specific instances: misunderstandings followed by clarifi-
cation.

Under misunderstanding I lumped disagreements, arguments, or dialog
with no apparent understanding at all. I also included questions for clarifica-
tion such as, "Where are you? We haven't heard from you in a week!" My
justification for such an approach was that I was not as interested in students
producing an end product as I was in students learning to negotiate electroni-
cally over distance.

Student Analysis Reports

At the end of the semester, and as part of their portfolio, students were
each to write an analysis of what they felt would make a successful collabora-
tion. I specifically asked them to include examples from that semester's col-
laboration in support of their claims. This approach, I believe, allowed stu-
dents to reflect upon the collaboration just conducted and apply their experi-
ences to the future. Students were told that the reports should be based pri-
marily on their insights into electronic distance collaboration. What experi-
ences would best serve them if future employers ask them collaborate elec-
tronically with someone in another office, province, or country? Students
were also told that the reports would weigh heavily in their final grades.
## Instructor's Rating of Success for Each Group Overall

I had hoped that the thoroughness of the students' answers would give me some indication of what the students learned about electronic distance collaboration. Students' insights and the experiences combined with the text analyses of student e-mail exchanges provided sufficient material for me to give a success rating overall for each group's participation in the electronic distance collaboration.

I devised the following scale to take into account both the text analysis of student electronic mail exchanges and the student analysis reports which together might allow me to determine the success of each student group in my class.

Beginning with Unsuccessful, the obvious total failure then in an electronic distance collaboration is having total misunderstanding. Moving toward the Successful end but still on the Unsuccessful half of the scale is the scenario where students have no misunderstanding at all, therefore no clarification is needed.

If there was one misunderstanding which resulted in clarification groups were rated in the middle. More than one misunderstanding/clarification gave me some indication that the participants had not achieved clarification purely by accident. And finally, I have decided that EDCs which included substantial misunderstanding and proportionate clarification were the most successful in terms of the goals I have as an instructor.

<table><caption>Table 1</caption><thead><tr><th colspan="2">Unsuccessful</th><th>Successful</th></tr></thead><tbody><tr><td>Total misunderstanding<br>No clarification<br>No insight</td><td>One misunderstanding<br>One clarification<br>Some insight</td><td>Misunderstanding<br>Clarification<br>Insight</td></tr><tr><td>No misunderstanding<br>No clarification<br>No insight</td><td colspan="2">More misunderstanding<br>More clarification<br>More insight</td></tr></tbody></table>

## WHAT FACTORS ARE RELATED TO SUCCESS?

Gauging the success of an electronic distance collaboration, or to some extent any collaboration, is difficult. And once you think you have seen success it is equally difficult to determine just what caused the EDC to work. Stylistic figures employed, the number of words written and phrases used, or other
structural devices may be insufficient for getting at the elements essential for a successful EDC.

While there could be a myriad of factors which in some way affect the success of electronic distance collaboration, I employed the following questionnaire to answer the question of computer experience.

## Method

### Questionnaire

I asked the students in my class to fill out a questionnaire (see Appendix) at the beginning of the semester. My purpose was to find out if students' familiarity with computers as writing tools affected students' successes in an electronic distance collaboration. My idea was to compare the results of the questionnaires with the groups I had rated as successful in the EDC and see if there existed any correlations. One might easily suppose that students having no prior computer experience might experience greater difficulty in an electronic distance collaboration and I wanted some verification of this myself.

The questionnaire was based on a similar questionnaire written and conducted by Mike Markel, Boise State University. The twenty questions are divided into three unmarked areas: experience in using the computer, using the computer as part of the writing process, and using the computer to replace traditional tools when writing in the classroom. The twenty-four students filled out the questionnaires at the beginning of the semester.

## RESEARCH RESULTS

### HOW SHOULD SUCCESS IN ELECTRONIC DISTANCE COLLABORATIONS BE DEFINED?

### *Text Analysis of Electronic Mail Exchanges between Students*

With an examination of e-mail exchanges I had hoped to spot misunderstandings (or confusions, complete lack of understanding) each group. Surprisingly, I found little which was of great help. You'll recall that students were locating sources of information. Electronic mail conversations rarely included any of the information found or sought; rather, students often wrote "we're getting together tomorrow at the library." Some lengthy exchanges turned out to be descriptions of how to use a distribution list within that group. Sometimes the conversations seemed not to take place at all, that is,
student A would say something, a few days later student B would write back but not respond directly to what student A said. In this sense it was almost as though students were posting announcements rather than interacting. It was, at best, difficult to know if their was a complete lack of understanding or a lack of intent to communicate.

It was my impression that students had no reasons for communicating with their partners, or that they were communicating only because they knew they were supposed to. However, students may have simply used the telephone or a "talk" utility on the computer system, neither possibility having occurred to me. As a result of examining the electronic mail exchanges I was unable to determine the success of the electronic distance collaborations.

In spite of these problems, I was able to identify some instances of misunderstandings. There were also instances of few apparent misunderstandings in the e-mail while the student analysis reports suggested otherwise. There were also a few misunderstandings not clarified via e-mail but I learned later, were clarified by telephone or by using a "talk" utility over the Internet.

## Student Analysis Reports

Students who experienced major difficulties in understanding and in being understood, and who were also able to clear up the confusions, gleaned strong insights and made good cases for approaches to future collaborations in their analyses. The one earlier who advocated humorless writing wrote in his analysis that third person, emotionless writing might communicate well in the medium of the scientific research journal but in an electronic distance collaboration, it may not. It is in the student analyses that I was able to see a relationship between what I considered successful collaborations and students' insights gained through misunderstandings and subsequent clarification.

<table><caption>Table 2</caption><thead><tr><td>Group</td><td colspan="3">Misunderstanding/Clarification</td></tr></thead><tbody><tr><td>1</td><td>1</td><td>/</td><td>1</td></tr><tr><td>2</td><td>0</td><td>/</td><td>0</td></tr><tr><td>3</td><td>5</td><td>/</td><td>5</td></tr><tr><td>4</td><td>2</td><td>/</td><td>0</td></tr><tr><td>5</td><td>3</td><td>/</td><td>1</td></tr><tr><td>6</td><td>2</td><td>/</td><td>2</td></tr><tr><td>7</td><td>2</td><td>/</td><td>0</td></tr><tr><td>8</td><td>0</td><td>/</td><td>0</td></tr></tbody></table>
Instructor's Rating of Success for Each Group Overall

Having a report from each of the three individual members of all eight groups allowed me to compare the experiences and insights of each group to the electronic mail texts and to then assign a rating of each group according to the scale of success. The results are as follows:

<table><caption>Table 3</caption><thead><tr><th>Group</th><th>Success Rating Overall</th></tr></thead><tbody><tr><td>1</td><td>(1 =Most Unsuccessful, 5=Most Successful)</td></tr><tr><td>2</td><td>3 = (one misunderstanding, one clarification]</td></tr><tr><td>3</td><td>2</td></tr><tr><td>4</td><td>5</td></tr><tr><td>5</td><td>1</td></tr><tr><td>6</td><td>4</td></tr><tr><td>7</td><td>1</td></tr><tr><td>8</td><td>2</td></tr></tbody></table>

WHAT FACTORS ARE RELATED TO SUCCESS?

Questionnaire

When comparing the collaborations which I deemed successful to the results of the questionnaire I found absolutely nothing which would suggest relationship exists between EDC success and computer literacy. Students who were successful in the EDC did not necessarily rate highly in the questionnaire, nor did the participants in unsuccessful EDCs necessarily score low on the literacy questionnaire. Combined with the small sample (21 students), I don't believe I can accurately point to a relationship between computer literacy and success in an EDC.

Below are the results of the questionnaire (see Appendix for questionnaire), along with my success rating for each group. The answers to the questionnaire are grouped into the three categories: experience in using the computer, using the computer as part of the writing process, and using the computer to replace traditional tools when writing in the classroom.
The data were analyzed using SPSS-X mainframe software. Inter-item consistency (i.e. reliability) was assessed using Cronbach's alpha. The reliability of the was quite good (=.8510).

<table><caption>Table 4</caption><thead><tr><th></th><th>Computer<br>Attitudes:<br>Prior Experience<br>(Mean)</th><th>Computer<br>Attitudes:<br>Process<br>(Mean)</th><th>Computer<br>Attitudes:<br>Replacement<br>(Mean)</th><th>Instructors<br>Success<br>Rating</th></tr></thead><tbody><tr><td>Group/<br>Student</td><td></td><td></td><td></td><td></td></tr><tr><td>1</td><td>2</td><td>3.3</td><td>2.8</td><td>3</td></tr><tr><td>IA</td><td>1.5</td><td>3</td><td>1.8</td><td></td></tr><tr><td>IB</td><td>1.8</td><td>3.5</td><td>2.7</td><td></td></tr><tr><td>IC</td><td>2.7</td><td>3.5</td><td>4.2</td><td></td></tr><tr><td>2</td><td>1.7</td><td>2.9</td><td>3</td><td>3</td></tr><tr><td>IA</td><td>1.3</td><td>3.5</td><td>3.5</td><td></td></tr><tr><td>IB</td><td>1.8</td><td>2</td><td>1.5</td><td></td></tr><tr><td>IC</td><td>2</td><td>3.3</td><td>4</td><td></td></tr><tr><td>3</td><td>2.7</td><td>3.1</td><td>3.7</td><td>5</td></tr><tr><td>IA</td><td>3.3</td><td>2.9</td><td>3.5</td><td></td></tr><tr><td>/B</td><td>3.7</td><td>3.5</td><td>4.8</td><td></td></tr><tr><td>/C</td><td>1.2</td><td>3</td><td>2.7</td><td></td></tr><tr><td>4</td><td>2.2</td><td>3.5</td><td>4.2</td><td>1</td></tr><tr><td>IA</td><td>2.3</td><td>3.1</td><td>4.3</td><td></td></tr><tr><td>IB</td><td>2.5</td><td>4.4</td><td>3.7</td><td></td></tr><tr><td>IC</td><td>1.7</td><td>3.1</td><td>4.7</td><td></td></tr><tr><td>5</td><td>2.4</td><td>3.4</td><td>3.6</td><td>4</td></tr><tr><td>IA</td><td>3</td><td>3.5</td><td>3.8</td><td></td></tr><tr><td>IB</td><td>2.3</td><td>3.5</td><td>2.8</td><td></td></tr><tr><td>IC</td><td>1.8</td><td>3.1</td><td>4.2</td><td></td></tr><tr><td>6</td><td>2.7</td><td>3</td><td>3.6</td><td>4</td></tr><tr><td>IA</td><td>1.8</td><td>3</td><td>2.7</td><td></td></tr><tr><td>IB</td><td>1.8</td><td>3</td><td>2.7</td><td></td></tr><tr><td>IC</td><td>3</td><td>2.8</td><td>3.7</td><td></td></tr><tr><td>7</td><td>2</td><td>3.4</td><td>3.6</td><td>1</td></tr><tr><td>IA</td><td>2</td><td>3.4</td><td>3.6</td><td></td></tr><tr><td>IB</td><td>2</td><td>3.3</td><td>4.3</td><td></td></tr><tr><td>IC</td><td>1.3</td><td>3.6</td><td>2.5</td><td></td></tr><tr><td>8</td><td>3.2</td><td>3.5</td><td>4.1</td><td>2</td></tr><tr><td>IA</td><td>2.2</td><td>2.6</td><td>3.7</td><td></td></tr><tr><td>IB</td><td>3.5</td><td>4</td><td>4.8</td><td></td></tr><tr><td></td><td>3.8</td><td>3.8</td><td>3.8</td><td></td></tr></tbody></table>
A flaw in the above data is that there were only 24 students used, therefore, I do not feel safe drawing any conclusions. Nonetheless, I did find it interesting that there were no obvious correlations between the success ratings I issued and the group means for computer experience. For future research I think I might be more inclined to test for previous skills in social negotiation than for computer experience.

## THE IMPORTANCE OF SOCIAL NEGOTIATION

It has been my claim throughout this article that students, in order to understand what it takes to conduct a successful electronic distance collaboration, must experience at least limited failure which can then be corrected with an accompanying understanding of what went wrong and how it should be avoided in the future. I now base this claim on the misunderstandings my students experienced in the EDC and upon their newly gained insights.

In "social negotiation," the electronic distance collaboration is similar to any other situation where negotiation takes place. Each party, each negotiator, brings forward theories and experiences which she believes to be appropriate at the moment, based on her assessment of the person with whom she is negotiating (Kent). The difference is that the experiences students, and all of us really, bring forward in computer mediated communication are limited or, if we aren't careful, of the wrong medium.

I do not believe that there is an easy, formulaic approach will somehow predict success. However, recognition of this combined with more experience and further research will make electronic distance collaboration more predictable and, hopefully, more successful.

### Computer Use Questionnaire

The information requested on the following questionnaire will be used for statistical purposes only. Your answers will not affect your grade.
For the following items, please circle the appropriate number.

1. I have had substantial experience word processing on a computer.
   strongly disagree | 2 3 4 5 strongly agree

2. I have had substantial experience using electronic mail (e-mail).
   strongly disagree | 2 3 4 5 strongly agree

3. I have had substantial experience using computers for communication (other than e-mail).
   strongly disagree | 2 3 4 5 strongly agree

4. I have had substantial experience programming a computer.
   strongly disagree | 2 3 4 5 strongly agree
5. I have had substantial experience in data management using a computer. strongly disagree | 2 3 4 5 strongly agree

6. I have had substantial experience playing games on a computer. strongly disagree 1 2 3 4 5 strongly agree

7. When you write a paper or lab report of more than a few pages, how often do you write an outline of some sort on a computer? Never 1 2 3 4 5 always

8. When you write a paper or a lab report of more than a few pages, how often do you write on some sort of paper? Never 1 2 3 4 5 always

9. When you write a paper or a lab report of more than a few pages, how often do you write your initial draft on a computer? Never 1 2 3 4 5 always

10. When you write a paper or a lab report of more than a few pages, how often do you write your initial draft on some sort of paper? Never 1 2 3 4 5 always

11. When you write a paper or lab report of more than a few pages, how often do you revise the document on a computer screen (instead of using a print-out)? Never 1 2 3 4 5 always

12. When you write a paper or lab report of more than a few pages, how often do you revise the document on paper, including a print-out? Never 1 2 3 4 5 always

13. When you write a paper or lab report on a computer, how often do you use a spell-checker program? Never 1 2 3 4 5 always

14. When you write a paper or lab report on a computer, how often do you use a thesaurus? Never 1 2 3 4 5 always

15. I feel that using a computer makes writing easier for me. strongly disagree 1 2 3 4 5 strongly agree

16. I feel that I write more quickly using a computer. strongly disagree 1 2 3 4 5 strongly agree

17. I feel that I write better using a computer. strongly disagree 1 2 3 4 5 strongly agree

18. I am very comfortable using a computer to do an in-class writing assignment. strongly disagree | 2 3 4 5 strongly agree

19. I am very comfortable writing on paper for an in-class writing assignment. strongly disagree 1 2 3 4 5 strongly agree
20. I find it easier to use a computer than writing on paper for an in-class writing assignment.
strongly disagree | 2 3 4 5 strongly agree

REFERENCES

Black, Phil. (1992). "More than a Typewriter." *Computer Assisted English Language Learning*. 3 (2) 2-5.

Bukovich, Julie and Black, Phil. (1994). "E-Mail Wars: Hazards of Long Distance Communication" *Computers and Writing Conference. Columbia*.

Goode, Joanne and Johnson, Maggie. (199 1). "Putting out the Flames: The Etiquette and Law of E-Mail." *Online*. 15(6) 61-65.

Kent, Thomas. 1993. *Paralogic Rhetoric: A Theory of Communicative Interaction*. Cranbury: Associated University Press.

Sutherland, Janet and Black, Phil. (1993). "Finding Common Ground." *Computer Assisted English Language Learning*. 4(2) 6-14.

Sutherland, Janet and Black, Phil. (1993). "Finding Common Ground Part 2." *Computer Assisted English Language Learning*. 4(3) 10-14.

Johnson, Betty S. and Bayless, Marsha L. (1993). "Using E-Mail in Business Communication Classes." *The Bulletin*, LVI(2) 37-39.

Markel, Mike. (1994). "Behaviors, Attitudes, and Outcomes: A Study of Word Processing and Writing Quality Among Experienced Word-Processing Students." *Computers and Composition*. II, 49-58.

Porter, James E. (1993). "E-Mail and Variables of Rhetorical Form." *The Bulletin*, LVI(2) 41-43.

Updegrove, Daniel A. (1991). "Electronic Mail in Education." *Educational Technology*. 31(4) 37-40 (1991). Willis, Jerry. "Computer Mediated Communication Systems and Intellectual Teamwork: Social Psychological Issues in Design and Implementation." *Educational Technology*. 31(4) 37-40.

Wittgenstein, Ludwig. (1958). *Philosophical Investigations*. G. E. M. scomme, Translator. New York: Macmillan.

AUTHOR

Phil Black is a Graduate Student/Teaching Assistant at the Iowa State University.
ACKNOWLEDGEMENTS

The author gratefully acknowledges the contributions of his colleagues in the
electronic distance collaboration: Julie Bukovich at the University of Mis-
souri - Kansas City and Jerry Parsons, Associate Professor at the Univer-
sity of Nebraska Lincoln. Further acknowledgement is given to the col-
laborating students at all three universities.
# The Perceptions and Needs of Faculty in Distance Education Courses in a Conventional University

Margaret Landstrom

**Abstract:** For instructors in a distance education program which employs prepared lessons, the lack of classroom meetings and of regular student-instructor contact sets up a solitary teaching environment which can be quite different from those used to classroom teaching.

In this study, a questionnaire and interview probes faculty perceptions of distance teaching. Further interviews with several faculty who have taught several times at a distance attempt to determine their attitudes regarding distance education and whether their initial perceptions changed as they have become more familiar with this mode of education.

For administrators and faculty in a dual mode institution, the results of this study outline the perception of traditional faculty in a distance teaching situation, changés in their opinions over time, and suggest strategies in course preparation and course management to address the concerns of instructors so that they are more comfortable and effective teaching distance education courses.

**Résumé:** Pour les enseignants oeuvrant dans un programme à distance qui utilisent des leçons préparées, le manque de réunion de classe et de contact entre les enseignants et les étudiants mènent à un environnement d'enseignement solitaire qui peut être très différent de l'enseignement en salle de classe.

Dans cette étude, un questionnaire et des entrevues nous ont permis de recueillir les perceptions des membres de la faculté vis à vis l'enseignement à distance. D'autres entrevues faites auprès des membres de la faculté qui ont enseigné à plusieurs reprises des cours à distance visaient à déterminer leurs attitudes vis à vis l'éducation à distance ainsi que l'évolution de leurs perceptions à mesure qu'ils devenaient plus familier avec cette façon d'enseigner.

Pour les administrateurs et les membres de la faculté ou les deux modes d'enseignement se cotootent, les résultats de cette étude soulignent la perception traditionnelle des membres dans une situation d'enseignement à distance ainsi que l'évolution de leurs perceptions au fil du temps. Des stratégies applicables dans le préparation et l'organisation des cours sont proposées. Ces stratégies viennent répondre aux inquiétudes des enseignants de façon à ce qu'ils deviennent plus confortables et efficaces dans des situations d'éduction à distance.
# THE PERCEPTIONS AND NEEDS OF FACULTY IN DISTANCE EDUCATION COURSES IN A CONVENTIONAL UNIVERSITY

## Introduction

The University of Windsor is a conventional educational institution, in which contact between the student and the instructor is regarded as available during and between classes and is considered important in order for the student to receive guidance on aspects of course content and feedback regarding assignments and tests. Also, contact with the instructor is considered a factor in motivating the student to spend the time and attention needed to complete courses successfully. Recently added to the university offerings are distance education courses in which courses are taught by full time and sessional faculty more familiar with in-class teaching.

As in most dual mode institutions, the distance education program has some detractors among faculty who fear or suspect that the courses are not as rigorous as regular courses, and that student contact will be less likely than in classroom-based programs. Even among those who have written the distance course guides, there is some ambivalence about being involved in the program. There have also been some questions raised about the academic standards possible in distance education generally at faculty councils.

This study is the first attempt to investigate the attitude of faculty participating in the program. This study searches for the factors beyond the basic pay-for-service which are important in making distance teaching rewarding for instructors under the assumption that the more rewarding the teaching environment, the better the learning situation is likely to be for students.

## Research Regarding Faculty Attitudes

Taylor and White (1991) note that the attractiveness for distance teaching centres on the flexibility of scheduling for the instructor. Of concern to instructors in distance education is the quality of the interaction with students, described as the most rewarding feature of teaching. The lack of face-to-face student interaction was found by Blanch (1994) as the most urgent academic concern of distance instructors. Holmberg (1981) notes that communication with the student is a critically important factor for students. However, faculty often feel less than well rewarded for distance education teaching (Dillon, 1989; Siaciwena, 1989). In many dual mode institutions, distance teaching is less prestigious than in-class teaching (Siaciwena, 1989; Stinehart, 1988). With this perception influencing the attitude of traditional faculty involved in distance teaching, it is important to provide a rewarding teaching environment in order for instructors to be motivated in their distance courses.
This study was undertaken to further the research on faculty attitudes in distance education and specifically to evaluate the attitudes and opinions of distance education instructors at the University of Windsor in the Winter semester of 1994.

## Background to the Study

The distance education program at the University of Windsor is ten years old and includes telecourse, correspondence and audiotape enhanced courses in Arts, Social Science, Science, and Business.

Business courses comprise the majority of the distance courses, and most of these are correspondence-type courses. Each course includes a textbook and a detailed course guide prepared to meet the curriculum requirements of the course in the degree program.

At the beginning of the semester, each registered student receives a package of materials including the textbook, course manual and study guide, any audio or video materials and semester specific information regarding assignments, due dates, examination dates, etc. Students are required to submit assignments to the instructor on specific dates. These are marked and returned by mail. Each instructor is required to have telephone access hours (six per week), including at least three evening hours.

Instructors in the distance program are appointed by the relevant department in the academic faculty. Currently, there is an agreement that the course writer will offer the course twice, for debugging and improvement purposes. Full time and sessional instructors are assigned to teach the courses, according to the Faculty contract.

Instructors review the course materials, select assignments and deadlines, prepare and mark the final exam, make themselves available for telephone tutoring, and evaluate materials for future revision.

Distance instructors, like campus instructors, receive little direction in their work. Each distance instructor receives a set of suggestions and regulation information from Continuing Education.

## Research Methodology

Twenty of the twenty three distance instructors for the Winter, 1994 semester were contacted for this study by telephone during the second half of the semester. Sixteen of those contacted were instructing courses which were basically correspondence-type courses, two were telecourses, one was a correspondence course with a set of audiotape lectures, and one was a correspondence course with a weekend workshop. A set of questions was prepared (see appendix) which included an open ended final question allowing instructors to make comments about any aspect of the program or their concerns. While
all were asked the set of questions listed, follow up questions permitted clarification of comments or pursuit of interesting or unusual comments.

## Survey Responses

### Instructor background.

The range of experience with distance education courses varied from those involved in instructing their first D.E. course, to one instructor who had taught over 20 D.E. course offerings. Only four were novices; all others had taught at least one distance education course previously.

Some instructors had been involved in course preparation; others had not. Seven had written the course guide for the course they were teaching. Eleven of the instructors were regular full time faculty at the university; one was a former (retired) faculty member, eight were experienced sessional instructors.

### Instructor/Student contact.

Instructors received information on how the students were doing by their performance on assignments and through student telephone contact. Most of the courses in the accounting/finance areas have printed solutions. In others, this is not appropriate for the assignments, (e.g. Expository writing, Sociology, Geology, Management, Marketing). In all courses, instructors indicate extensive marking is required in order to highlight errors and indicate what information the students are missing, or where they are mistaken. Four instructors mentioned that they often prepared a written commentary based on frequent errors on student assignments. All found the marking in D.E. courses to be extensive and time consuming.

A major part of the discussion with the instructors in the program included their thoughts on what they liked and disliked about classroom teaching and about distance education. By establishing this comparison, it was hoped we could learn how we might enhance the experience of distance instructors, thereby reducing negative criticism of the program.

Without exception, contact with students was mentioned as the major reward for teaching in-class and without exception, the lack of contact and the anonymity of distance students was the major drawback for distance courses from the instructors' point of view. One instructor mentioned that not having student contact prevented the instructor from testing his command of the subject, and the level of his ability to teach effectively. Another missed the opportunity to "play the dynamics of the class" in discussions.

While student contact was extremely important to instructors, seven mentioned that large classes were least desirable about in-class teaching because large classes led to lack of instructor-student contact and anonymity. A detailed comparison of the amount of contact with students in current classes
with the amount of contact with distance education students may indicate if
the difference is real or perceived.

In the distance courses, instructors had particular telephone office hours
for students, but most received few calls from students. The proportion of the
class that called was estimated by most in the 20% - 30% range. One instruc-
tor insisted students call to discuss their major project in the course; in that
class 100% of the students who finished the course called. Of the experi-
enced D.E. instructors, most estimated that of the students who called, they
made contact one to four times during the semester. First time D.E. instruc-
tors were experiencing calls that would result in a similar call pattern as
those who had taught in the program several times. In one course that had fre-
quent assignments, the instructor noted that a small number of students
called several times per week. Three others indicated they had frequent calls
from two or three students in their class.

In all but three classes, it was reported that students called mostly about
logistics in the courses (expectations regarding assignments and exams, re-
quests to submit assignments late, etc.). In the class where the students had to
call the instructor about their assignment, this was the major subject of the
conversation. In the course in which there were frequent assignments, calls
focused on the expectations of the instructor regarding the assignment, and
questions about the grading of previous assignments. In one business course
in which solutions to assignments were *not* provided with the graded returns,
the questions were mainly for clarification of the course material and re-
turned assignment difficulties. From the calls received, all instructors
thought that students had no greater difficulty with the course material than a
typical class of campus students.

In the interview, faculty members were advised that some distance educa-
tion institutions have the instructors call each student early in the course to in-
troduce themselves and encourage the students. When asked if they thought
this was a good idea, ten gave an unqualified "yes" answer indicating that
this might help to encourage the students to call more often, two indicated it
might be a good idea, but personally would not want to do it because of the
time involved. One opposed to instructor initiated calls said it would be hard
to know what to say, while another stated that instructor calls were not a
good idea because the spirit of distance education includes placing the onus
on the students to study at their own pace, and be responsible for initiating
any contact, if needed or desired. Two others said they already sent a friendly
note to students early in the course, encouraging them to call with questions
or problems. Three were unsure of the value of instructor calls.

Attitude toward distance education.

Thirteen instructors remarked that their attitude toward distance education
had not changed during the time they had taught in the program. One
mented that he had been involved in a professional society's distance program for many years as well as the university's, and that there would be more need for distance education, and more technologies used in all instructional programs in the future. Two in their first distance course felt they had too little experience yet to answer this question.

Of the remaining instructors, several commented about the students in distance education, rather than their own activities. Comments regarding students are noted below.

One instructor who was a course writer stated that his perceptions about teaching had changed because of his involvement in distance education because preparing the course guide forced him to view the course through the eyes of the student. He had to make the material accessible without an instructor to explain concepts, and he felt this had given him a way of approaching "underachievers" who do not see themselves coping with a daunting amount of unfamiliar material. He was rewarded that students found the course concepts understandable and relevant, even if they believed previously that these were difficult. One often-mentioned attraction for instructors to distance education courses is that there is no need to appear at class for three hours each week. Instructors appreciated the flexibility of completing the work involved in a distance course at times and in places that suit them.

*Instructor perception of the distance students.*

Fourteen of the instructors believed that the majority of the distance students were more disciplined, more motivated and more mature than campus students. Three felt there was little difference among students, and two noted that they had so little contact, they could not comment on the distance students. Distance students had more complicated lives, noted three instructors, with more personal problems, and barriers to learning such as babies heard in the background while a student was attempting to clarify a course-related matter over the phone.

Two were surprised that some students who never or rarely contacted them did so well in the course. One concluded that it was more difficult for students to pass a distance education course than a regular course because of fewer opportunities for feedback and reinforcement.

*Conclusions*

The major concern of most instructors regarding their experience in the distance education courses is the lack of interaction with students. This is what they find most rewarding about in-class teaching, and most lacking in distance education. Other comments are related to logistical difficulties regarding assignments, most of which could be alleviated if there were better
ways of sending and receiving information, which is another type of contact between the instructors and their students.

In order to increase instructor satisfaction in the courses by assuring more student to instructor feedback, several initiatives will be undertaken:

1. Cost analysis of using a fax for assignment receipt and/or return

2. Establishing e-mail opportunities for distance classes

3. Analysis of 800 number calling system to instructors. This may include a survey of students to determine their willingness to contribute a set fee each term for access to their instructor on an as needs basis for the term

4. Suggesting instructors initiate a written personal introduction inviting student calls

5. Workshops for new and former instructors for the sharing of experiences and techniques for effective interaction with distance students

6. Introduction of a newsletter for students and faculty

7. Dissemination of more information about distance education and relevant research to all faculty

*For Further Study*

1. Analysis of the behaviour of instructors that encourages student contact.

2. Search for additional means of increasing instructor-student contact.

3. Research regarding effective tutoring techniques for distance education instructors.

4. Followup surveys to assess changes in conventional instructor needs and perceptions when they instruct distance education courses.

## REFERENCES

Blanch, Gregory. (1994). Don't all faculty want their own TV show? Barriers to faculty participation in distance education. *DEOSNEWS, 4 (1).*

Clark, T. (1993). Attitudes of higher education faculty toward distance education: A national survey. *The American Journal of Distance Education, 7 (2), 19-33.

Dillon, C. and Walsh, S. (1992). Faculty: The neglected resource in distance education. *The American Journal of Distance Education, 6 (3), 5-21.

Dillon, Connie. (1989). Faculty rewards and instructional telecommunications: A view from the telecourse faculty. *The American Journal of Distance Education, 3* 35-43.

Holmberg, B. (1981). *Status and Trends of Distance Education*. New York: Nichols.
Paul, Ross. (1989). Canada's open universities: Issues and perspectives. In R. Sweet (Ed.), *Post-secondary distance education in Canada, Policies, practices and priorities* (145-160). Edmonton: Athabasca University.
Siaciwena, Richard M.C. (1989). Staff attitudes towards distance education at the University of Zambia. *Journal of Distance Education, IV* (2), 47-62.

Stinehart, Kathleen. (1988). Increasing faculty involvement in distance teaching. In Sewart, David and Daniel, John (Eds.), *Developing Distance Education* (pp. 412-415). Oslo: International Council for Distance Education.

Taylor J. C. and White V.J. (1991). Faculty attitudes towards teaching in the distance education mode: An exploratory investigation. *Research in Distance Education, 3* (3) 7-11. Athabasca: Athabasca University.

Verduin Jr., John R. and Clark, Thomas A. (1991). *Distance education: The foundations of effective practice*, San Francisco: Jossey Bass.

# APPENDIX

## Faculty Perceptions of D.E.

1. Including this, how many times have you taught this D.E. course?

2. Have you taught other D.E. courses?

3. Were you the course writer?

4. Are you full time faculty? sessional? retired?

5. What is the frequency of student calls?
% of class that calls
Of those that call:
#of calls per student from to AVG.:

6. What types of questions do students ask most often?

7. In some D.E. programs, instructors call students early in the semester. Do you think this is advisable? Why or why not?

8. What type of feedback on assignments do you provide?
solutions?
detailed marking of student submission
suggestions for study
other

9. Do a significant number of students indicate they have difficulty understanding the material? (Is this similar to regular courses)

10. What is most satisfying about instructing a D.E. course?
Least?

11. What is most satisfying about teaching an in class course?
Least?
12. If you have taught through D.E. more than once, have your perceptions about this type of teaching changed?

13. From your experience in the D.E. program, how would you compare the students with on campus full or part time students? (such areas as maturity, motivation, intellectual ability?)

14. What do you think would improve the D.E. program:
technology materials contact arrangements other

15. Are there changes in the D.E. program you would suggest to improve your ability to help the students?

16. Are there changes in the course that would improve your satisfaction in being the instructor of the course?

AUTHOR

Margaret Landstrom is the Director of Credit Programs in the Division of Continuing Education at the University of Windsor, 401 Sunset Ave, Windsor, ON. N9B 3P4
Ministry of Education, Culture and Sport

Department of Teacher Education

MOFET INSTITUTE

<figure><img src="image_2.png" /></figure>

משרד החינוך התרבות וה

נף להכשרת עובדי הוראה

מכון מופת

## THE SECOND INTERNATIONAL CONFERENCE
TEACHER EDUCATION: STABILITY, EVOLUTION AND REVOLUTION

Israel, June 30 - July 4, 1996

Under the auspices of the colleges of education in Israel

# CALL FOR PAPERS

## CONFERENCE THEMES

A. SOCIETY, VALUES AND TEACHER EDUCATION

B. KNOWLEDGE CONSTRUCTION, TEACHING AND TEACHER EDUCATION

C. ALTERNATIVE PERSPECTIVES IN THE PROFESSIONAL DEVELOPMENT OF TEACHERS

D. COMPUTER MEDIATED COMMUNICATION (CMC) AND TEACHER EDUCATION

E. PERSPECTIVES FROM TEACHER PRACTICE

F. TEACHER EDUCATORS: A MISSING LINK

## INVITED SPEAKERS (partial list)

David C. Berliner, Arizona State University, USA

Elliot W. Elsner, Stanford University, USA

Judy Shulman, Stanford University, USA

Martin Haberman, University of Wisconsin, USA

Lee Shulman, Stanford University, USA

## FORMATS OF PRESENTATION

Poster sessions (the preferred format); Round-table discussions; Panel discussions; Symposia; Oral presentations.

## LANGUAGES

English and Hebrew. Simultaneous translation will be provided from Hebrew to English.

## IMPORTANT DATES

Deadline for submission of abstracts: November 30, 1995

Notification of acceptance of abstracts: February 29, 1996

## CONFERENCE SECRETARIAT

The Zinman College for Physical Education and Sport, Wingate Institute
Netanya 42902, ISRAEL

Tel.+972 9 639222 (General) +972 9 639291 (Direct); Fax.+972 9 650960

Email <ephraty@wincol.macam98.ac.il>

<figure><img src="image_3.png" /></figure>

Hosted by the Zinman College for Physical Education and Sport, Wingate Institute, Israel
USING A TRANSACTIONIST MODEL IN
EVALUATING DISTANCE EDUCATION
PROGRAMS

Mary F. Kennedy
Blair W. Kettle

Abstract: The purpose of this study was to select and apply an appropriate evalu-
ation methodology for a particular program - the Distance Education for Literacy
Providers (DELP) Course. DELP was a federally and provincially funded pilot pro-
ject designed to assist community-based adult literacy tutors acquire skills and
knowledge to help them in the delivery of literacy programs.

Stake's Responsive Evaluation Mode1 was used as the guiding methodology. It
was chosen because it had been modified for use in similar programs (Lert-
pradist, 1990; Janes, 1993); it is the most widely used of the transactionist ap-
proaches; it offers a flexible, rigorous, and context-sensitive methodology for
audience identification, concerns and issues identification, and standards devel-
opment.

In implementing Stake's mode1 a qualitative approach was used. Data indicated
that the particular transactionist approach was suitable for evaluating this type of
program, and, incidentally, that the DELP course was a success.

Résumé: Cette étude visait à sélectionner et appliquer une méthode d'évalu-
ation appropriée pour une programme déterminé - le cours de Distance Educa-
tion for Literacy Providers (DELP). DELP, un projet pilote, était financé par le gou-
vernement fédérai et provincial. Ce cours était destiné à assister les instructeurs
en alphabétisation dans leur formation afin qu'ils acquièrent des habiletés et des
connaissances utiles dans la livraison des services d'alphabétisation.

Le Stake Responsive Evaluation Mode1 (SREM) servait de guide méthodologique.
Ce modèle d'évaluation fut choisi puisqu'il avait déjà été adapté pour son utili-
sation dans des programmes d'alphabétisation similaires (Lertpradist, 1990;
Janes, 1993) en plus d'être l'approche transactionniste la plus utilisée. SREM offre
une méthodologie qui est à la fois flexible, rigoureuse, et sensible au contexte
pour bien identifier l'audience, les craintes et les problématiques ainsi que le
développement du niveau à atteindre.

Nous avons utilisé une approche qualitative lors de l'implantation du SREM. Les données ont indiqué que l'approche transactionniste était valable pour l'évaluation de ce type de programme, et, en plus, que le cours DELP était une réussite.
INTRODUCTION

The increasing emphasis being placed on community and regionally appro-
priate educational development programs, particularly those being delivered
through distance technology mode, has required the identification of effective
and practical means to determine the merit and worth of such programs. The
Distance Education for Literacy Providers (DELP) course is one such program.

DELP was initiated as a pilot project, utilizing the technical resources of Memorial University of Newfoundland's Telemedicine and Educational Technology Resources Agency (TETRA). The program was to deliver, via distance, an educational development program to adult literacy practitioners in volunteer, community college, and community-based sectors of Newfoundland and Labrador. The principal instructional media used were audio teleconference, video, and print, with video and print materials being mailed to all participants prior to the course.

The thrust of the program was the delivery of a semester-long and non-
credit course through the provincial teleconference network, which would be
accessible to all regardless of geographic location. DELP was intended to serve
a wide audience of literacy providers, including both volunteers - some of
whom had previous training in adult literacy development - and professionals
in the field of adult education, who worked primarily in community colleges.

THE STUDY

This study attempted to find a rigorous and thorough evaluation model that would be consistent with, and supportive of, the spirit and context of the DELP course. One of the desired outcomes of the DELP course was that community-based literacy providers should take ownership of literacy education, and refine and promote it to maximize its usefulness and application. Nevo (1986) describes this as the socio-political function of evaluation. Selection of an evaluation approach, then, was governed in part by its ability to foster the social empowerment engendered by the DELP course - that it be congruent with the socio-political function of that course.

The study was also focused on establishing the efficiency and effectiveness
of the DELP course. Course designers and implementers were obviously inter-
ested in finding an evaluation model that could be applied beyond the pilot
experience, but they were also interested in the outcome of the evaluation - the
merit and worth of the DELP course in its pilot offering. Thus testing of the
evaluation model should also have resulted in comprehensive data to support a
judgement of success or failure of the DELP course.
EVALUATION MODELS

Evaluation has traditionally been given higher priority in distance education settings than in conventional education settings (Alvarado, D'Agostino, and Bolanos, 1991). But what does evaluation of distance education programs consist of? Frequently, in the case of conventional institutions such as Memorial University of Newfoundland, distance educational evaluation activity consists of no more than distribution of a computer-generated and scored evaluation form to be completed by students at the end of a course - as students call these, "happy sheets."

<caption>Figure 1</caption>

A Taxonomy of Major Evaluation Models (House, 1978, p. 12)

<table><caption>A TAXONOMY OF EVALUATION MODELS</caption><thead><tr><th>Model</th><th>Proponents</th><th>Audiences</th><th>Assumes Consensus</th><th>Methodology</th><th>Outcome</th><th>Typical Questions</th></tr></thead><tbody><tr><td>Systems Analysis</td><td>Rivlin</td><td>Economists/ Managers</td><td>Goals, known cause/effect, quantified variables</td><td>PPBS; cost-benefit analysis</td><td>Efficiency</td><td>Are expected effects achieved? What are most efficient programs?</td></tr><tr><td>Behavioral Objectives</td><td>Tyler, Popham</td><td>Managers, Psychologists</td><td>Prespecified objectives, quantified outcome variables</td><td>Objectives, achievement tests</td><td>Accountability, productivity</td><td>Are students meeting objectives?</td></tr><tr><td>Decision-Making</td><td>Stufflebeam, Alkin</td><td>Aministrators/ Managers</td><td>General goals, criteria</td><td>Surveys, questionnaires</td><td>Quality control effectiveness</td><td>Is the program effective?</td></tr><tr><td>Goal-Free</td><td>Scriven</td><td>Consumers</td><td>Consequences, criteria</td><td>Bias control logical analysis</td><td>Consumer choice, social utility</td><td>What are all the effects?</td></tr><tr><td>Art Criticism</td><td>Eisher, Kelly</td><td>Connoiseurs, Consumers</td><td>Criteria, panel procedures</td><td>Critical review</td><td>Improved standards</td><td>Would a critic approve of this program?</td></tr><tr><td>Accreduatuib</td><td>North Central Association</td><td>Teachers, Public</td><td>Procedures, judgement</td><td>Self-study, panel review</td><td>Professional acceptance</td><td>How would professionals rate this program?</td></tr><tr><td>Adversary</td><td>Owens, Levine, Wolf</td><td>Jury</td><td>Negotiations, activities</td><td>Quasi-legal procedures</td><td>Resolution</td><td>What are arguments for and against this program?</td></tr><tr><td>Transaction</td><td>Stake, Parlett-Hamilton</td><td>Clients Practitioners</td><td>Negotiations, activities</td><td>Case studies, interviews, observations</td><td>Undertanding, diversity</td><td>What does the program look like to different people?</td></tr></tbody></table>
Educational program evaluation is much more complex than opinion sheets from a single source - the students. In the past few decades various approaches to program evaluation have been developed by a number of theorists, resulting in a wide range of evaluation models. House (1978) developed a taxonomy of program evaluation models, comparing each model on principal components, major audiences, principal evaluation measures, methodology, typical questions, and outcomes (See Figure 1).

House's ordering of models was particularly useful to the authors, because it provided a rationale for selecting the evaluation approach for the DELP course. House (1978) states:

> In the taxonomy the models are related to one another in a systematic way. Generally, the more one progresses down the column of major audiences, the more democratic or less elitist the audience becomes. The more one moves down the consensus column, the less consensus is assumed on goals and other elements. The more one moves down the methodology column, the more subjective and less objective the research methodology becomes. The more one moves down the outcomes column, the less overall concern becomes social efficiency and the more it becomes personal understanding (p. 5).

## THE TRANSACTIONIST APPROACH

The most democratic of all approaches, and inherently qualitative (House, 1980), the transactionist approach seeks opinions of a broad cross-section of people who have been involved in the program that is being evaluated. It attempts to provide findings which reflect the diversity of audience opinions. Methodologically, reliance is placed on interviews with program audiences, and on-site observation. The approach engages program participants and stakeholders as if they are collaborators in a process which culminates in a judgment about a program.

Among principal proponents of the transactionist approach is Stake's Responsive Model. Evaluators applying the Responsive Model usually "negotiate with the client as to what is to be done... and respond to what different audiences want to know" (House, 1980, p. 40). It is an emergent form of evaluation "that takes as its organizer the concerns and issues of stakeholding audiences" (Guba and Lincoln, 1981, p. 23).
AN EVALUATION MODEL FOR THE DELP PROGRAM

As indicated earlier, Stake's Responsive Model was chosen to evaluate the DELP program. This model, with its extremely democratic thrust and its qualitative approach, was deemed by evaluators to be the most suited to the unique context of the DELP program, and it had been adapted for use in a similar circumstance by Lertpradist (1990).

The Responsive Model has an underlying framework of qualitative research, although implementation of the model does not preclude the collection of quantitative data. Patton (1980) notes that the same assumptions undergird qualitative research and responsive evaluations, including:

> The importance of understanding people and programs in context; a commitment to studying naturally occurring phenomena without introducing external controls or manipulation; and the assumption that understanding emerges most meaningfully from an inductive analysis of open-ended, detailed, descriptive, and quotive data gathered through direct contact with the program and its participants (p. 55).

Stake himself states, "An evaluation is responsive (1) if it orients more directly to programme activities than to programme intents, (2) if it responds to audience requirements for information, and (3) if the different value perspectives present are referred to in reporting the success and failure of the program" (p. 163).

An inherent belief of the responsive approach is that standards and criteria against which the evaluator judges a program should emerge from the concerns and issues of all stakeholding audiences, and that these concerns and issues should be gathered from interviews with persons associated with the program. Standards for a responsive evaluation, then, are whatever the program participants deem to be indicators of success.

In the responsive rubric, evaluations can serve many purposes, but the purpose for any given evaluation is defined by the information needs of all program audiences or groups. It is this relating of purpose and information needs that increases the usefulness of the findings, and hence increases the actual implementation of the recommendations. Where evaluation sponsors or clients desire an evaluation that serves and speaks to the community at large, approaches such as the responsive model are highly favorable. The responsive approach recognizes that some programs, more than others, hold great interest to many individuals and groups within a community, and that any effort to establish the worth of such programs should focus on considering community information needs. This was certainly the case with the DELP program, which purported to train community-based literacy providers.
The Responsive Model would offer program participants, the neighborhood and community literacy providers for whom it was designed, as much input in determining the concerns and issues on which the evaluation would focus as it would any other group, including the evaluation sponsors. It would therefore provide all audience groups with results that would not only demonstrate wide consultation, but would also offer feedback for improving the DELP program to make it suit the practical realities of life, work, and economics for literacy providers in rural communities and urban neighborhoods in Newfoundland and Labrador. In short, the Responsive Model suited better than any other approach the historical, cultural, and emancipative spirit in which the DELP program was conceived and the social context in which it would be delivered.

## EVALUATION PROCEDURES

The evaluation of the DELP course was undertaken using a modified version of Stake's original Responsive Model. Lertpradist (1990) used the Responsive Model in the evaluation of the Community-based Artificial Fish Breeding Training Program in Thailand - a program designed by the Department of Fisheries, Thailand and offered at numerous rural sites by the Extension Service of that department.

In designing the evaluation Lertpradist made modifications to the Responsive Model, as indicated in Figure 2. The modifications mainly consisted of combining certain activities, and of being less specific in delineating procedures before entering the setting. It should be noted that activities described in

<caption>Figure 2</caption>

Comparison of Original Stake Evaluation Events with Lertpradist Modifications

<table><thead><tr><td>STAKES EVENTS</td><td>LERTPRADIST EVENTS</td></tr></thead><tbody><tr><td>Talk with clients, program staff, audiences</td><td>Identify audiences, program scope</td></tr><tr><td>Identify program scope</td><td>Identify concerns, issues</td></tr><tr><td>Overview program activities</td><td>Set standards</td></tr><tr><td>Discover purposes, concerns</td><td>Select/develop instruments and methods</td></tr><tr><td>Conceptualize issues, problems</td><td>Observe program transactions, outcomes</td></tr><tr><td>Identify data needs re issues</td><td>Apply criteria and standards</td></tr><tr><td>Select observers, judges, instruments, if any</td><td>Summarize data, prepare reports</td></tr><tr><td>Observe designated antecedents, transactions, and outcomes</td><td></td></tr><tr><td>Thermatize, prepare portrayals and case studies</td><td></td></tr><tr><td>Validate, confirm, attempt to disconfirm</td><td></td></tr><tr><td>Winnow; format for audience use</td><td></td></tr><tr><td>Assemble formal reports, if any</td><td></td></tr></tbody></table>
the Responsive Model are usually placed in the form of a clock face, so that they can be read and followed in a clockwise, counter-clockwise, or cross-clockwise fashion.

## AUDIENCE IDENTIFICATION

Preliminary interviews with program developers and coordinators identified all audience group that had a stake in the evaluation. They were categorized in three groups (See Figure 3), and all audience members were contacted through either face-to-face interviews, telephone interviews, or short questionnaires. The purpose of this initial contact was to gather the concerns and issues of all participants in and around the program.

<caption>Figure 3</caption>

*Audience Groups Consulted in Determining Concerns and Issues*

<table><tr><td>Curriculum Committee</td><td>Program developers<br>Curriculum advisors<br>Program administrators<br>Program instructors</td></tr><tr><td>Course Participants</td><td>Volunteers (no training)<br>Volunteers (short course training)<br>Professionals (degrees)<br>Professionals (degrees + short course training)</td></tr><tr><td>Course Sponsors</td><td>Provincial government representatives<br>Federal government representatives<br>Literacy agency representatives</td></tr></table>

The initial contact with all audiences yielded over thirty concerns and issues, which, on analysis, could be categorized in seven general areas as follows: knowledge gains for both groups of literacy tutors - the untrained volunteers and the extensively trained literacy personnel; positive attitude gains for both groups; program versatility in terms of other delivery modes; relevance of program for intended audience; efficacy of distance delivery mode; efficacy of combination of media used; cost implications of teleconference approach.

## EVALUATION STANDARDS AND CRITERIA

The concerns and issues, plus an analysis of all program documents (including original proposals for funding) provided evaluators with the basis to develop a comprehensive set of standards and criteria (See Figure 4). These standards were ratified by all audiences as valid and acceptable measures for evaluators to employ in the process of making judgements. It should be noted that not all standards could be measured during the implementation of the
DELP program. Several standards would require follow-up evaluation activity with participants.

<caption>Figure 4</caption>

DELP Evaluation Standards and Criteria

<table><tr><td>Standard Criteria:</td><td>Curriculum meets participants' needs<br/>Provide participants with increased knowledge<br/>Meet expectations of various participant groups<br/>Can achieve stated program goals and objectives<br/>Participants want to complete program</td></tr><tr><td>Standard Criteria:</td><td>Objectives are clearly deined<br/>Stated in writing in course materials<br/>Meet expectations of various audiences</td></tr><tr><td>Standard Criteria:</td><td>Participants able to apply knowledge to tutoring practice<br/>Future literacy tutoring assimilates new techniques<br/>Participants can verbalize how they can use new knowledge/skills</td></tr><tr><td>Standard Criteria:</td><td>Knowledge beneficial to tutors and low-literacy learners<br/>Low-literate learners attest to positive change in literacy tutoring</td></tr><tr><td>Standard Criteria:</td><td>Goals and objectives feasible and achievable<br/>All goals/objectives met six months after program ends</td></tr><tr><td>Standard Criteria:</td><td>Sufficient opportunity in program for participant interaction<br/>Regular time scheduled for discussion weekly<br/>Activities encourage interaction/sharing<br/>Teleconference leaders promote interactoin<br/>Opportunity for participation deemed adequate by participants</td></tr><tr><td>Standard Criteria:</td><td>Instructional content suits participants' prior knowledge levels<br/>Participants attest to suitability of curriculum content and method or presentation</td></tr><tr><td>Standard Criteria:</td><td>Media combination used suited to content and participants<br/>Text materials modularized to match weekly teleconference sessions<br/>Text materials deemed easy to read and attractive in format<br/>Videotapes interesting and informative, and relevant to course modules<br/>Teleconference sites accessible<br/>Frequency of teleconferences deemed suitable by participants<br/>Length of teleconferences deemed suitable by participants<br/>Participants comfortable with delivery systems</td></tr><tr><td>Standard Criteria:</td><td>Course suited for future delivery through other modes<br/>Course suited for total packaging i.e., audiotapes<br/>Participants able to implement parts of course for other literacy tutors<br/>All support materials complete and self-instructional</td></tr></table>

DATA COLLECTION TECHNIQUES

The DELP program was offered through teleconference one night weekly over a ten week period to fifteen sites around the province. Observations were conducted by evaluators each week, and for one week evaluators visited three different sites so that they could assess conditions at the more remote sites. Observations were conducted with the aid of observation forms and checklists, to ensure that data relevant to the standards and criteria were collected.

A number of instruments were also used throughout the evaluation period. These consisted of an open-response questionnaire to elicit concerns and
issues, a semi-structured questionnaire for literacy tutors enroled in the pro-
gram, an interview guide to allow indepth exploration of a sample number of
literacy tutors for case profiles, an attitude scale, and a post-program semi-
structured interview guide administered by telephone six months after the
DELP program had ended. In addition to the observational data, the data from
interviews, written questionnaires and scales, the evaluators collected and ana-
lysed data from all documents and records generated in the design and imple-
mentation of the DELP program.

AN ANALYSIS OF THE RESPONSIVE EXPERIENCE

The evaluators analysed a number of evaluation models that were exem-
plary of the eight major evaluation frameworks as delineated by House (1978).
They chose, as most fitting for the evaluation of the DELP program, the Stake
'Responsive Model. Following the adaptation made by Lertpradist (1990) they
applied the model to the implementation of the program.

The Responsive Model was chosen in the expectation that (a) its emergent
and naturalistic approach would allow evaluators to be flexible and sensitive to
programs where the social setting or context plays an integral role; (b) its
democratic stance would ensure that the information needs of all groups in and
around the program would be considered; and (c) results would provide mean-
ingful information to a diverse group of people.

The Responsive Model proved to be applicable to any small to medium
community-based adult education program. The approach was designed to em-
phasize evaluation issues that are important to all program participants. The
consultative process of setting standards and criteria, based on the concerns and
issues expressed by all of the diverse program audiences, ensured that the data
summarized and reported on addressed the information needs of all.

Democracy and participative management are important considerations in
the evaluation of programs which are intended to support community-based
economic, educational, and social development activities. The DELP program
was intended to provide volunteer and professional literacy tutors with the
skills and tools with which to reduce the level of adult literacy within their
communities. The Responsive Model provided those individuals who were
closest to the front lines of the literacy problem with a sense that they were full
and significant players in the direction of their programs. It gave the literacy
providers a sense of control and ownership of their unique problems, and ulti-
mate resolutions. In sum, it encouraged, recognized, and respected self-deter-
mination by placing value on and responding to the needs of the audiences of
the DELP program.

The Responsive Model gave evaluators the opportunity for prolonged inter-
action with and exposure to the DELP training program. Evaluators observed
the whole program as implemented over a ten week period in four different sites. Such prolonged interaction gave them a true picture of the program, and dissipated the possibility of events as observed being isolated occurrences.

The Responsive Model provided a surfeit of data gleaned from a variety of techniques. Rich data, according to Guba and Lincoln (1981) are one of the major advantages of Stake's model. Evaluators estimated that they gathered much more data than that needed to minimally address each evaluation standard. However, the duplication of data served the purposes of grounding the study and triangulating findings. Data gathered through one technique or source were compared and contrasted with that from other sources, establishing consistency and credibility.

The Responsive Model, with its emphasis on detailed description of all program components as opposed to emphasis solely on program outcomes, proved to be valuable to program administrators. In most cases where program evaluation is implemented, the purpose is not to determine the continuance or cancellation of the program, but to seek means of improving it. Evaluations that rely heavily on description provide program administrators with the detailed data on program strengths and weaknesses, and pinpoint those areas in need of improvement.

Most of all, the Responsive Model did ensure that all of the data needed to weigh against the standards and their criteria were collected. At the end of the evaluation period, evaluators were able to state, and support with summarized data, that every criterion of every standard had either been attained or had not been attained.

## CONCLUSIONS

The application of the Responsive Model to the DELP course was a gratifying experience for evaluators and for all program audiences. However, certain caveats should be considered.

1. The evaluators selected one model for application. No other models were tried in the DELP course setting, hence it is possible that other evaluation approaches would have yielded good data and strong results. However, the evaluators do believe that only the transactionist approach - in particular the Responsive Model - could have provided all participant groups with an equal voice in the evaluation, and the empowerment that accompanies such recognition.

2. The evaluation context was more ideal than is usually the case. The DELP program was planned and created by a group of instructional developers, who were cognizant of the benefits of a comprehensive evaluation. Hence they consulted evaluators prior to program implementation so that evaluation plans could be made well in advance. They also provided as much time as the implementation of the approach required,
including time for follow-up six months later. Lack of time to imple-
ment any evaluation model well is the norm in evaluation contracts, in
these authors' experience.

3. The program administrators (paying clients) permitted the evaluators to select whichever approach that they deemed most suitable, and were willing to accept as standards indicative of success concerns and issues of other participant groups. They were also willing to accept the kinds of data and information that the Responsive Model usually produces - very qualitative data reported in the participants' own language.

It is the experience of the authors, who have completed numerous evalu-
ations using a variety of models and approaches, that the DELP program was
an ideal evaluation context. Maybe the circumstances, as much as the Respon-
sive Model itself, were responsible for the success of the evaluation. Incidentally,
the DELP course itself proved to be very successful, achieving all evaluation
standards.

REFERENCES

Alvarado, J., D' Agostino, G., & Bolanos, G. (1991). Orientation and conceptualization of the academic quality control centre. In R. Schuemer (Ed.), *Evaluation concepts and practices in selected distance education institutions*. FernUniversitat, Hagen, West Germany. (ERIC Document Reproduction No. 329 784).

Berk, R.A. & Rossi, P.H. (1991). *Thinking about program evaluation*. New-bury Park, CA: Sage.

Guba, E.G. & Lincoln, Y.S. (1981). *Effective evaluation*. San Francisco: Jossey-Bass.

House, E.R. (1978). Assumptions underlying evaluation models. *Educational Researcher*, 7,4-12.

House, E.R. (1980). *Evaluating with validity* Beverly Hills: Sage.

Lertpradist, P. (1990). *A study of the application of a selected evaluation methodology in an extension setting*. Unpublished Master's thesis, Memorial University of Newfoundland, St. John's, Newfoundland.

Nevo, D. (1986). Conceptualization of educational evaluation: An analytical review of the literature. In E.R. House (Ed.), *New directions in educational evaluation*. London: Falmer Press.

Patton, M.Q. (1985). Culture and evaluation. (Vol. 25). New directions for program evaluation. San Francisco: Jossey-Bass.

Patton, M.Q. (1990). *Qualitative evaluation and research methods*. Newbury Park: Sage.

Striven, M. (1972). Pros and cons about goal-free evaluation. *Evaluation Comment*, 3(4), 1-7.
Stake, R.E. (1975). To evaluate an arts program. In R.E. Stake (Ed.), *Evaluating the arts in education*. Columbus, OH: Charles E. Merrill.

Stufflebeam, D.L. & Webster, W.J. (1983). An analysis of alternative approaches to evaluation. In G.F. Madaus, M. Scriven, & D. Stufflebeam (Eds.), *Evaluation models: Viewpoints on educational and human services evaluation* (pp. 23-43). Boston: Kluwer-Nihoff.

Worthen, B.R. & Sanders, J.R. (1973). *Educational evaluation: Theory and practice*. Worthington, OH: Charles A. Jones.

## AUTHORS

Mary F. Kennedy is Associate Professor, Faculty of Education, Memorial University of Newfoundland, St. John's, Newfoundland.

Blair W. Kettle is an Educational Technology Consultant and holds an M.Ed. in Educational Technology. He is a graduate of Memorial University of Newfoundland, St. John's, Newfoundland.
MEDIAWARE REVIEW

A Critical Examination of Two Resources for Creating
HTML Documents for the World Wide Web

L. F. (Len) Proctor, Editor

HTML Sourcebook: A Complete Guide to HTML by Ian S. Graham. Published by John Wiley & Sons, Toronto, 1995, 416 pages, $41.95.

HTML Web Weaver by Robert Best (Robert.Best@Potsdam.edu). For the latest release and pricing information you can contact:
http://www.potsdam.edu/Web.Weaver/About.htm

Reviewed by Richard Schwier

This review examines two resources currently available for creating
HTML documents for the World Wide Web, not because they are unusual,
but rather because they are good examples of a rapidly growing pool of simi-
lar resources, and both point in the direction these types of resources are tak-
ing. One is a book; one is a text editor. One is Canadian; one is inexpensive.
Both are comparatively good resources.

The Canadian book, by Ian Graham from the Instructional and Research Computing Group at the University of Toronto, addresses a very real and growing need. Most anyone trying to write documents for the World Wide Web realizes that although the initial commands are easy to learn and use, difficulty lurks just beneath the surface. I have hungered for a book about creating HTML documents that is easy to read and full of examples, and this book goes a long way in the right direction.

The book appears to be thorough, at least it went well beyond what I
was looking for in terms of content. Its primary focus is on the HyperText
Markup Language (HTML), but it also considers URLs (Uniform Resource
Locators), HTTP (HyperText Transfer Protocol) and CGI (Common Gate-
way Interface). In all, it is a comprehensive treatment of things you will need
to consider to prepare resources and mount them on the World Wide Web. In
fact, it includes much more than we commoners require to produce web docu-
ments.

Permit me to define a couple of terms so the uninitiated can still in-
terpret most of this review. HTML is the major language used to create docu-
ments for the World Wide Web. It allows anyone to create documents on
most any platform, and make them available to most everyone else. When
you view documents on the World Wide Web, you use a program called a
browser. There are many different browsers, but some common ones are Mo-
saic, Netscape and Lynx. These programs can read HTML documents (and
other things) on the web.

First of all, for anyone who has not yet encountered the bittersweet
characteristics of HTML, it is a **language** for formatting text on a
screen. It is a *MIME* content type EM and it employs embedded com-
mands in the text to orchestrate the presentation of elements.
For instance, this paragraph, if saved as an HTML document, would
be interpreted by my Netscape browser as:

In the first three chapters, Graham introduces the reader to HTML
and some technical design issues. Chapter 1 gives general rules and ideas for
constructing HTML documents, and Chapter 2 wades in with more detail.
Chapter 3 explains how to construct and use URLs, the method used in
HTML documents to address documents and make hypertext links. Graham
is careful to emphasize that design is critical, and he quite properly nags the
reader to use proper HTML constructions and syntax when creating HTML
documents. In just my first reading, I picked up a handful of ideas for cor-
recting mistakes I'd inadvertently made in files. For instance, Graham in-
forms those writing HTML documents that HTML recognizes only the larg-
est spacing value of a logical structure element, What? This means that if
one uses several paragraph tags `<P>` to create some space between items on a
page, only the first will be invoked, and the rest ignored. Thus,
 `<P><P><P><P><P>` renders the same spacing on the page as `<P>`. Graham
also reminds the reader that logical highlighting elements should be used
wherever possible so that various users can set the preferences of their
browsers to impose the styles they want on the page. For example, page de-
signers can be tempted to bold-face a word or phrase by placing the element
between `<B>` and `<B>` in the text. This will force the browser to display the
text as bold, and it is called a physical highlighting element. Alternatively,
the page designer can use `<STRONG>` to identify the highlighted text, and
the browser will use whatever convention the user prefers to display text
with strong emphasis. I might set my browser to display as red italics rather
than as bold faced text.

This is precisely the point at which instructional designers flinch.
Give the user control over document design? Has the world gone mad?
With HTML documents, users are given enormous control over the "look" of
documents they view, and a designer who wishes to wrest that control away
from the user must go to extreme lengths. Thus, the point needs to be made
that HTML is not, first and foremost, an instructional design tool-nor is it
an authoring language for CBI. It is simply a protocol for structuring informa-
tion and providing Hypertext links within and between documents. As a de-
sign tool, it is not as powerful as many other programs, and as an authoring
language, it pales next to most any other available program. But it is a won-
derful tool for structuring information, it is truly a cross-platform tool, and
everyone (well almost everyone) is using it.

And this is where Graham's book shines. It acknowledges, but ig-
nores, the platform debate and focuses directly on using the tool properly.
The book is filled with examples, and I found this to be one of the best fea-
tures of the book. In most of the HTML references I have seen (most down-
loaded from web sites), I am left with the impression that I joined half of a
conversation in progress. While the references are useful, there seem to be
missing instructions or assumptions about the reader's knowledge. Graham
clears up much of the confusion with clear examples of HTML construc-
tions, and he wisely follows them with illustrations of how the constructions
might be rendered by different browsers. Let me publicly say, "Thank you,
Ian Graham."

There are a host of other good tips and reminders. At the same time,
the reader must remember to take some of the advice with a grain of salt. For
example, Graham admonishes:

"You also don't want the link [highlighted text] to be gratuitously long. It is much more effective to link a single work or selection of words than to a whole sentence." (p.29)

Now, while I would agree with the intention of the statement, and I
would probably try to follow the advice, there is no evidence I know which
suggests that words and phrases provide more effective links than sentences
for the user. There may be times when complete sentences or even para-
graphs might be effectively linked to another document, although I can think
of few cases where it would be advisable. Similarly,

"Although you can build hypertext links within a document to other points
inside the same document, this is generally more difficult to navigate
than a collection of smaller files." (p.44)
I'm not sure what exactly is meant by "more difficult to navigate" but links appear the same regardless of the document structure, so I doubt he means that one is easier to use than another. Perhaps Graham is suggesting that movement within a large document is slower than between documents, and certainly it is true that larger documents can take a long time to transfer from the host to your machine. It is also consistent with his advice to keep documents and image file sizes small. But this is not consistent with my own experience when remote access is initiated each time a new tile is requested from a very busy site. Performance can be sluggish, and sometimes a request is even refused by the host. In these cases, I might prefer links within a document I've already accessed.

These are not severe criticisms of Graham's advice, but rather a reminder to instructional designers who use this book that it emphasizes how to write effective HTML documents-not effective instructional documents, and some of the advice may be at odds with good instructional design.

In short, the first three chapters of this book made it worth having, and I recommend it for anyone wanting to create HTML documents. Turning to the rest of the book, Chapter 4 discusses the HTTP protocol for delivering hypertext materials and the Common Gateway Interface (CGI) specification which defines the way a server communicates with gateway programs. This is very illuminating and important information if you are setting up your own server, or have occasion to communicate with people operating servers at a web site.

Chapter 5 discusses HTML and CGI tools, and it is particularly useful if you want to do anything beyond basic HTML document development. For example, we have been struggling with how to use a single graphic in our designs and make various sections of the graphic "hot" to establish links with other documents. Sounds simple, doesn't it? Well, it is a fairly convoluted process which requires additional tools and some technical expertise. This and other operations require special utility programs and CGI programs to create and execute the commands, and it is one of the places where mere mortals get a little weak in the knees. Graham's style is very inviting in this chapter, but the information is still somewhat daunting for most readers.

Chapter 6 briefly describes several editors available for PC, Macintosh and UNIX platforms, complete with ordering information. It also lists a number of document translation packages which allow you to create documents with a word processor and then either translate them or save them as HTML documents. This chapter is helpful, but it will certainly suffer the ravages of a changing marketplace very quickly. There are many new editors available since the publication of this book, and there will most certainly be many more available by the time this review is published. One thing we can look forward to is the inclusion of HTML as a "save as" option in all major word processing packages. Extensions are already available for many of
them. A similar fate awaits Chapter 7, which reviews several of the different browsers available for exploring the web. The web seems to be solidifying, or at least curdling, around one or two browsers. In fact, the wildly popular “Netscape” is mentioned only as a coming attraction, demonstrating just how volatile information in this field can be. Still, Graham’s discussion of cross-browser design contains good advice that will certainly transcend immediate changes in the marketplace.

Chapter 8 describes HTTP servers and server utilities, and it is important reading for anyone setting up a web site. It considers some of the complications one can encounter, and does a credible job of comparing some of the advantages of some platforms over others as servers.

Chapter 9 is a clear departure from the rest of the book, and it is one of the book's most engaging sections. Graham chose a handful of exemplary web sites and asked the creators of the sites to write descriptions of the important issues and design challenges they faced. This chapter displays some of the diverse and creative issues one faces in site development, and I found the descriptions to be accessible and illustrative.

Overall, I found this work to be among the best I have yet encountered. Certainly this is a mushrooming area, and there will no doubt be other excellent resources available in short order. I found that much of the technical information in the middle chapters to be either beyond my knowledge level, or well beyond the scope of my interest. I'm confident, however, that others will find these chapters more useful than the ones I singled out for praise. Regardless, this is a good reference, and I think Graham can keep very busy in the next several years writing new editions.

## An HTML Document Editor

As outlined in Graham's book, a large and growing number of editors are available. Think of these editors as word processors specifically designed for creating HTML documents. Editors are vitally important tools for creating HTML documents, because they remove some of the drudgery and complexity from the process, thus making it much easier for individuals who have little acquaintance with programming protocols to create documents. Generally speaking the role of an HTML editor is to take text and add the HTML markup tags and instructions.

HTML Web Weaver by Robert Best is a good example of such editors. It is designed for Macintosh computers and it employs a number of icons and three windows of items the document author can use to mark text with HTML commands (see figure I).
<caption>Figure 1</caption>

Main document construction page from HTML Web Weaver, including menu buttons and command windows.

<figure><img src="image_4.png" /></figure>

To give an idea of how *HTML Web Weaver* can be used, a few simple examples will be used. First, text can be inserted, constructed, pasted or imported into the document. Once on the screen, text is selected and then the desired HTML function selected from one of the windows. For this example, the text "document page" is selected and the HTML command for emphasizing the phrase is chosen (see Figure 2).

<caption>Figure 2</caption>

Selection of text and associated emphasis command from HTML Web Weaver windows.

<figure><img src="image_5.png" /></figure>