CANADIAN JOURNAL OF EDUCATIONAL COMMUNICATION

Volume 24, Number 1, Spring 1995

ISSN 0710-4340

The Mythology of Colour in Multimedia Screen Design: Art, Science and Connoisseurship

Earl R. Misanchuk

Richard A. Schwier

Learning Strategies for Interactive Multimedia Instruction: Applying Linear and Spatial Notetaking

Elleen E. Schroeder

Richard F. Kenny

Engagement as a Design Concept for Multimedia

Richard Jacques
Tom Carey

Jenny Preece

Using Video-on-Demand for Educational Purposes: Observations from a Three Month Experiment

Pierre C. Bélanger

Sébastien Clément

An AMTEC Publication
EDITORIAL BOARD

Farough Abed
Central Connecticut State University

Cheryl Amundsen
McGill University

Robert C. Branch
Syracuse University

Robert M. Bernard
Concordia University

Katy Campbell-Bonar
State University of New York

Bruce Clark
University of Calgary

Don Coldewoy
Athabasca University

Thomas M. Duffy
Indiana University

Rodney S. Earle
Brigham Young University

D. J. Engel
University of Alberta

George L. Geis
OISE

Edward S. Halpern
AT&T Bell Laboratories

Michael Hannafin
Florida State University

Denis Hlynko
University of Manitoba

W. J. Hunter
University of Calgary

Robert Jones
Sheridan College of Applied
Arts and Technology

Lorne Koroluk
University of British Columbia

Richard Kenny
University of Ottawa

James J. LoFollette
University of Alberta

Richard F. Lewis
University of Windsor

David A. Mappin
University of Alberta

Earl R. Misanchuk
University of Saskatchewan

Ronald Owston
York University

Beverley A. Park

Avalon Consolidated School Ed. of
St. John's

Lauran Sandals
University of Calgary

Louise Salme
Tele-Universite

Richard F. Schmid
Concordia University

R. J. Schmidt
Strathcona County Board of
Education

Richard A. Schwier
University of Saskatchewan

Steven Show
Concordia University

Mariela Tovar
Concordia University

Rolland Viau
Univiersite de Sherbrooke

Clayton R. Wright
Grant MacEwon Community
College

AMTEC BOARD OF
DIRECTORS

President
Allen LeBlanc
Sault Ste. Marie School Board

Part President
Ross Mutton
Carleton University

President Elect

Secretary / Treasurer
Lillian Carefoot
School District #68, Nonaimo

Director
Bob Christie
Christie Communication

Director
Danielle Fortosky
Univiersite of Saskatchewan

Membership
Genevieve Gallant
Cabot College
# Canadian Journal of Educational Communication

Volume 24, Number 1
Spring 1995

## Editor

Mary Kennedy

## Guest Editor

David A. Mappin

## MEDIAWARE REVIEW EDITOR

Len F. Proctor

## BOOK REVIEW EDITOR

Diane P. Janes

## Translation

Michele Gour, Montreal

## Production Manager

Mary Genova, WRITEWORKS

ISSN 0710-4340

The Canadian *Journal* of Educational Communication is published by the Association for Media and Technology in *Education in Canada: 3-1750 The Queensway*, Suite 1318, Etobicoke, Ontario M9C 5H5; Attention: Ms. Mary Anne Epp, Secretary/Treasurer. *Notification of address change should be sent to the above. All articles are copyright by AMJEC and may be reproduced for non-profit use without permission provided credit is given to CJEC. Book issues of CJEC are $16.05 Canadian and may be obtained by contacting the Editor. CJEC is indexed in the Canadian Education Index and ERIC.*

Second Class Mail Registration No. 6956

Special Issue on Multimedia Development

## ARTICLES

The Mythology of Colour in Multimedia Screen Design: Art, Science, and Connoisseurship
Earl R. Misanchuk
Richard A. Schwier

Learning Strategies for Interactive Multimedia Instruction: Applying Linear and Spatial Notetaking.
Eileen E. Schroeder
Richard F. Kenny

Engagement as a Design Concept for Multimedia
Richard Jacques
Jenny Preece
Tom Carey

Using Video-On-Demand For Educational Purposes: Observations from a Three Month Experiment
Pierre C. Belanger
Sebastien Clement

MEDIAWARE REVIEW 85

BOOK REVIEWS 89

All correspondence should be addressed to:

DR. MARY F. KENNEDY
Editor — CJEC
Faculty of Education
Memorial University of Newfoundland
St. John's, Newfoundland
A1B 3X8
Membership Form

AMTEC c/o Genevieve Gallant
3-1750 The Queensway, Suite 1818
Etobicoke, Ontario, Canada M9C 5H5

name

title

address

city

prov/postal code

phone (w)

phone (h)

Fax

Netmail

<figure><img src="image_1.png" /></figure>

I am willing to have pertinent information from the above included in the next Members' Handbook

ALL RENEWALS COME DUE ON EITHER JANUARY 1ST OR JUNE 1ST

If this is a renewal then please indicate your Membership Number

<figure><img src="image_2.png" /></figure>

Student of Educational Technology (includes GST) $32.10

<figure><img src="image_3.png" /></figure>

Retiree (includes GST) $32.10

<figure><img src="image_4.png" /></figure>

Individual (includes GST) $80.25

<figure><img src="image_5.png" /></figure>

Commercial (includes GST) $80.25

<figure><img src="image_6.png" /></figure>

Subscription Only - Canada $42.80

<figure><img src="image_7.png" /></figure>

Subscription Only - USA $40.00 U.S.

<figure><img src="image_8.png" /></figure>

Subscription Only - Outside North America $55.00 U.S.

*Students must have this form signed by your Department Head:

GST # 122939853

Special Interest Areas:

___ Media Production

___ Media Management

___ Telecommunications

___ School Libraries

___ Distance Education

___ Media Literacy

___ Other

___ Micromputer Applications

___ Instructional Development

___ Mass Communications

___ Educ. Tech. Research

___ Curriculum Implementation

___ Interactive Video

Work Environment

___ Elementary

___ Secondary

___ College

___ University

___ Other

___ Business

___ Government

___ Health Care

Please duplicate this form.

CJEC is typeset on an Apple Macintosh Plus™ in PageMaker 3.0™. Galley proofs to authors and final camera-ready impressions are output on a Qume CrystalPrint™ Publisher.

Concordia University Printing Services

Acknowledgement

The Canadian Journal of Educational Communication is supported by a grant from the Social Sciences and Humanities Research Council of Canada.
# The Mythology of Colour in Multimedia Screen Design: Art, Science, and Connoisseurship

Earl R. Misanchuk

Richard A. Schwier

**Abstract:** The effective use of colour in designing computer screen displays is both an art and a science. The literature reflects this duality, bringing together advice based on experience, tradition, and opinion with that based on empirical research. As generalizations - whether based on empirical research or on practical experience - are handed down from researcher or practitioner to novice, "truths" which may no longer be valid inadvertently get promulgated. Indeed, the literature appears to orbit a collection of common guidelines, a core of undefended advice based on little evidence. This article summarizes the results of an exhaustive search of the literature on the use of colour in screen design for instructional purposes. Critical analysis of the papers contributing to the knowledge base reveals that major problems exist with if, leaving precious little valid guidance to instructional designers, while appearing to offer a considerable amounf. Renewed research, and the development of a connoisseurial approach to screen design which values the contributions of both research and aesthetic experience, are proffered as useful approaches.

**Résumé:** L'utilisation efficace de la couleur dans la conception de l'affichage sur écran d'ordinateur est à la fois *un arf* et *une science*. La documentation écrite sur le sujet réflète cette dualité en réunissant conseils et opinions basés respectivement sur l'expérience et *la* tradition, et sur la recherche empirique. Qu'elles soient basées sur l'expérience ou sur la recherche empirique, ce sont des <<généralisations= qui sont transmises aux néophytes par les chercheurs et les praticiens. Ces <<vérités>>, qui ne sont probablement plus valides, sont perpétuées comme par inadvertance. En effet, *la* documentation semble orbiter autour d'un ensemble de *lignes* directrices ou de *conseils non défendus* basés sur des évidences à peine prouvées. Cet article fait le point sur les conclusions tirées d'une recherche approfondie de la documentation écrite sur l'utilisation de la couleur dans la conception d'écrans pour fin d'enseignement. Une analyse critique des articles qui constituent la base de la connaissance sur le sujet révèle que *cell-ci* recèle de grands problemes. En effet, bien qu'il existe un grand nombre de lignes directrices, les concepteurs pédagogiques ne peuvent s'y rapporter parce qu'elles sont peu fiables. Une recherche renouvelée et le développement d'une approche avertie, basées sur la contribution des chercheurs et sur l'expérience esthétique, sont privilégiés comme approches valides dans la conception de l'affichage sur écran d'ordinateur.
The effective use of colour in designing computer screen displays is both an art and a science. It is an art in two senses of the word: it is a "... system of rules or principles governing a particular human activity" (Hanks, 1986, p.83); and a collection of wisdom, based on experience, handed down from one practitioner to another. It is also an aesthetic undertaking, inasmuch as attractive screen displays are a goal. There is no shortage of advice and opinion on how colour should be used in screen design; a sizable literature exists which is composed largely or exclusively of experiential summaries.

There are also two kinds of science involved in designing colour screen displays: there exists a well-established knowledge base on the psychophysical aspects of colour, and there is also a considerable collection of empirical research on the use of colour on video display terminals (VDTs) and computer screens, and a number of summaries of this research. To avoid possible confusion between these two "branches" of science, we will refer to the latter as the "science" of using colour in designing screen displays.

This article looks at the "art"- some rules of thumb that advise on how to avoid the garish atrocities that are sometimes produced by neophytes, and the "science" — generalizations gleaned from empirical studies of screen design. To lay the groundwork for a closer look at the art and the science, we first undertake a quick review of the psychophysical factors, of which we have not done an extensive review as part of this article. Psychophysical processes relating to colour are well-understood, and the literature on them is well-developed, but relatively distinct from our purpose here.

## PSYCHOPHYSICAL FACTORS

To briefly review some rather well-established concepts, there are four main characteristics of colour, each of which has importance in considering the use of colour in screen design: hue, brightness, saturation, and contrast (Adkins & Pease, 1991; Durrett & Trezona, 1982; England, 1984; Faiola, 1990; Faiola & DeBloois, 1988; Murch, 1988; Tufte, 1992). Hue is what we generally identify as colour, for example, red, blue, or mauve. Brightness (also known as luminance or value) is the intensity of light reaching the retina. A higher intensity is generally perceived as brighter, although individuals confronted with different colours at the same level of intensity will often perceive one as brighter than the other. Saturation (or chroma) is the interaction of hue and brightness, and is often referred to as the depth or richness of a colour. Saturation of a colour can be diminished by adding white; pastel colours are de-saturated. Finally, contrast is the relative perceived brightness of two colours on a display, and it is related to the notion of figure-ground in visuals. Contrasting colours are easy to separate visually. Another factor which can interact with the four characteristics mentioned above is the amount and quality of ambient light -the natural or artificial light in the setting.
Specialized cells in the retina of the eye called cones provide initial colour sensation through the stimulation of photopigments (light-sensitive chemicals). Most cones are concentrated in the centre of the retina, so that is the area of the eye most sensitive to colour, while little more than grays are perceived at the periphery of vision. Each cone contains one of three different types of photopigments, and each is sensitive to different wavelengths (hues) of light. Type 1 (blue photopigment) is insensitive to wavelengths longer than about 520 nm, so it responds exclusively to wavelengths in the blue to violet range. Type 2 (green photopigment) responds to everything, but is maximally sensitive to 535 nm. Type 3 (red photopigment) responds to everything, but is maximally sensitive to 575 nm. Colour is determined by an interaction among the three photopigments; the perceived colour is a mixture of the relative responses of the red, green, and blue photopigments, in much the same way as a television camera creates colour. Given a dramatic imbalance among the percentages of cells containing red (approximately 64%), green (approximately 32%), and blue (approximately 2%) photopigments, it is clear that the perception of colour is both highly specialized and physiologically biased (data from Murch, 1984).

## CATEGORIES OF ARTICLES REVIEWED

There have been a number of recent major reviews of the literature pertaining to the use of colour that can likely be generalized to multimedia screen design. Some deal with colour more or less exclusively (e.g., Brockmann, 1991; Christ, 1975; Davidoff, 1987; Holcomb, 1991; Horton, 1991; Murch, 1987; Winn, 1991) while others deal with colour on screens in passing, as part of a review of a related topic (e.g., Gillingham, 1988; Hathaway, 1984; Isaacs, 1987; Mills & Weldon, 1987; Sawyer, 1985; Shaw, 1991; Tullis, 1983). In addition, there have been numerous reviews which are either mostly focused on topics other than the use of colour, or less comprehensive in nature (e.g., Chapman, 1993; Milheim & Lavix, 1992; van Nes, 1986). Finally, there are frequently brief reviews of related literature associated with empirical studies in related areas (e.g., Anglin & Towers, 1993; Baek & Layne, 1988; Baker, Belland, & Cambre, 1985, 1986; Bruce & Foster, 1982; Clausing & Schmitt, 1989, 1990; D'Angelo, 1991; Hativa & Teper, 1988; Kerr, 1987; Livingston, 1991; McDonald, Molander, & Noel, 1988; Ohlsson, Nilsson, & Ronnberg, 1981; Pace, 1984; Pastoor, 1990; Radl, 1980; Simmers, 1988; Tullis, 1981; Wright & Lickorish, 1988).

Table 1 classifies articles on the basis of whether they are primarily:

* summaries of empirical research (i.e., they do not include new empirical data);

* empirical studies (i.e., they do include new empirical data) that may include brief but not necessarily comprehensive reviews of related
literature; or

non-empirical in nature (i.e., generalizations gleaned from experiential or theoretical propositions).

<table><caption>**TABLE 1**
*Articles Dealing with Co/our Grouped According to Type of Information Included*</caption><thead><tr><th>Primary Article Type</th><th>Author(s)</th></tr></thead><tbody><tr><td rowspan="12">Largely or Exclusively Summaries of Empirical Research</td><td>Brockmann, 1991</td></tr><tr><td>Chapman, 1993</td></tr><tr><td>Christ, 1975</td></tr><tr><td>Davidoff, 1987</td></tr><tr><td>Gillingham, 1988</td></tr><tr><td>Hathaway, 1984</td></tr><tr><td>Horton, 1991</td></tr><tr><td>Isaacs, 1987</td></tr><tr><td>Murch, 1987</td></tr><tr><td>Rice, 1991</td></tr><tr><td>Sawyer, 1985</td></tr><tr><td>Shaw, 1991</td></tr><tr><td>Tullis, 1983</td></tr><tr><td>Winn, 1991</td></tr><tr><td rowspan="16">Largely or Exclusively Empirical Research Studies</td><td>Anglin & Towers, 1993</td></tr><tr><td>Baek & Layne, 1988</td></tr><tr><td>Baker, Belland, & Cambre, 1985*</td></tr><tr><td>Baker, Belland, & Cambre, 1986*</td></tr><tr><td>Bruce & Foster, 1982</td></tr><tr><td>Clausing & Schmitt, 1989</td></tr><tr><td>Clausing & Schmitt, 1990</td></tr><tr><td>D'Angelo, 1991</td></tr><tr><td>Hativa & Teper, 1988</td></tr><tr><td>Holcomb, 1991</td></tr><tr><td>Kerr, 1987</td></tr><tr><td>Livingston, 1991</td></tr><tr><td>McDonald, Molander, & Noel, 1988</td></tr><tr><td>Ohlsson, Nilsson, & Ronnberg, 1981</td></tr><tr><td>Pace, 1984</td></tr><tr><td>Pastoor, 1990</td></tr><tr><td>Radl, 1980</td></tr><tr><td>Simmers, 1988</td></tr><tr><td>Tullis, 1981</td></tr><tr><td>Wright & Lickorish, 1988</td></tr></tbody></table>
<caption>TABLE 1 (continued)</caption>

Articles Dealing with Colour Grouped According to Type of Information Included

<table><thead><tr><th>Primary Article Type</th><th>Author(s)</th></tr></thead><tbody><tr><td rowspan="24">Largely or Exclusively Non-Empirical Summaries</td><td>Adkins & Pease, 1991</td></tr><tr><td>Baecker & Buxton, 1987b</td></tr><tr><td>Baker, 1983</td></tr><tr><td>Brou, Sciascia, Linden, & Lettvin, 1986</td></tr><tr><td>Collery, 1985</td></tr><tr><td>Durrett & Trezona, 1982</td></tr><tr><td>England, 1984</td></tr><tr><td>Faiola, 1990</td></tr><tr><td>Faiola & DeBloois, 1988</td></tr><tr><td>Galitz, 1989</td></tr><tr><td>Garner, 1991</td></tr><tr><td>Heines, 1984</td></tr><tr><td>Milheim & Lavix, 1992</td></tr><tr><td>Murch, 1984</td></tr><tr><td>Olson & Wilson, 1985</td></tr><tr><td>Rambally & Rambally, 1987</td></tr><tr><td>Reilly & Roach, 1986</td></tr><tr><td>Shneiderman, 1992</td></tr><tr><td>Steinberg, 1991</td></tr><tr><td>Thorell & Smith, 1990</td></tr><tr><td>Tufte, 1990</td></tr><tr><td>Tufte, 1992</td></tr><tr><td>van Nes, 1986</td></tr><tr><td>Waller, Lefrere, & Macdonald-Ross, 1982</td></tr></tbody></table>

Instructional designers naturally turn to recent review articles in order to keep themselves abreast of the most current thinking on how to use technology most effectively. However, the advice they get there may not be the best possible. While we have no desire to impugn the scholarship of the authors of the articles mentioned above, or others, we wish to point out some major problems in most summaries of the literature dealing with aspects of multimedia, using the case of colour in screen design as a case in point. Through the normal practice of exemplary scholarship, myths and legends have crept into our knowledge base on screen design. Authors cite previous authors' works, but in their efforts to be comprehensive sometimes report outdated or

<footnote>*These two papers appear to report the results of the same study.</footnote>
only marginally related literature which then tends to become part of main-
stream advice and generalization (much as a legend gets handed from one
generation to the next). Thus a certain amount of current instructional design
practice, as it is applied to screen design, may actually be based on myth. This
phenomenon appears in both the "art" and the "science" literatures.

WHY COLOUR?

What does colour accomplish? Writers have attributed much to colour in
instructional materials. Colour may promote deep processing of important
information, aid in organizing lesson content, allow reasonable learner-
control options, promote interaction between the learner and lesson content,
and facilitate lesson navigation (Hannafin & Hooper, 1989). In some cases
"color stirs the heart.. . and other vital organs" by influencing everything from
blood pressure and endocrine functions to brain wave patterns and strength
(Horton, 1991, p. 161).

In multimedia screen design, it has been claimed, colour can be used to
link logically-related data; differentiate between required and optional data;
highlight student errors; separate various screen areas such as prompts,
commands, or input/output fields; emphasize key points; and communicate
overall structure (Horton, 1991; Milheim & Lavix, 1992; Rambally &
Rambally, 1987; Strickland & Poe, 1989). While this list is not exhaustive, it
illustrates the wide range of roles colour may play in instructional materials,
and probably hints at the reason for so much confusion surrounding its use.
The effective use of colour is tied intimately to the role it is intended to play
in a particular instructional situation-it is difficult to talk about using colour
out of the context it serves. What may be an effective use of colour for title
screens may be wholly inappropriate in information screens, visual data
bases, or testing contexts. For example, while claims are often made for the
motivational value of colour in instructional materials, the findings on
motivation are by themselves not strong enough arguments for using colour
(Brockmann, 1991; Waller, Lefrere, & MacDonald-Ross, 1982).

THE ART

As well understood as the physiology of colour is, it provides little
explanation for our opinions of colour and colour combinations. At the very
least, opinions of colour are learned and highly associative. For example, as
children we often had a "favorite colour" and we liked everything-clothes,
toys, books- that matched our preference. Over time, we learned a variety
of colour schemes, and in most cases our tastes became more refined. But even
as adults, we are influenced by fashion, and may still associate our more
sophisticated sense of colour with increasingly more sophisticated emotions,
desires, or impressions. For example, even a cursory examination of changes
in interior design from the 1950s to the present reveals a dramatic evolution
of what were considered warm or even comfortable colour combinations. A
lively debate still rages about the psychology of colour, and various claims are
made for using colour in the environment to stimulate, calm, or enhance the
performance of individuals.

There is a significant literature of opinion growing around screen design,
and there is a large tradition drawn from earlier media which is being
generalized, sometimes indiscriminately, to newer media. How does experi-
ential information or advice have value? One often feels set adrift without
guidance in the design of multimedia, and it is comforting to have guidelines
— any guidelines — to justify decisions. A convergence of opinion is a good
place to start when facts are scarce. At the same time, one must always be
careful not to confuse a collective opinion with fact; such guidelines may in fact
conspire to constrict creativity and promote the development of products
which, over time, become hackneyed and trite. What makes us uneasy about
depending primarily on experiential knowledge is that we often don't know
how (or even by whom) certain generalizations were derived, what kinds of
tasks they were derived from, and, particularly, when they were derived (as
that often is related to the state of the art of display equipment). With these
cautions in mind, this paper examines some advice available in non-empirical
literature.

What advice is available about using colour in screen displays? There is
considerable consensus on guidelines, although there are a few significant
contradictions. One cannot easily judge from the material whether the
consensus is based on independently derived judgements, or primarily on a
shared and confined literature. Indeed, some of the more substantial guide-
lines emerge from the tradition of print (see, for example, Horton, 1991;
Waller, Lefrere, & MacDonald-Ross, 1982).

We want to emphasize that the authority for the guidelines offered below
is variable, and the comments offered after each set of principles are intended
to summarize the main consensus of opinion rather than to endorse one
explanation or approach. With these cautions in mind, guidelines and
principles were extracted from the non-empirical literature and categorized.
No attempt has been made to validate the guidelines; the purpose of the
categorization is to locate points where opinions converge and diverge. Advice
is broken into the categories of amount of colour, consistency, choice of colour,
and coding and cueing with colour. Principles or guidelines within each
category are listed along with sources of the advice. A HyperCard(TM) stack
(Misanchuk & Schwier, 1995) which illustrates a number of these principles,
and permits viewers to arrive at their own conclusions about various pieces of
advice, is available from the authors. (Electronic mail requests are preferred,
addressed to Earl.Misanchuk@USask.Ca)
## Amount of Colour

Use colour conservatively: limit the number and amount of colours used (Brockmann, 1991; Durrett & Trezona, 1992; Garner, 1991; Horton, 1991; Shneiderman, 1982).

Limit the palette per screen to what the eye can actually keep track of at one glance (usually about six colours, depending on the complexity of the screen design). (Bailey & Milheim, 1991; Baker, 1983; Faiola, 1990; Faiola & DeBloois, 1988; Hoekema, 1983; Milheim & Lavix, 1992; van Nes, 1986).

Design first for monochrome displays, and then add colour (Brockmann, 1991; Garner, 1991; Shneiderman, 1992).

Long term users are capable of perceiving and responding to a broader range of colour and coding relationships, so the number of colours used can increase with experience (Faiola, 1990; Faiola & DeBloois, 1988).

Use colours selectively to manipulate attention. Colour can be used to highlight text or graphics to make them conspicuous (Durrett & Trezona, 1992; Garner, 1991; van Nes, 1986).

"Material presented in colour is generally processed faster than the same material presented in black-and-white." (Durrett & Trezona, 1992, p. 16).

Use colour to help in formatting (Shneiderman, 1992).

Use colour in graphic displays for greater information density (Shneiderman, 1992).

Electronically generated colours take on different properties in relation to each other (England, 1984).

Wavelength affects colour differentiation: luminance affects legibility (England, 1984; Murch, 1984).

Changes in brightness seem to cause changes in hue for all colours except blue (470 nm), green (505 nm), and yellow (572 nm). These should be used where colour shifting due to luminance changes would be detrimental (Horton, 1991).

As viewers age, higher levels of brightness are needed to distinguish colours.

As evidenced by the above principles, designers are less captivated by colour than one might anticipate. The most resonant advice among writers is to limit the amount of colour to what is useful or necessary, depending on the purpose of the product being developed. The notion of designing for monochrome first, and then adding colour, seems to offer a practical method of harnessing an indiscriminate use of colour. Another striking feature of the advice is the role played by luminance in the portrayal of colour. An instructional designer must consider more than hue when designing screens for legibility, contrast, and constancy.
## Consistency

Be consistent in colour choices (Brockmann, 1991; Faiola, 1990; Faiola & DeBloois, 1988; Milheim & Lavix, 1992).

Carefully select colours for all visual devices such as touch screens, buttons, menus, and titles (Faiola, 1990; Faiola & DeBloois, 1988).

If colour coding is used in an information system, it should be used consistently (van Nes, 1986).

As with all uses of colour, consistency is crucial when using colour for coding information. (Durrett & Trezona, 1982; Shneiderman, 1992).

Using colour consistently may sound like obvious advice, but it is advice often ignored. Consistency is a hallmark of good instructional design; if items are consistent throughout instruction, then the learner can devote more energy to dealing with the content of a presentation than to learning (and re-learning) the conventions of the delivery system.

## Choice of Colour

In selecting colour combinations, make sure they are compatible (avoid saturated complementary colours such as blue/orange, red/green, violet/yellow) (Bailey & Milheim, 1991; Brockmann, 1991; Durrett & Trezona, 1982; Faiola, 1990; Faiola & DeBloois, 1988; Milheim & Lavix, 1992). Murch (1984) qualified this advice. He argued that opponent colours, especially desaturated colours, can go well together for simple colour displays.

Gray is a versatile colour (Tufte, 1992). Use gray in inactive screen areas and backgrounds to enhance two or three other colours (Bailey & Milheim, 1991; Faiola, 1990; Faiola & DeBloois, 1988; Milheim & Lavix, 1992; Tufte, 1992).

Avoid background colours too high in brightness and saturation (Bailey & Milheim, 1991; Faiola, 1990; Faiola & DeBloois, 1988; Horton, 1991; Milheim & Lavix, 1992).

Against gray backgrounds use light, highly saturated borders for active windows. One suggestion is that yellow is the only colour satisfying this requirement (Tufte, 1992), while others argue that one should always use red, white, or yellow text on black (Durrett & Trezona, 1982). Regardless, attend more closely to brightness than hue for building contrast for legibility (Brockmann, 1991; Faiola, 1990; Faiola & DeBloois, 1988; Horton, 1991).

Similarly, use high colour contrast for character/background pairs. Incorporate shape as well as colour when possible to make the system usable for those with colour-deficient vision (Bailey & Milheim, 1991; Garner, 1991; England, 1984; Milheim & Lavix, 1992; Tufte, 1992).

Dark text on a bright background is more legible than the reverse (van Nes, 1986).
Avoid using red and green at the edges of screens if you want people to notice those elements. People are less sensitive to red/green at the periphery of vision. If you must use them, make items blink before resorting to continuous display, to attract attention (Durrett & Trezona, 1982).

Don't use blue for text — "limit blue to large nonfoveal areas" (Durrett & Trezona, 1982, p. 14), but use it as a background colour to enhance depth perception (Horton, 1991).

Avoid using pure blue for text, thin lines, and small shapes. Individuals have difficulty focusing on blue (Horton, 1991; Murch, 1984).

Strong colours should not be used over large adjacent areas. Use strong colours sparingly between dull background tones (Horton, 1991; Tufte, 1992).

When a quick response is necessary, use colours with higher degrees of saturation (Faiola, 1990; Faiola & DeBloois, 1988).

Use colours found in nature, particularly toward the lighter side: grays, blues, yellows. These colours are widely considered harmonious. (Tufte, 1992):

For users with colour-deficient vision, use dramatic changes in colour to discriminate among elements by making changes in at least two of the three main colours. For example, displays in which only the red pigment is changed, while blue and green remain constant, will cause problems for these users (Murch, 1984).

The literature of advice on colour choice goes dramatically beyond issues of aesthetics. Many of the recommendations are based on the physiological response of the eye. For example, the advice on how to use blue is based largely on the inability of the eye to focus clearly on blue images. Thus, the general advice is to relegate blue to a supporting role in screen design. There are, however, aesthetic concerns that complement the physiological explanations. Many writers expressed displeasure with garish colour combinations, and one went so far as to call for the harmonious colours found in nature. Regardless of whether the justification is scientific or aesthetic, the general consensus of opinion is to avoid using highly saturated, bright colours for text, large areas, and backgrounds; or adjacent to other strong colours from the extremes of the colour spectrum. Contrast should be built by carefully using colour on muted or subtly-coloured backgrounds.

## Coding/Cueing with Colour

* Colour can assist learning if used as a redundant cue (Durrett & Trezona, 1982).

* Colour coding can link logically related data; differentiate between required and optional data; highlight errors; and separate prompts, commands, and other elements in the interface (Adkins & Pease, 1991; Rambally & Rambally, 1987).
Use commonplace denotations (red = danger, yellow = yield) (Adkins & Pease, 1991; Bailey & Milheim, 1991; Brockmann, 1991; Durrett & Trezona, 1982; Faiola, 1990; Faiola & DeBloois, 1988; Horton, 1991; Milheim & Lavix, 1992; Rambally & Rambally, 1987). Care must be taken to ensure that denotations are indeed shared, as some are culturally determined, such as the colours of political parties. Similarly, resultant "cultural connotations" may emerge, such as red denoting socialism, in turn connoting revolution (Wailer, Lefrere, & MacDonald-Ross, 1982). These denotations and connotations may not be shared by different cultures.

Choose distinctive hue, brightness, and saturation differences for discrimination among major items. Poor colour memory may be overcome by carefully using colour to enhance discrimination (Faiola, 1990; Faiola & DeBloois, 1988).

Ensure colour coding supports the task (Shneiderman, 1992).

Have colour coding appear with minimal user effort (Shneiderman, 1992).

Place colour coding under user control (Shneiderman, 1992).

Be alert to common expectations about colour codes (Shneiderman, 1992).

Use colour changes to indicate status changes (Shneiderman, 1992).

All of the advice about using colour for coding information emphasizes one simple principle: colour is one dimension of communication, and to exploit it well requires us to use it consciously and deliberately. Colour can be used to accomplish a wide array of instructional tasks, but it can also can also interfere with communication if it is used thoughtlessly or clumsily.

## THE “SCIENCE”

There is a large and diverse body of empirical literature on the use of colour in screen design, but not all of it deals with instructional treatments. The review of empirical literature that follows eschews a considerable body of research on the use of colour on VDTs where the particular use seems quite different than would be found in instructional situations (e.g., air traffic monitoring, airline arrival/departure schedules, pilot/driver navigation systems, on-line job aids). This poses some risk, of course, as sometimes instruction and training approximate the situations we have chosen to ignore. Nevertheless, the danger of over-generalization seems greater than that of under-generalization, and we have chosen to present only the most generic conclusions.

Furthermore, we have reviewed here only that body of literature that claims to have specific relevance to VDTs. This leaves out a great body of research on the use of colour in instruction, conducted on media other than
VDTs (e.g., Dwyer, 1972,1978,1987; Fleming & Levie, 1993). We do not mean to imply that these signal works do not have relevance for multimedia screen displays, but that because that research was done on different media, we cannot be sure that generalizability is automatic (a point which is addressed later in the article).

Careful perusal of the articles listed in Table 1 which are empirical research studies or summaries of empirical research reveals three fundamental problems with the information base:

Some of the advice that is promulgated from article to article may be obsolete, in the sense that the generalizations were formulated using equipment that has been superseded technologically.

The nature of the task used in the research is not sufficiently similar to tasks typically performed during teaching and learning.

The generalizations being passed on are either apocryphal in origin or else have been based on empirical results from different display media and transferred to VDTs on the assumption that, say, whatever was found to be an effect of colour when paper was the medium of display would automatically transfer to VDTs.

## Possibly Obsolete Advice

A shortcoming of many of the published summaries of research on the use of colour in screen design is related to the rapid advance of technology. Generalizations based on studies employing obsolete equipment tend to be included in summaries of research alongside contemporary ones, although they may no longer be valid as a result of technological advances. For example, we examined the lists of references attached to the articles in Table 1, and counted the number of times certain articles were cited. The two most often cited articles are in one case more than a decade old (Tullis, 1981) and in the other, two decades old (Christ, 1975). In the latter review, 75% of the articles reviewed were written in or prior to 1971,50% of them were written in or prior to 1965, and 25% of them were written in or prior to 1960. Thus the widespread citing of Christ's conclusions promulgates what may be some dubious, aging generalizations.

Age alone does not necessarily invalidate generalizations, but one has to wonder whether the results of investigations conducted on the hardware available in the 1960s or 1970s or even the 1980s really has currency in today's rapidly shifting technological world. For example, the rapid emergence and widespread dissemination of high resolution, many-bits-deep colour monitors throws into question generalizations derived from studies conducted on relatively coarse-grained monitors capable of displaying only six or eight colours. Today, sixteen-bit colour is fairly common, and many systems sport twenty-four-bit and thirty-two-bit colour. In addition to the greater number of hues these systems make available, they afford much more control over
saturation and, therefore, contrast, which has been shown to have considerable importance (Adkins & Pease, 1991; Baker, Belland, & Cambre, 1985, 1986; Faiola & DeBloois, 1988; Mills & Weldon, 1987; Pace, 1984; Radl, 1980; van Nes, 1986). Given that the human eye can distinguish many different colours and that we are capable of providing many thousands of different colours on commonly-available VDTs, how useful is empirical evidence about the optimality of a given colour of text on a given colour of background, given that the research was conducted on equipment capable of generating only eight colours? Unless the colours involved are described in a much more specific fashion (e.g., Munsell colour system coordinates, or RGB values) than has been done to date, little useful knowledge obtains.

As another example, consider the widely-promulgated advice that navigation elements of a screen be consistent in placement and type, a notion that appears to be much more experientially derived than empirically. Consistency may still be good advice, but given that the generalization was derived primarily on the basis of experience with mainframes that were character-display and command-line or text-menu-based, one wonders how rigidly to apply that advice to a graphical user interface with hypertext capabilities. To take a more extreme example, some of the literature contains advice that is plainly obsolete (e.g., "use character sets with true descenders").

What this means, then, is that instructional designers must learn to pay close attention to the dates when research was conducted, and attempt to ascertain the currency of the equipment used, before accepting generalizations as guides to their efforts. By the same token, researchers and authors of literature reviews should be sensitive to the issue, and make the reader aware when generalizations might no longer be valid.

## The Nature of the Task

Another shortcoming of many of the published summaries of research on the use of colour in screen design is they do not take into account the type of task used in the research being summarized, hence both instructional and non-instructional uses of colour in screen displays are lumped together in recommendations for practice. Results of studies conducted in pursuit of improved air traffic control systems are sometimes mixed in with others to set the stage for an experimental procedure in education, or are offered by reviewers as purportedly relevant to instructional practice. But are they? A careful analysis of task demands seems warranted.

We classified recent empirical studies according to whether the nature of the task employed was similar to instruction. Only about one third of those employed tasks that were clearly similar to instruction; about one-sixth were classified as "maybe" instructional (meaning that arguments might be made for them, or that it was not possible to tell from the description what the task was). The remaining half of the studies used tasks that were not related to common instructional activities. Clearly, at minimum, great caution must be used when generalizing the results of the third group to instructional situations.
<table><caption>**TABLE 2**
*Tasks Employed in Recent Empirical Research*</caption><thead><tr><th>Task Type</th><th>Task Description</th></tr></thead><tbody><tr><td>Clearly Instructional</td><td><ul><li>CAI teaching mathematical rule for average speed (animation) (Baek & Layne, 1988)</li><li>color cueing in geometry lesson (Hativa & Teper, 1988)</li><li>selecting colour combination preference (D'Angelo, 1991)</li><li>oral reading and rating of comfort (Simmers, 1988)</li><li>locating facts in materials previously read (Wright & Lickorish, 1988 [1]*, Wright & Lickorish, 1988 [2])</li><li>reading from a screen for rate and comprehension (Claiming & Schmitt, 1989, 1990)</li></ul></td></tr><tr><td>Maybe Instructional</td><td><ul><li>recognizing a color picture of an object presented on a monochrome display (Baker, Belland, & Cambre, 1985*, Baker, Belland, & Cambre, 1986** )</li><li>searching through menus to locate a specific page of information (Kerr, 1987)</li><li>reading information from one part of screen and inputting it on another (Pace, 1984 [2])</li></ul></td></tr><tr><td>Clearly Non-Instructional</td><td><ul><li>operating a fast food cash register with a keyboard-like layout (McDonald, Molander, & Noel, 1988)</li><li>searching for nonsense words (Pace, 1984 [1])</li><li>counting frequency of random target letter (Anglin & Towers, 1993; Ohlsson, Nilsson, & Ronnberg, 1981)</li><li>"Concentration" game with alphabets (Livingston, 1991)</li><li>transcribing letters shown on the screen and identifying number flashed on the screen (Radl, 1980[1], [2])</li><li>naming the color of small moving squares appearing on the screen (Radl, 1980[3])</li><li>detecting and naming color of a moving square on a colored background (.Radl, 1980[4])</li><li>rating preference for color combinations (Holcomb, 1991; Pastoor, 1990 [1])</li><li>oral reading of nouns in random order; locating target word in list of similar words; rating preferences (Pastoor, 1990 [2])</li><li>speed and accuracy of interpreting schematic in trouble shooting problems on telephone lines (Tullis, 1981)</li><li>identifying letters and digits flashed on a screen (Bruce & Foster, 1982)</li></ul></td></tr></tbody></table>

<footnote>* Some papers report more than one study. In this table, multiple studies by the same author(s) are designated with numerals in square brackets.</footnote>

<footnote>** These two papers appear to report the results of the same study.</footnote>
Table 2 classifies recent empirical studies according to task employed.

Research methods for studying colour in screen design are subject to the same debate that has flourished since nonsense syllables were first used to investigate processes of learning: How important is it that the tasks employed in research studies approximate real life? Is it better to risk confounding from the content or instruction, or to employ a task that is “content-free”? The current consensus seems to be that a high degree of similarity between a research task and real life is essential. That fact seems to have been glossed over in some of the recent research on colour in screen design.

*Different Display Media*

Another problem centres on studies involving colour that were conducted on media other than VDTs. While the results of such studies may, in fact, be valid for VDTs as well, there is reason to be cautious in making the generalization. For example, it has been shown that reader preferences for fonts in printed materials are quite consistent (Misanchuk, 1989a; Tinker, 1963, 1965). However, users prefer quite different fonts on computer screens than they do on paper (Misanchuk, 1989c). There is also some indication that leading (vertical spacing of text) on paper and on a VDT might show similar differences (Misanchuk, 1989b). Might colour effects and preferences suffer similar changes in response to changed display media?

Furthermore, CRTs are radiant light sources, operating on the additive colour system, while traditional colour theories are based primarily on reflected light, using the subtractive system. It is a qualitatively different experience to view text or images on paper and on a VDT, and “[artists] are discovering that certain aspects of colour theories used in traditional art media are not applicable to computer graphics” (Collery, 1985, p.1).

*What Has “Science” Taught Us About Colour?*

To determine what we really know-from empirical evidence-about the use of colour for screen design of instructional materials, we examined those empirical studies that employed tasks that were identified in Table 2 as either clearly instructional or maybe instructional.

Table 3 summarizes those studies into four categories, representing those that:

* showed no significant difference;

* indicated user preferences;

* possibly showed a negative effect; and

* may have restricted generalizability because of special characteristics of the subjects or the hardware used.

Holcomb's study used only five colour combinations (hence did not cover the complete range of possibilities), tested only for preference (hence efficacy is unknown), and involved subjects with special characteristics (over age 40).
<table><caption>**TABLE 3**
Summary of Empirical Research Studies</caption><thead><tr><th>Category</th><th>Author(s)</th></tr></thead><tbody><tr><td>No significant difference</td><td>Baek & Layne, 1988<br>Clausing & Schmitt, 1989<br>Clausing & Schmitt, 1990<br>D'Angelo, 1991<br>Kerr, 1987</td></tr><tr><td>Preferences identified</td><td>Holcomb, 1991</td></tr><tr><td>Possibly negative results</td><td>Wright & Lickorish, 1988 [1]<br>Wright & Lickorish, 1988 [2]</td></tr><tr><td>Possibly restricted generalizability</td><td>Baker, Belland, & Cambre 1985*<br>Baker, Belland, & Cambre, 1986*<br>Hativa & Teper, 1988<br>Simmers, 1988</td></tr></tbody></table>

*These two papers appear to report the results of the same study.

Wright and Lickorish used fairly rudimentary equipment by today's standards (a Z-80 machine with an 80-column by 32 row display). Their major finding was no significant difference in reading time, but there was some evidence that gains resulting from practice at the task were smaller for the colour-cued versions than for non-colour-cued versions, hence they speculated that the colour cues may have interfered. Baker, Belland, and Cambre use Apple II low-resolution graphics, which are also quite rudimentary by today's standards. Hativa and Teper's use of a computer as an "electronic chalkboard" is not one that immediately springs to mind when one talks about using computers for instruction. Is it safe to generalize their findings to screen design? Simmers' subjects were partially-sighted, hence his results may not generalize to the larger population.

So, what do we know from the "scientific" literature about the use of colour in screen design for instruction? Clearly, not much. There is an obvious need for considerable research to be conducted in the area of using colour in the design of screen displays for instruction.

Generally speaking, we place high value on the results of empirical research, even while recognizing that empirical research may not be capable of providing all the answers, and that research results are coloured by the way in which the research was conducted. At this stage, it appears that opinion, myth, and legend may inform instructional screen design practice more than
science. Hopefully, this review will serve as a wake-up call to researchers about the need for more investigation into an area in which it may seem, at first blush, we already know a good deal.

## A CALL FOR FURTHER RESEARCH

One level of research required is merely the replication of the best of earlier studies, on newer technology, and with more attention to reporting specifics of that technology. We need to test conclusions in light of improvements in display technology. For example, do earlier findings of "good" and "bad" (however they are defined) combinations of text on backgrounds hold when de-saturated colours are employed? Both propositions derived from physiological research (Murch, 1984) and experiential advice (e.g., Faiola, 1990; Faiola & DeBloois, 1988, among others) would lead one to conclude that de-saturated colour, especially for backgrounds, is preferable to saturated colour, but we have been unable to locate any research in which this hypothesis was tested in an instructional situation. (Our own observations have led us to speculate that beige or light gray might form the most pleasing and effective background against which to present text, for example, but we have not yet subjected that speculation to empirical verification.)

On another level, we now have the technological wherewithal to go beyond static displays on VDTs; we need to recognize that emerging multimedia technologies introduce new questions. What is the role played by compressed colour video in displays? How can colour be used effectively to present animated graphics? How do various compression strategies influence colour? Does colour really motivate, as is often claimed, or does it interfere and distract, as has also been alleged (Brockmann, 1991; Rubens, 1986)?

The number of possible research questions involving the use of colour in screen design for instructional purposes is very large. In investigating any of them researchers would do well to ensure that their tasks are relevant to the population to which they hope to generalize, that the equipment used is fairly contemporary, and that they provide a great deal of technical detail (with respect to that equipment and the way in which it was used) when reporting their results. Prospective summarizers of research should also keep these imperatives in mind when teasing out generalizations. Finally, instructional designers seeking to apply generalizations to their work should act as a second level of filtering, by once again checking to see that the imperatives were applied at earlier stages.

## TOWARD A CONNOISSEURIAL APPROACH

The literature does not favor either an exclusively scientific or a non- empirical treatment of screen design. On the one hand, we have the "art"
literature, which offers empirically unsubstantiated and sometimes conflict-
ing advice, traditions, and opinions; on the other, the "science" literature,
which when examined critically is reduced to providing little useful guidance
to designers of multimedia screens for instruction. Neither empiricists nor
artists can find much comfort in the current state of affairs.

To consider screen design as a bipolar issue with rational and creative
emphases at either end of the spectrum creates a false dichotomy. We favor
a connoisseurial treatment where science and experience both have value and
can contribute to a rich understanding of how screens can be designed for
multimedia presentations. Aesthetically pleasing multimedia can employ
scientifically derived principles of instructional design, and well-designed
instruction can be creative and aesthetically pleasing.

One way to approach the convergence of science and opinion in screen
design is through a model of connoisseurship. A connoisseur has a refined set
of skills and principles for making judgments. Belland (1991) discussed four
critical aspects of performing connoisseurship. First, a connoisseur must be
able to make fine discriminations; that is, perceive differences and elements
that are too subtle for unschooled individuals to notice. Next, the connoisseur
develops an hierarchical system of concepts for making judgments. Systems
of key ideas and subordinate notions are developed, even if not articulated to
others. Third, principles are developed to describe the structure of relation-
ships among concepts. Finally, a connoisseur develops strategies to focus on
salient aspects, and ignore less important aspects of the item being judged.

A connoisseurial approach, with its emphasis on complexity and fine
discrimination, lends itself well to multimedia design. In sophisticated
multimedia productions, designers are concerned with a combination of
technical, aesthetic, and narrative issues in which colour might play a role.
How, for example, might colour be used to convert a dramatic scene to a
comedic scene? How might colour act as a metaphor for emotions such as joy,
anticipation, or contentment? How can colour be used to convey elegance or
sophistication?

Clearly, to develop a connoisseurial appreciation for multimedia, one
must be informed about both science and art, in both senses of both words. In
developing the notion of connoisseurship, Belland (1991) further argues that
an individual's involvement with a work must be extensive and intensive. A
connoisseur draws on a broad body of knowledge and experience which allows
the work being examined to be interrelated with other significant works. It
is also important for a connoisseur to give attention in an analysis to those
elements which are important (ignoring or downplaying trivial elements), and
to reflect on the meaning of the new experience. Judgments are made within
a context of acquired knowledge, taste, and experience, and aesthetic judg-
ments are amalgams, not precise articulations of specific rules.
## CONCLUSION

We ask you to consider the entirety of the above advice within the contexts of projects, not as a set of generalizable principles. Consider your project and its design as whole cloth rather than a series of micro-decisions to be made. "Design quality and consistency grow from a coherent set of ideas, not from personal taste or committee compromises, not from the baggage of past user interfaces, not from the ad hoc reasoning about each little part of the computer screen" (Tufte, 1992, p. 15).

One can extract guidance on the use of colour exclusively from the empirical literature or from less rigorous sources, but peril lurks in both camps. We need more and better studies about colour in screen design to be sure, but we also recognize the value of judgement, common sense, and a refined connoisseurial sensitivity.

Given the literature we have reviewed, we conclude with the single best piece of advice about using colour we have found:

"Above all, do no harm." (Tufte, 1992, p.16)

## REFERENCES

Adkins, M., & Pease, W. (1991, May). *Using color as information in computer displays: Problems with perception and communication*. Paper presented at the Annual Meeting of the International Communication Association, Chicago, IL. (ERIC Document Reproduction No. ED 338 222)

Anglin, G. J., & Towers, R. L. (1993, January). *The effects of text and background color combinations on viewer preference, and speed and accuracy in scanning letters or words displayed on a CRT*. Paper presented at the Annual Meeting of the Association for Educational Communications and Technology, New Orleans, LA.

Baecker, R. M., & Buxton, W. A. S. (1987). The visual channel. *In Readings in human-computer interaction: A multidisciplinary approach* (pp. 299-307). Los Altos, CA: Morgan Kaufmann Publishers.

Baek, Y. K., & Layne, B. H. (1988). Color, graphics, and animation in a computer-assisted learning tutorial lesson. *Journal of Computer-Based Instruction, 15*, 131-135.

Bailey, H. J., & Milheim, W. D. (1991, February). A comprehensive model for designing interactive video based materials. *Proceedings of the Ninth Conference on Interactive Instruction Delivery, 1991*. Society for Applied Learning Technology Conference, Orlando, FL.

Baker, P. (1983). Computer color graphics and monochromatic display: Are they compatible? *Educational Technology, 23*(12), 22-24.
Baker, P. R., Belland, J. C., & Cambre, M. A. (1986, January). *The display of color graphics on monochrome monitors: A concern for designers and an opportunity for researchers*. Paper presented at the Annual Convention of the Association for Education Communications and Technology, Las Vegas, NV. (ERIC Document Reproduction Service No. ED 267 756)

Baker, P., Belland, J., & Cambre, M. (1985). Recognition of computer-generated pictures on monochrome monitors. *Journal of Computer-Based Instruction, 12*(4), 104-107.

Belland, J.C. (1991). Developing connoisseurship in educational technology. In D. Hlynka & J.C. Belland (Eds.), *Paradigms regained: The uses of illuminative, semiotic and post-modern criticism as modes of inquiry in educational technology* (pp. 23-25). Englewood Cliffs, NJ: Educational Technology Publications.

Brockmann, R. J. (1991). The unbearable distraction of color. *IEEE Transactions on Professional Communication, 34*(3), 153-159.

Brou, P., Sciascia, T. R., Linden, L., & Lettvin, R. Y. (1986). The color of things. *Scientific American, 255*(3), 84-92.

Bruce, M., & Foster, J. J. (1982). The visibility of colored characters on colored backgrounds in Viewdata displays. *Visible Language, 16*, 382-390.

Chapman, W. (1993). Color coding and the interactivity of multimedia. *Journal of Educational Multimedia and Hypermedia, 2*(1), 3-23.

Christ, R. (1975). Review and analysis of color coding research for visual displays. *Human Factors, 17*, 542-570.

Clausing, C. S., & Schmitt, D. R. (1989, November). *The effects of computer usage on computer screen reading rate*. Paper presented at the Annual Conference of the Mid-South Educational Research Association, Little Rock, AR. (ERIC Document Reproduction Service No. ED 317 192)

Clausing, C. S., & Schmitt, D. R. (1990, November). *Does computer screen color affect reading rate?* Paper presented at the Annual Meeting of the Mid-South Educational Research Association, New Orleans, LA. (ERIC Document Reproduction Service No. ED 324 666)

Collery, M. T. (1985). Color in three-dimensional shaded computer graphics and animation. Unpublished master's thesis, Ohio State University. (ERIC Document Reproduction Service No. ED 281498)

D'Angelo, J. J. (1991). A study of the relationship between the use of color for text in computer screen design and the age of the computer user. *Disser-tation Abstracts International, 52*(12), 6497-6498-B.

Davidoff, J. D. (1987). The role of colour in visual displays. *International Reviews of Ergonomics, 1*, 21-42.

Durrett, J., & Trezona, J. (1982). How to use color displays effectively: The elements of color vision and their implications for programmers. *Pipeline, 7*(2), 13-16.

Dwyer, F. M. (1972). *A guide for improving visualized instruction*. State College, PA: Learning Services.

Dwyer, F. M. (1978). *Strategies for improving visual learning*. State College, PA: Learning Services.
Dwyer, F. M. (1987). *Enhancing visualized instruction: Recommendations for practitioners*. State College, PA: Learning Services.

England, E. (1984). Colour and layout considerations in CAL materials. *Computers and Education, 8*, 317-321.

Faiola, T. (1990) Principles and guidelines for a screen display interface. The *Videodisc Monitor*, 8(2), 27-29.

Faiola, T., & DeBloois, M. L. (1988). Designing a visual factors-based screen display interface: The new role of the graphic technologist. *Educational Technology, 28*(8), 12-21.

Fleming, M., & Levie, W. H. (1993). *Instructional message design: Principles from the behavioral and cognitive sciences* (2nd ed.). Englewood Cliffs, NJ: Educational Technology Publications.

Galitz, W. O. (1989). *Handbook of screen format design*. Wellesley, MA: QED Information Sciences, Inc.

Garner, K. H. (1991). 20 rules for arranging text on a screen. In R. B. Frantzreb (Ed.), *Training and development yearbook 1991 edition*. Englewood Cliffs, NJ: Prentice Hall.

Gillingham, M. (1988). Text in computer-based instruction: What the research says. *Journal of Computer-Based Instruction, 15*, 1-6.

Hall, R. (1989). *Illumination and color in computer generated imagery*. New York: Springer-Verlag.

Hanks, P. (Ed.) (1986). *Collins dictionary of the English language*. London: Collins.

Hannafin, M. J, & Hooper, S. (1989). An integrated framework for CBI screen design and layout. *Computers in Human Behavior, 5*(3), 155-165.

Hathaway, M. (1984). Variables of computer screen display and how they affect learning. *Educational Technology, 24*(1), 7-11.

Hativa, N., & Teper, A. (1988). Differential effectiveness of three color treatments in learning geometric concepts via computer-guided teaching. *Journal of Computing Research, 4*, 303-320.

Heines, J. (1984). *Screen design strategies for computer-assisted instruction*. Bedford, MA: Digital Press.

Hoekema, J. (1983). Interactive videodisc: A new architecture. *Performance and Instruction, 22*(9), 6-9.

Holcomb, T. (1991, February). *Computer screens and the ouer 40crowd: Color screen design and the aging process*. Paper presented at the Annual Conference of the Association for Education Communications and Technology, Orlando, FL.

Horton, W. (1991). Overcoming chromophobia: A guide to the confident and appropriate use of color. *IEEE Transactions on Professional Communication, 34*(3), 160-171.

Isaacs, G. (1987). Text screen design for computer-assisted learning. *British Journal of Educational Technology, 1*(18), 41-51.
Kerr, S. T. (1987, February). *Finding one's way in electronic space: The relative importance of navigational cues and mental models*. Paper presented at the Annual Convention of the Association for Education Communications and Technology, Atlanta, GA. (ERIC Document Reproduction No. ED 285 545)

Livingston, L. A. (1991). The effect of color on performance in an instructional gaming environment. *Journal of Research on Computing in Education*, 24, 246-253.

McDonald, J. E., Molander, M. E., & Noel, R. W. (1988). Color-coding categories in menus. In *CHI '88 Conference Proceedings on Human Factors in Computing Systems* (pp. 101-106). New York: Association for Computing Machinery.

Milheim, W. D., & Lavix, C. (1992). Screendesign for computer-based training and interactive video: Practical suggestions and overall guidelines. *Performance and Instruction*, 31(5), 13-21.

Mills, C. B., & Weldon, L. J. (1987). Reading text from computer screens. *ACM Computing Surveys*, 19(4), 329-358.

Misanchuk, E. R. (1989a). *Learnerpreferences for typeface (font) and leading in print materials*. Saskatoon, SK: Division of Extension and Community Relations, The University of Saskatchewan. (ERIC Document Reproduction Service No. ED 307 854)

Misanchuk, E. R. (198913). Learner preferences for screen text attributes in a Macintosh@ microcomputer learning environment. In *Transitions: Proceedings of the AMTEC '89 Conference*. (pp. 339-355). Edmonton, AB: Association for Media and Technology in Education in Canada.

Misanchuk, E. R. (1989c). Learner/user preferences for fonts in microcomputer screen displays. *Canadian Journal of Educational Communication*, 18(3), 193-205.

Misanchuk, E. R., & Schwier, R. A. (1995). Appropriate uses of color in multimedia screen design: An illustrated guide to research results [unpublished HyperCard stack]. Saskatoon, SK: Extension Division, University of Saskatchewan.

Murch, G. M. (1984). Physiological principles for the effective use of color. *IEEE Computer Graphics and Applications*, 4(11), 49-54.

Ohlsson, K., Nilsson, L. G., & Ronnberg, J. (1981). Speed and accuracy in scanning as a function of combinations of text and background colors. *International Journal of Man-Machine Studies*, 14, 215-222.

Olson, S., & Wilson, D. (1985). Designing computer screen displays. *Performance and Instruction*, 24(1), 16-17.

Pace, B. K. (1984). Color combinations and contrast reversals on visual display units (pp. 326-330). In Aluisi, M. J., de Groot, S., & Aluisi, E. A. (Eds.) *Proceedings of the Human Factors Society 28th annual meeting volume 1*.

Pastoor, S. (1990). Legibility and subjective preference for color combinations in text. *Human Factors*, 32(2), 157-171.
Radl, G. W. (1980). Experimental investigations for optimal presentation-mode and colours of symbols on the CRT screen. In E. Grandjean & E. Vigliani (Eds.), *Ergonomic aspects of visual display terminals* (pp. 127-136). London: Taylor & Francis.

Rambally, G. K., & Rambally, J. W. (1987). Human factors in CAI design. *Computers and Education, 11*, 149-153.

Reilly, S. S., & Roach, J. W. (1986). Designing human/computer interfaces: A comparison of human factors and graphic arts principles. *Educational Technology, 26, 36-40*.

Rice, J. F. (1991). Display color coding: 10 rules of thumb. *IEEE Software, 8*(1), 86-88.

Rowland, G. (1993). Designing and instructional design. *Educational Technology Research and Development, 41*(1), 79-91.

Rubens, P. (1986). Online information, traditional page design, and reader expectation. *IEEE Transactions on Professional Communications, PC-29*(4), 75-80.

Sawyer, T. A. (1985). Human factors considerations in computer-assisted instruction. *Journal of Computer-Based Instruction, 12*, 17-20.

Shaw, D. (1991). The human-computer interface for information retrieval. *Annual Review of Information Science and Technology, 26*, 155-195.

Shneiderman, B. (1992). *Designing the user interface: Strategies for effective human-computer interaction* (2nd ed.). Reading, MA: Addison-Wesley Publishing.

Simmers, M. J. (1988). Visual impairment and computer displays: The effects of foreground and background color on oral reading speed. *Dissertation Abstracts International, 50*(2), 406-A.

Sloane, P. (1989). *The visual nature of color*. New York, NY: Design Press.

Steinberg, E. R. (1991). *Color in computer-assisted instruction*. Urbana, IL: University of Illinois, Computer-Based Education Research Lab. (ERIC Document Reproduction Service No. ED 343 568)

Strickland, R.M., & Poe, S.E. (1989). Developing a CAI graphics simulation model: Guidelines. *Technological Horizons in Education, 16*(7), 88-92.

Thorell, L. G., & Smith, W. J. (1990). *Using computer color effectively: An illustrated reference*. Englewood Cliffs, NJ: Prentice-Hall.

Tinker, M. A. (1963). *Legibility of print*. Ames, IA: Iowa State University Press.

Tinker, M. A. (1965). *Bases for effective reading*. Minneapolis, MN: University of Minnesota Press,

Travis, D. (1991). *Effective color displays: Theory and practice*. London: Academic Press.

Tufte, E. R. (1990). Envisioning information. Cheshire, CT: Graphics Press.

Tufte, E. (1992). The user interface: The point of competition. *Bulletin of the American Society for Information Science*. (June/July, 1992), 15-17.

Tullis, T. (1981). An evaluation of alphanumeric, graphic, and color information displays. *Human Factors, 23*(5), 541-550.
Tullis, T. S. (1983). The formatting of alphanumeric displays: A review and analysis. *Human Factors, 25*, 657-682.

van Nes, F. L. (1986). Space, colour and typography on visual display terminals. *Behaviour and Information Technology, 5*(2), 99-118.

van Nes, F. L. (1988). The legibility of visual display texts. In G. C. van der Veer & G. Mulder (Eds.), *Human-computer interaction: Psychonomic aspects* (pp. 14-25). Berlin, Germany: Springer-Verlag.

Waller, R., Lefrere, P., & Macdonald-Ross, M. (1982). Do you need that second color? *IEEE Transactions on Professional Communication, PC-25*(2), 80-85.

Winn, W. (1991). Color in document design. *IEEE Transactions on Professional Communication, 34*(3), 180-185.

Wong, W. (1987). *Principles of color design*. New York: Van Nostrand Reinhold.

Wright, P., & Lickorish, A. (1988). Color cues as location aids in lengthy texts on screen and paper. *Behaviour and Information Technology, 7*(1), 11-30.

## AUTHORS

Earl R. Misanchuk is Professor of Extension, University of Saskatchewan, Extension Division, Saskatoon, SK S7N OWO.

Richard A. Schwier is Professor of Educational Communications and Technology, Department of Curriculum Studies. University of Saskatchewan, Saskatoon, SK S7N OWO.
Learning Strategies for Interactive
Multimedia Instruction: Applying
and Spatial

Eileen E. Schroeder
Richard F.

Abstract: Current interactive multimedia instruction often takes the form of an open-ended, unstructured database. In many systems, learners are guided to manipulate the lesson content through notetaking. The learner is able to open a notetaking screen by clicking a button, type in notes about the current topic being explored, or copy sections from different screens into the notebook, and either print or save these notes to disk. Guidance on taking notes is rarely given. This presents a problem since learners frequently do not incorporate structure in their notetaking or fail to elaborate on the new information. Since research has indicated that students are generally incomplete note-takers (Kiewra, 1987, 1988). this may represent a serious design flaw. A number of notetaking techniques have been proposed which could be incorporated into interactive multimedia to enhance learner encoding in a more structured fashion. These include linear approaches like outlining and spatial learning strategies like concept maps.

This paper provides an overview of the research on notetaking and two related learning strategies- outlining and concept maps. It examines the potential effectiveness of the approaches to aid the learner in encoding new information, examines current uses of these strategies in interactive multimedia instruction and offers suggestions for the incorporation of these learning strategies into future designs.

Examples of strategy use in current programs are provided.

Résumé: L'enseignement interactif multimédia se présente souvent sous la forme d'une base de données non-structurée. Fréquemment, afin d'assimiler le contenu d'une leçon, les apprenants et les cpprentantes sont amenés prendre des notes. On peut alors ouvrir un bloc-notes en cliquant un bouton, écrire ses idées sur le sujet exploré ou copier, dans le bloc-notes, des sections provenant de différents écrans.

On peut aussi imprimer ces notes ou les sauvegarder sur un disque.

On ne donne presque jamais de directives sur "comment prendre des notes. AE Ceci constitue un problème parce que, fréquemment, les apprenants et les apprenantes ne structurent pas leurs notes et ne détaillent pas la nouvelle information. On reconnaît dans la littérature que les étudiants prennent des notes incomplètes (Kiewra, 1987, 1988), ce qui constitue, un défaut majeur de conception. Certaines techniques de prises de notes ont été proposées et peuvent être incorporées dans l'enseignement interactif multimédia de façon à permettre l'apprentissage d'une manière plus structurée. Ceci comprend les approches linéaires comme l'écriture des titres et des passages imporants et ds approches spatiales comme les cartes

Cet article founit un aperçu de la recherche sur la prise de notes et sur deux approches particulières : l'écriture des titres et des passages importants et
l'élaboration de cartes conceptuelles. On examine l'influence de ces techniques d'aide à l'encodage de nouvelles informations et les usages courants de ces strategies dans l'enseignement interactif multimedia et présente des suggestions quant à l'utilisation de ces sfrated'apprentissage dans le design d'activites. On foumit aussi des exemples de strategies utilisées dans des programmes en vigueur.

## INTRODUCTION

With a growing interest in interactive multimedia comes an increased requirement for research on instructional design strategies for this hybrid technology. Although a universal definition of multimedia has eluded even technology experts (Galbreath, 1992), the term "interactive multimedia instruction" is generally taken to describe a computer-controlled system which provides the possibility of varying combinations of digitized audio, graphics and text, analog and digitized video, accessed through the computer itself and/or a variety of peripheral devices such as videodisc players, compact disc players and music synthesizers (c.f., Schwier and Misanchuk, 1993; Gayeski, 1993). As such, the term incorporates related terminology such as interactive video and hypermedia.

Interactive multimedia instruction can be linear and/or structured or more non-linear, resembling a database incorporating multiple formats. The organization of the information in a tightly structured program, those designed for a specific instructional purpose or objective, is generally very clear. Programs such as Eduquest's *Stories and More*, a literature-based curriculum system, offer information in multiple formats (digitized audio, graphics, text), allowing the learner choices, but still guide the user in a structured fashion. However, many of the newest programs, such as IBM's *Illuminated Manuscripts* and Intellimation's *Letter from Birmingham Jail* are basically open-ended, with the structure of the information less obvious to the user. These represent multimedia databases where the user selects the path, information, and format to view.

Such open-ended approaches are consistent with a constructivist view of learning which holds that the learner individually constructs knowledge through interpreting perceptual experiences of the external world (Jonassen, 1991). Learners develop unique associations between prior knowledge stored in long term memory structures and new information. Constructivists encourage inductive, or discovery, learning in which learners engage a domain and "construct their own concepts and rules based on their interpretation of the instances encountered" (Rieber, 1992, p.96).

Interactive multimedia instruction can provide the rich learning environment central to a constructivist view of learning by furnishing databases of information in multiple formats and perspectives which nurture incidental learning. However, such environments necessarily provide a limited amount of structure and require learners to create their own. Learners may benefit from incorporated learning strategies that help them organize, encode, and
integrate the information they encounter in interactive multimedia instruc-
tion, strategies such as paraphrasing, generating questions, outlining, cogni-
tive mapping, creating images and summarizing. These activities could be
supplied by the instructional system or generated by the learner.

This paper will provide an overview of the research on three widely used
learning strategies: notetaking, perhaps the most frequently used strategy of
all, and two related techniques, outlining and concept mapping. It will
provide an analysis of their effectiveness to aid the learner in encoding new
information, examine current uses of these strategies in interactive multime-
dia instruction, and offer suggestions for the incorporation of these learning
strategies into future designs.

LEARNING STRATEGIES TO FACILITATE ENCODING

Bruning (1983, p. 93) defines learning strategies as "any internally or
externally mediated cognitive process that will facilitate transfer of informa-
tion to be learned from short-term into long-term memory". Information-
processing theory holds that short term memory has a limited processing
capacity such that learners are forced to select from all possible information
perceived for processing. Short term memory holds information for only
seconds before it is lost or encoded for storage in long term memory. Learning
strategies are generally called into use at this point to facilitate this transfer
of information (Bruning, 1983).

Weinstein and Mayer (1986) also link learning strategies to encoding.
Using Cook and Mayer's (1983) analysis of the encoding process, they describe
encoding as comprised of four components: a) selection- the learner actively
pays attention to some of the information impinging on sense receptors and
transfers it to working memory; b) acquisition — the learner actively
transfers information between working and long-term memory for further
study; c) construction — the learner actively builds internal connections
between ideas in the information that reaches working memory; and d)
integration- the learner actively searches long-term memory for prior knowl-
edge and transfers it to working memory to construct external connections with
the new information.

Learning strategies are used to assist the learner to rehearse, organize,
and elaborate information to make it more meaningful. Rehearsal strategies,
such as underlining and repetition, help focus attention on important infor-
mation and encode it in short term memory (selection and acquisition).
Organization strategies, such as outlining, categorization, and mapping, help
in selecting appropriate information and constructing connections among the
ideas (construction). Elaboration strategies, such as mental imagining,
forming analogies, inserted questions, paraphrasing and analyzing key
points, help transform information by making the material more meaningful
and building connections among new ideas and prior knowledge (integration).
Notetaking

The majority of the research has focused on notetaking from such linear
presentations as oral or videotaped lectures (e.g., Hartley & Davies, 1978;
Carrier and Titus, 1979; Rickards, 1979; Kiewra, 1985,1987) or on notetaking
from text (e.g., Kiewra, DuBois, Christensen, Kim & Lindberg, 1989; Wade &
Trathen, 1989). Notetaking from non-linear approaches such as interactive
multimedia has yet to be studied. To date, the authors are aware of only two
studies which have broached the topic indirectly through studies of the use of
participatory graphic organizers with interactive video (Kenny, Grabowski,
Middlemiss, &Van Neste-Kenny, 1991; Kenny, in press).

The graphic organizer was derived from the advance organizer and is
more pictorial, visual, or chart-like in structure. It can be participatory —
students participate in the completion of the organizer — or given to the
student completed (the final form organizer). The participatory graphic
organizer, in effect, engages learners in a form of structured notetaking. Kenny
et al. (1991) compared participatory graphicorganizers to the identical final form
versions on the learning of nursing students from an interactive video program
onnursing elderly patients withpulmonary disease. The participatory graphic
organizer group substantially outperformed the final form group on a test of
learning, scoring an average of 1.77 points higher on an 18-question multiple
choice test. The difference, however, was not statistically significant. Consid-
erable extraneous notetaking by subjects in both groups likely confounded any
differences that might have been fostered by the structured notetaking treat-
ment.

In the second study, Kenny (in press) compared the use of an advance
organizer to that of participatory and final form graphic organizers with
interactive video on cardiac nursing. This time, extraneous notetaking was
controlled. The final form graphic organizer proved most effective, garnering
the highest mean scores on both tests of learning and retention, while the
participatory version group had the lowest mean scores. The difference
between the final form and participatory organizer group means was statis-
tically significant at the p<0.01 level for both learning and retention. This
study, then, like the first, provided no- support for the use of structured
notetaking with a non-linear medium.

*The encoding and external storage hypotheses.* Two hypotheses (Divesta & Gray, 1972) have been advanced to explain the potential effectiveness of notetaking to facilitate learning: (a) notetaking assists initial encoding (the encoding hypothesis), or (b) notetaking provides a product which can be reviewed later (the external storage hypothesis). The encoding hypothesis or process function holds that notetaking is beneficial, independent of review, because it increases attention during the lecture and, therefore, facilitates the initial encoding of lecture ideas into long term memory (Kiewra, 1985, 1987; Hartley &Davies, 1978). Peper and Mayer (1986) advance three sub-hypotheses to explain why the encoding hypothesis may or may not be true. The first two are based on how much is learned while the third focuses on the degree to which the
learner is able to actively relate material to existing knowledge. The attention hypothesis states that notetaking facilitates learning by forcing the learner to pay more attention to presented material or to process presented material more deeply. The distraction hypothesis holds that, on the contrary, notetaking actually impedes learning by forcing the learner to concentrate on the motor act of writing instead of more fully listening to lecture. The generative learning hypothesis (Wittrock, 1974), claims that notetaking helps the learner to generate meaning by relating presented information to prior knowledge and thus building amoreintegratedlearningoutcome.

The external storage, or product, hypothesis holds that notetaking is beneficial because the notes comprise a tangible product which can be retrieved and used once the instructional event has passed. Rickards (1979) suggests two possible functions for the external storage idea: (a) a rehearsal function where enhanced recall is only due to remembering material from the notes just reviewed (the notes provide the learner with more information) and (b) a reconstruction function in which recall of notes allows learners to reconstruct parts of the passage on which no notes were taken (the notes help the learner to recall other information).

*The effectiveness of notetaking.* Research evidence for the encoding hypothesis is mixed. Combined findings (Kiewra, 1987) of review papers by Hartley and Davies (1978) and Kiewra (1985) reported 35 studies on notetaking from lectures supporting the initial encoding function, 23 indicat- ing no significant differences between those who do and those who do not record notes and 3 indicating the activity of notetaking to be dysfunctional relative to listening only. For notetaking from text material, there is some evidence that notetaking served a minimal encoding function and may even have interfered with processing (Kiewra, DuBois, Christensen, Kim & Lindberg, 1989). Also, a study by Wade and Trathen (1989) indicated that noting information (including notetaking) has little effect on the recall of that informa- tion.

Initial research findings appear to strongly support the external storage hypothesis. Combined findings (Kiewra, 1987) of review papers by Hartley and Davies (1978) and Kiewra (1985) reported 24 studies on notetaking from lectures supporting the product function (those who reviewed their notes achieved more), 8 studies indicating no significant differences between reviewers and non-reviewers and no study indicating reviewing notes to be dysfunctional.

However, Kiewra and his associates (Kiewra, DuBois, Christensen, Kim & Lindberg, 1989; Kiewra, DuBois, Christian, Mcshane, Meyerhoffer and Roskelly, 1991) claim that what has traditionally referred to as "external storage" is actually a combination of encoding and external storage. External storage treatment groups have consisted of students both making their own notes (encoding) and later reviewing them. Learners had, in effect, two chances to process the information (a repetition effect). When "external storage" is redefined as those who review notes created by the instructor but who have not
previously viewed the lecture, the results appear less conclusive. This reformu-
lated product function was shown to be less effective for factual recall and
recognition than anencodingplus storage treatment condition, although not for
higher order (synthesis) performance. Encoding only was consistently the least
effective treatment and no more effective than listening to a lecture without
notetaking. Thus, notetakers who review outperform notetakers who do not
review.

Further, the encoding process per se (without review) did not appear to be
aided by recording notes on linear or matrix frameworks (Kiewra, DuBois,
Christian, Mcshane, Meyerhoffer & Roskelly, 1991). This is consistent with
results from a study (Kenny, in press) of fill-in versus completed graphic
organizers used with interactive video where the notetaking treatment
appeared to interfere with, rather than facilitate, learning from the program.
In fact, when learning from text was tested, students who read twice from the
material but did not take notes outperformed those using any form of
notetaking (Kiewra, DuBois, Christensen, Kim & Lindberg, 1989). Other
studies comparing notetaking to repetitive reading (Hoon, 1974; Dynes, 1932;
Stordahl & Christensen, 1956) found notetaking no different from reading alone.
Notetaking, whether from lecture or text, may be a sufficiently demanding
process that relatively little encoding actually occurs during the act of notetaking
(Kiewra, DuBois, Christian, Mcshane, Meyerhoffer & Roskelly, 1991; Kenny, in
press). Not only does little encoding take place, but the students' notes are
generally incomplete (Kiewra, 1987; Kiewra, DuBois, Christian, Mcshane,
Meyerhoffer & Roskelly, 1991). In fact, the review of detailed instructors's notes
provides the best results (e.g. Kiewra, 1985, Kiewra & Frank, 1988; Risch &
Kiewra, 1990), probably due to their completeness and accuracy. Overall,
notetaking seems to function best as a rehearsal strategy using an accurate,
detailed product after the student has engaged the material through reading or
listening.

Outlining

Although the product/process research has not been extended to other
learning strategies, there has been research on the efficacy of strategies such
as outlining. Outlining is defined as "a high level skill which involves
identifying relationships between concepts and arranging those concepts in
an order which demonstrates the superordinate and subordinate nature of the
concepts involved" (Anderson-Madaus, 1990, p. 3). Outlining: (a) causes
focusing on important points, (b) helps students gain familiarity with text
structure, (c) aids retention, (d) generates useful alternative texts to supple-
ment materials read, and (e) causes active participation in learning (Bianco &
McCormick, 1989). However, outlining, like strategies such as imaging and
paraphrasing, requires a major intrusion in the reading processes and also
necessitates a significant amount of training to use properly (Anderson, 1980).

*The effectiveness of outlining.* Several studies have shown that outlining
results in improved recall of facts (Barton, 1930; Annis & Davis, 1975; Glynn &
DiVesta, 1977; Shimmerlik & Nolan, 1976). The use of a hierarchical summarization strategy, a form of outlining, improved comprehension and recall in the middle school students (Taylor & Beach, 1984), while ninth-graders who completed an outline grid when reading did better on multiple choice recall tests (Slater, Graves, & Piche, 1985). Further, research asking students to generate outlines provides some support for their facilitation of recall. For example, Tuckerman (1993) studied coded elaborative outlines, a method which involves outlining chapters, coding the main points using a six-code scheme, and adding elaborations of main points, such as examples and explanations. College students required to create coded elaborative outlines of chapters performed significantly better than students who did so voluntarily, those who created standard outlines, and those who did not outline (Tuckerman, 1993). Students given the option of outlining generally chose not to do so, but those not required to write these outlines performed better the more they outlined.

Outlining has also been compared to other learning strategies. Some studies report no advantage of outlining over other strategies when the subjects were not instructed in the strategy (Arnold, 1942; Stordahl & Christensen, 1956; Todd & Kessler, 1971; Willmore, 1966 cited in Iovino, 1989) or a marginal advantage for outlining in writing papers (Branthwaite, Trueman, & Hartley, 1980; Emig, 1971). The main difference between these and earlier studies is the lack of training in the strategy before use. However, more recent studies appear to support the use of the strategy, especially where training is provided (Palmatier, 1971; Snyder, 1984; Iovino, 1989; Kellogg, 1990).

Thus, Palmatier (1971) found that college students using outlining had the highest level of essential content in their notes compared to those using a three-column method, the Bartush Active Methods or no method, while Snyder (1984) found a significantly higher recall performance for the outlining method in a study comparing the use of SQ3R and underlining to study college textbooks. Iovino (1989) found that, after being taught the techniques, outlining significantly helped academically under-prepared college students to achieve higher immediate recall than did networking, but networking significantly improved their ability to retain information over time. In a study of the use of outlining and clustering in prewriting (Kellogg, 1990), outlining was most beneficial when only the topic was provided and students had to generate and organize ideas, but did not help if the topic, ideas, and organization were given.

Tenny (1988) and Anderson-Inman and others (1992) have studied computer outlining in low achieving high school students and have found the technique effective. Tenny (1988) found that such outlining was significantly more effective than rereading for all his subjects. He suggests that it is an effective study strategy because it allows students to manipulate information and put it in their own words, to monitor their own learning, and to take personal responsibility for their learning. Anderson-Inman and others (1992) found that the strategy was successful for only some of their subjects, generally those with more experience with the strategy, who had a more positive attitude toward it, who could complete it easily, who saw it as part of the study process, and who were reflective on the process.
Overall, then, it appears that outlining can aid in both organizing new information and integrating new knowledge into one's personal knowledge structure. Learners should, however, be taught to use the outlining strategy correctly (e.g. focusing on main ideas, organizing from abstract to concrete in the hierarchy, working from a classification of concepts) and should not perceive the strategy as being too intrusive in the study process, (Hoffler, 1983; Anderson 1980).

## Concept Mapping

An outgrowth of schema theory (Kiewra, 1988) is the study of learning strategies involving the reorganization of linear information into a spatial representation that specifies relationships among concepts. The process of creating a spatial arrangement requires a relatively deep level of processing aimed at determining internal connections among ideas.

Perhaps the most widely known and researched spatial learning strategy is concept mapping (Novak, Gowin & Johansen, 1983; Novak & Gowin, 1984; Heinze-Fry & Novak, 1990; Novak, 1990). Concept mapping was developed as a spatial knowledge representation technique based on Ausubel's Theory of Meaningful Learning which holds that knowledge in memory is hierarchical, with more general, more inclusive concepts subsuming progressively less inclusive, more specific ones (Novak & Gowin, 1984). Concept maps, then, are drawn hierarchically, with more inclusive concepts at the top of the map and progressively more specific ones arranged below, linked by labelled lines to form semantic units.

Concept maps are viewed, first and foremost, as a tool for negotiating meanings. Maps are constructed collaboratively by the instructor and the learner(s). However, they can also be used as a pre-instructional tool in the form of an advance or graphic organizer and as a notetaking technique for extracting key concepts from printed or oral material (Novak & Gowin, 1984).

*The effectiveness of concept mapping.* A recent meta-analysis of nineteen studies (Horton, McConney, Gallo, Senn & Hamelin, 1993) provides an overview of the general effectiveness of the technique. Meta-analysis is a technique which permits quantitative reviews and syntheses of the research issues (Wolf, 1986) and Glass' effect size statistic (E.S.) in particular (e.g., Glass, McGaw, and Smith, 1981). The E.S. allows the comparison of studies which vary in design, sample selection, and setting in order to form conclusions and, because it is based on standard deviations, also permits an assessment of degree of effect. Thus, for t-tests of independent means, an E.S. of 0.20 could be considered of mild strength (i.e. the mean of the population with the higher mean score exceeds the scores of 58% of the group with the lower mean score), and E.S. near 0.50 (moderate (mean of upper group > 69% of lower group scores) and those 0.80 and above as strong (mean of higher group > 79% of lower group) (Cohen, 1988, pp. 25-26).

Studies occurred in actual classrooms and used concept mapping as an instructional tool compared to an alternate technique as a control. Nearly all studies examined involved science content, material which could be argued lends
itself to a hierarchical depiction. In 15 of the 19 studies, the students prepared the maps. Effect sizes (ES.) for achievement ranged from -0.31 to + 2.02 with a mean E.S. of 0.46. For measures of student attitude towards the particular subject matter, E.S.'s ranged from 0.05 to 4.88 with a mean E.S. of 1.57. The investigators concluded that concept mapping has medium positive effects on achievement and large positive effects on attitude.

Research results also indicate that the primary benefit of concept map-
ping accrues to the person who constructs the map (Novak, 1990; Horton et al.,
1993), that teacher-prepared maps may be helpful to students, but only after
they have had practice preparing their own maps and that, at first (for 2-4
weeks), there is generally an average decline in performance for strategies
that require meaningful learning although they finish up significantly higher
(Novak, 1990). The implication is that time is needed for students to learn and
learn to appreciate meaningful learning strategies such as concept mapping.
Concept mapping, then, can function as a rehearsal, organizational, and/or
elaboration strategy. In its most effective form, it represents a combination
of all three.

INTEGRATING LEARNING STRATEGIES IN
INTERACTIVE MULTIMEDIA INSTRUCTION

As discussed previously, current interactive multimedia instruction often
takes the form of a multiple format database. In many systems, learners are
guided to manipulate the lesson content through notetaking. The learner is
able to open a notetaking screen by clicking a button, type in notes about the
current topic being explored, or copy sections from different screens into the
notebook, and either print or save these notes to disk. Guidance on taking
notes is rarely given. This presents a problem since learners frequently do not
incorporate structure in their notetaking or fail to elaborate on the new
information. Since research has indicated that students are generally
incomplete note-takers (Kiewra, 1987, 1988), this may represent a serious
design flaw. A number of notetaking techniques have been proposed which
could be incorporated into interactive multimedia to enhance learner encod-
ing in a more structuredfashion (Anderson-Inman, Redekopp, & Adams, 1992).
These include linear approaches like outlining (Kiewra, DuBois, Christian &
McShane, 1988) and spatial learning strategies such as concept maps (Novak
& Gowin, 1984).

Notebooks or notepads included in programs are often fairly rudimentary
and if used by students as is, risk replicating the same shortcomings of
notetaking, outlining and mapping on paper. The computer offers capabilities
that might be used to incorporate the best aspects of each of these learning
strategies while avoiding some of their pitfalls. It also offers the ability to offer
a degree of guidance to the user if necessary or if desired by the instructor.
Capabilities that might be exploitedinincorporatinglearning strategiesinto a
notebook include:

* using screen titles and/or topics to organize information;

* cutting, pasting and manipulating text and media (graphics, sound, video);

* the ability to connect and manipulate ideas and pull them together into a spatial map;

* creating a multimedia document from information gathered.

Anderson-Inman and Zeitz (1994) add global expansibility (i.e. the ability to automatically expand to accommodate inserted material anywhere), infinite modifiability, and focusability (i.e., the ability to hide unwanted detail). Kozma (1987) has also proposed the idea of anelectronic notebook. The *Learning Tool* by Intellimation and *Inspiration* by Inspiration Software for the Macintosh are examples of such programs. They present the learner with blank workspaces and tools to develop key points, connect them in networks, and provide textual and graphic information about each. The program cues, evokes, models, and supplements the learner's thought processes and, in effect, it provides a tool for outlining, concept mapping, and related spatial notetaking techniques.

Outlining in a computerized environment can add several new features to print-based outlining. Students can incorporate text as well as various other media types in a outline, by either cutting and pasting or linking to external resources. When using an interactive multimedia program, all the information can be included, no matter what the format. Applications such as *Mediatext*, *WordPerfect* and *Microsoft Word* already offer these features.

Many word processors also offer the ability to expand and compress outlines. If this feature were incorporated into a program's notebook, students could view their information at different levels of the hierarchy, for example, the degree of abstraction or detail. Instructional designers could provide an intact outline in which to take notes and include media, acting as guide through the system and showing one view of the overarching structure in a linear, hierarchical fashion. Different outlines might be available to show different perspectives on the information provided in the interactive multimedia instruction. Students might chose to view or not to view this as desired. Concept mapping tools add the ability to label the relationships between concepts, use different shapes for different concepts, create crosslinks, and indicate unidirectional or bidirectional links on top of the features of outlining (Anderson-Inman & Zeitz, 1994).

## Types of Notebooks

The design of notebooks for interactive multimedia systems can incorporate some or all of the above features. These allow learners to use strategies that assist in rehearsing, organizing and elaborating information to make it more personally meaningful.

*Standard notebooks.* The most common types of notebook either allow users
to make their own notes on program material, to copy and paste text from the
instructional program to the notebook, or both. Sometimes, they also permit the
copying of graphics and other media. While making one's own notes may
facilitate encoding, the act of cutting and pasting requires little or no cognitive
effort on the part of the student, other than selecting the information. This would
represent a true product function of notetaking. However, later review of the
notes produced by the student would constitute a form of rehearsal strategy. If
this same notebook allows the user to arrange and rearrange the text and
graphics, its use could be considered an organizational strategy. The use of
notebooks that allow learners to add their own text and graphics acts as an
elaboration strategy where the learner incorporates prior knowledge and relates
new with old information. The notebook then incorporates both the process and
product function of notetaking.

<figure><img src="image_9.png" /><figcaption>Figure 1.
Outlining</figcaption></figure>

Outlining *notebooks*. A notebook could also provide outlining capabilities where the learner can easily arrange information in a hierarchical fashion. Here learners create their own hierarchies of information and use the notebook for organization. Learners can also incorporate prior knowledge in either text or graphic form and link it to the new information, thus using the notebook for elaboration. When the notes produced are reviewed by the learner at a later time,
this use constitutes rehearsal. Figure 1 displays a sample screen from a hypermedia program on the circulation of blood through the heart, based on the materials of Dwyer and Lamberski (1978). The material has been organized into an outline.

The structure of the instructional program itself may be automatically incorporated into an outline in a notebook, acting as a supplanted organizational strategy. If learners can add text and graphics to this from the program itself

<figure><img src="image_10.png" /><figcaption>Figure 2. Concept Mapping</figcaption></figure>

and/or from prior knowledge, the notebook allows elaboration. As for other types of notebooks, if the material is then used for review, this constitutes a rehearsal strategy.

*Spatial mapping notebooks.* Learning strategies such as concept mapping can also be incorporated into a notebook. Figure 2 illustrates a potential notebook created using *Learning Tool*. The information selected can be incorporated into a concept map that can be manipulated as desired with links between concepts described. This type of notebook permits organization. Additional text, graphics, and media can be added to each concept (See Figure 3). This feature acts as an elaboration strategy where the user can add his/her own knowledge.

Each of these notebooks can also be incorporated into a student multimedia production where the information is analyzed, organized, synthesized, supplemented, and presented by the individual. The features of these notebooks each have their strengths and weaknesses. Several possibilities are detailed in the
<figure><img src="image_11.png" /><figcaption>**Figure 3.**
*Concept Map with Additional Text and Graphics*</figcaption></figure>

matrices below.

## INSTRUCTIONAL DESIGN CONSIDERATIONS

When determining the type of notebook to include with a program, the designer is advised to take into account the research on notetaking strategies, outlining and concept mapping as well as the capabilities of the computer and the needs of the learners. The designer should also be aware that notetaking in interactive multimedia instruction is not the same as note taking in a lecture, both because pacing is normally not an issue and because of the capabilities offered by the computer.

### *Advantages*

When working with interactive multimedia instruction learners have time to copy text and elaborate, similar to underlining with margin notes or two-column notetaking, and then to elaborate with their own text and graphics. Further, the ease of cutting and pasting allows learners to organize or categorize information either as they navigate the program or after gathering all desired information. They can also create graphical representations of the information or use the potential ease of expanding and collapsing outlines to assist them in perceiving the structure of the information at various levels. Tuckerman's (1993) research on coded elaborative outlining provides a basis for this use of a notebook.
<table><caption>Table 1. Advantages of different types of notebooks.</caption><thead><tr><th>ADVANTAGES</th><th>Copy &amp; paste whole screen of text &amp; graphics</th><th>Copy &amp; paste selections of text &amp; graphics</th><th>Arrange &amp; rearrange text &amp; graphics</th><th>Add own text &amp; graphics to information copied</th><th>Create own outline</th><th>Use headings to create hierarchy of outline</th><th>Create concept map</th><th>Add text &amp; graphics to concept map</th></tr></thead><tbody><tr><td>Learner actively selects relevant information</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td></tr><tr><td>Includes media as well as text in notes</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td></td><td></td><td>X</td></tr><tr><td>Recording of Information accurate</td><td>X</td><td>X</td><td>X</td><td></td><td></td><td>X</td><td></td><td></td></tr><tr><td>Can select only needed information</td><td></td><td>X</td><td>P</td><td>P</td><td>X</td><td>X</td><td>X</td><td>X</td></tr><tr><td>Accurate depiction of developer's organization</td><td></td><td></td><td></td><td></td><td></td><td>X</td><td></td><td></td></tr><tr><td>Can create unique structure via selection of headings/concepts</td><td></td><td></td><td></td><td></td><td></td><td>X</td><td>X</td><td>X</td></tr><tr><td>Can reorganize information to create own personal meaning</td><td></td><td></td><td>X</td><td>X</td><td></td><td></td><td></td><td></td></tr><tr><td>Can display own conception of relationships</td><td></td><td>X</td><td>X</td><td>X</td><td></td><td></td><td>X</td><td>X</td></tr><tr><td>Can elaborate on new information with prior knowledge to make meaningful</td><td></td><td></td><td>X</td><td>X</td><td></td><td>X</td><td></td><td>X</td></tr><tr><td>Can create own knowledge framework to use in program</td><td></td><td></td><td>X</td><td>X</td><td></td><td></td><td>X</td><td>X</td></tr><tr><td>May be able to hide or display layers of hierarchy as needed</td><td></td><td></td><td></td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td></tr><tr><td>Instructor can create initial framework as organizer</td><td></td><td></td><td>X</td><td></td><td></td><td>X</td><td>X</td><td>X</td></tr><tr><td>Graphical portrayal of relationships (hierarchical or heterarchical)</td><td></td><td></td><td></td><td></td><td></td><td></td><td>X</td><td>X</td></tr><tr><td>Can paraphrase to add own retrieval cues</td><td></td><td></td><td>X</td><td>X</td><td></td><td></td><td></td><td>X</td></tr></tbody></table>

Note: X=likely, P=possibleNote:
2. Disadvantages of different types of notebooks.

<table><thead><tr><th>DISADVANTAGES</th><th>Copy & paste whole screen of text & graphics</th><th>Copy & paste selections of text & graphics</th><th>Arrange & rearrange text & graphics</th><th>Add own text & graphics to information coupled</th><th>Create own outline</th><th>Use headings to create hierarchy of outline</th><th>Create concept map</th><th>Add text & graphics to concept map</th></tr></thead><tbody><tr><td>Requires little depth of processing</td><td>X</td><td>X</td><td></td><td></td><td></td><td>P</td><td></td><td></td></tr><tr><td>Encourages external storage of information, not encoding</td><td>X</td><td>X</td><td></td><td></td><td></td><td>X</td><td></td><td></td></tr><tr><td>Organization shown/created not personally meaningful</td><td>X</td><td></td><td></td><td></td><td></td><td>X</td><td></td><td></td></tr><tr><td>Difficult to get overview of organization of information</td><td>X</td><td>X</td><td>X</td><td>X</td><td></td><td></td><td></td><td></td></tr><tr><td>No personal elaboration of material</td><td>X</td><td>X</td><td>X</td><td></td><td></td><td>X</td><td></td><td></td></tr><tr><td>Cannot reorganize information to create own personal meanings</td><td>X</td><td>X</td><td></td><td></td><td></td><td>X</td><td></td><td></td></tr><tr><td>No inherent way to show relationships</td><td>X</td><td>X</td><td>X</td><td>X</td><td></td><td></td><td></td><td></td></tr><tr><td>May need training to use strategy</td><td></td><td></td><td></td><td></td><td>X</td><td></td><td>X</td><td></td></tr><tr><td>Strategy format may be cumbersome</td><td></td><td></td><td></td><td></td><td>X</td><td></td><td>X</td><td>X</td></tr><tr><td>Material may not be suitable to hierarchy</td><td></td><td></td><td></td><td></td><td>X</td><td>X</td><td></td><td></td></tr><tr><td>May be difficult to incorporate non-text into outline or map</td><td></td><td></td><td></td><td></td><td>X</td><td></td><td>X</td><td></td></tr><tr><td>User may not be graphically oriented</td><td></td><td></td><td></td><td></td><td></td><td></td><td>X</td><td>X</td></tr><tr><td>Superfluous information may be included</td><td>X</td><td></td><td></td><td></td><td></td><td>X</td><td></td><td></td></tr><tr><td>Inaccurate information may be included</td><td></td><td></td><td>X</td><td>X</td><td></td><td></td><td>X</td><td>X</td></tr></tbody></table>
Notebooks can even be provided which allow learners to use them to create more elaborate multimedia presentations. The notebook could be used as an organizer for drafting such a presentation and, ideally, would make the transition to a finished multimedia product seamless.

The previous review of research has provided evidence that the combination of the encoding and external storage function of notetaking is most valuable. Electronic notebooks provide learners with the capability to gather and later review organized and elaborated information. This allows them to process the information again and assists by cuing retrieval. The generative learning hypothesis (Wittrock, 1974), in particular, supports the use of a notebook to copy text, graphics, etc. and then to add one's own elaborations. Learners can then arrange information into their own framework, be it outline or spatial map. This allows for both repetition of the information and additional processing. Notebooks can also be used to supply learners with teacher-provided questions, outlines, frameworks, or keywords. Any of these strategies can act as an advance organizer for the learner, especially those with low prior knowledge. They can provide a framework for gathering information and stimulate recall of prior knowledge. Learners could also be asked to generate their own outline or framework of prior knowledge before commencing interactive multimedia instruction in order to stimulate recall and provide hooks for integrating the new information. They could even generate their own questions to be answered at the start and fill in the blanks as they go through the program.

## Disadvantages

While notetaking in interactive multimedia offers these varied expanded capabilities, designers are advised to use them with caution. The notetaking research reviewed above has also indicated that learners who elaborate on the information recorded interact more with the new material and link it to their existing knowledge. Learners need to go beyond mere recording of information to organizing and elaborating on the program's information to help them to encode it, relate it to prior knowledge, and to make the new information personally relevant. While elaboration may be an effective learning strategy, designers providing such features in a notebook may well find that learners need to be convinced to use them. They may also find that learners using complex materials such as interactive multimedia instruction prefer to read and reread the information as their strategy of choice, even in lieu of standard notetaking. Approaches like outlining, and especially concept mapping, may not be in learners' repertoires of strategies. Even if learners are aware of a particular strategy, they may tend to use those they feel most comfortable with, even if it may not be the best for the situation.

A notebook, then, may be provided, but not used especially in the case of a notebook that provides some of the more sophisticated features such as concept mapping. Designers incorporating such features are advised to include an introduction to the particular elaborative technique, or even tutorial instruction,
to ensure that learners use the notebookin other than simple cut and paste mode. Such guidance could be made available and then extinguished as the learner becomes more comfortable with its use. No matter what type of notebook is incorporated, there is a need to teach use of the learning strategy. Students using even simple notetaking have difficulty selecting the major concepts and includ- ing the appropriate level of detail. Many simply copy verbatim and never elaborate with their own prior knowledge, a strategy provenineffective. Outlin- ing and concept mapping are even more challenging. Unless the strategies are understood and carried out correctly, they will be ineffective, considered burden- some, and probably not used.

REFERENCES

Anderson, T.H. (1980). Study strategies and adjunct aids. In R.J. Spiro, B.C. Bruce, and W.F. Brewer (Eds.), *Theoretical issues in reading comprehension*. Hillsdale, NJ: Lawrence Erlbaum.

Anderson-Inman, L., Redekopp, R. &Adams, V. (1992). Electronic studying: Using computer-based outlining programs as study tools. *Reading and Writing Quarterly: Overcoming Learning Difficulties*, 8(4), 337-358.

Anderson-Inman, L. & Zeitz, L. (1994). Beyond notecards: Synthesizing information with electronic study tools. *Computing Teacher*, 21(8), 21-25.

Anderson-Madaus, L. (1990). *Cognitive mapping as a 'bridge' activity*. Paper at the annual meeting of the International Reading Association (35th, Atlanta, GA, May 6-11,1990) (ERIC Document Reproduction Service No. ED 320140).

Annis, L. & Davis, J.K. (1975). The effect of encoding and an external memory device on note taking. *Journal of Experimental Education*, 44, 44-46.

Arnold, H.F. (1942). The comparative effectiveness of certain study techniques in the field of history. *Journal of Educational Psychology, 33*(5), 449-457.

Barton, W.A., Jr. (1930). *Outlining as a study procedure*. NY: Columbia University Bureau of Publications.

Bean, T.W., Singer, H., Sorter, J., & Frazee, C. (1983). Acquisition of summarization rules as a basis for question generation in learning from expository text at the high school level. In J.A. Niles (Ed.), *Searches for Meaning in Reading/Language Processing and Instruction, Thirty-second yearbook of the National Reading Conference*. Rochester, NY: National Reading Conference.

Bianco, L. & McCormick, S. (1989). Analysis of effects of a reading study skill program for high school learning-disabled students. *Journal of Educational Research, 82*(5), 282-288.

Branthwaite, A., Trueman, M. &Hartley, J. (1980). Writing essays: The actions and strategies of students. In J. Hartley (Ed.), *The psychology of writing* (pp. 98-109). London: Nichols.

Bruning, I.L. (1983). An information-processing approach to a theory of instruction. *Educational Communications and Technology Journal, 31*,1-101.
Carrier, C.A. & Titus, A. (1979). The effects of notetaking: A review of studies. *Contemporary Educational Psychology, 4*, 299-314.

Cohen, J. (1988). *Statistical power analysis for the behavioral sciences*. Hillsdale, NJ: Lawrence Erlbaum Associates.

Cook, L.K. & Mayer, R.E. (1983). Reading strategy training for meaningful learning from prose. In M. Pressley & J. Levin (Eds.), *Cognitive strategy training*. NY: Verlag.

DiVesta, F.J. & Gray, S.G. (1972). Listening and note-taking. *Journal of Educational Psychology, 63*, 8-14.

Dynes, J.J. (1932). Comparison of two methods of studying history. *Journal of Experimental Education, 1*, 42-45.

Emig, J. (1971). *The composingprocessoftwelfthgraders* (ResearchReportNo. 13). Urbana, IL: National Council of Teachers of English.

Emig, J. (1975). The composing process. In R. Winterowd (Ed.), *Contemporary Rhetoric, a Conceptual Background with Readings*. NY: Harcourt Brace Jovanovich.

Galbreath, J. (1992). The educational buzzword of the 1990's: Multimedia, or is it hypermedia or interactive multimedia, or.. ? *Educational Technology, 32*(4), 15-19.

Gayeski, D.M. (1993). Making sense of multimedia: Introduction to this volume. In D.M. Gayeski (Ed.), *Multimedia for Learning: Development, Application, Evaluation*. Englewood Cliffs, N.J: Educational Technology Productions.

Glass, G.V., McGaw, B., & Smith, M.L. (1981). *Meta-analysis in social research*. Beverly Hills, CA: Sage Publications.

Glynn, S.M., Britton, B.K., & Muth, K.D. (1985) Text comprehension strategies based on outlines: Immediate and long-term effects. *Journal of Experimental Education, 53*(3), 129-135.

Glynn, S.M. & DiVesta, F.V. (1977). Outline and hierarchical organization as aids for study and retrieval. *Journal of Educational Psychology, 69*, 89-95.

Hartley, J. & Davies I.K. (1978). Note-taking: A critical review. *Programmed Learning and Educational Technology, 15*(3), 207-224.

Heinze-Fry, J.A. & Novak, J.D. (1990). Concept mapping brings long-term movement toward meaningful learning. *Science Education, 74*(4), 461-472.

Hoffler, D.B. (1983). Outlining-Teach the concept first. *Reading World, 23*, 176-177.

Hoon, P.W. (1974). The efficacy of three common study methods. *Psychological Reports, 35*(3), 1057-1058.

Horton, P.B., McConney, A.A., Gallo, M., Woods, A.L., Senn, G.J. & Hamelin, D. (1993). An investigation of the effectiveness of concept mapping as an instructional tool. *Science Education, 77*(1), 95-11.
Iovino, S.F. (1989). The *effect of dominant hemispheric processing modes and notetaking strategy on the comprehension and retention of academically underprepared college readers*. Paper presented at the annual meeting of the College Reading Association (33rd, Philadelphia, PA, November 3-5, 1989) (ERIC Document Reproduction Service No. ED 333337).

Jonassen, D.H. (1985). Integrating learning strategies into courseware to facilitate deeper processing. In David H. Jonassen (Ed.), *The Technology of Text, Volume Two: Principles for Structuring, Designing and Displaying Text*. Englewood Cliffs, NJ: Educational Technology Publications, Inc.

Jonassen, D.H. (1991). Objectivism versus constructivism: Do we need a new philosophical paradigm? *Educational Technology: Research and Development* 39(3), 5-14.

Kellogg, R.T. (1988). Writing performance: Effects of rough draft and outline strategies. *Journal of Experimental Psychology: Learning, Memory, and Cognition, 14*, 355-365.

Kellogg, R.T. (1990). Effects of prewriting strategies as a function of task demands. *American Journal of Psychology, 103*(3), 327-342.

Kenny, R.F. (in press). The generative effects of instructional organizers with computer-based interactive video. *Journal of Educational Computing Research*.

Kenny, R.F., Grabowski, B.L., Middlemiss, M. & Van Neste-Kenny, J. (1991). *The generative effects of graphic organizers with computer-based interactive video by global and analytic thinkers*. Paper presented at the annual meeting of the Association for the Development of Computer-Based Instructional Systems. (St. Louis, MO, November 1991) (ERIC Document Reproduction Service No. ED 341378).

Kiewra, K.A. (1985). Investigating note taking and review: A depth of processing alternative. *Educational Psychologist, 20*, 23-32.

Kiewra, K.A. (1987). Notetaking and review: The research and its implications. *Instructional Science, 16*, 233-249.

Kiewra, K.A. (1988). Notetaking and review strategies: Theoretical orientations, empirical findings and instructional practices. *Research and Teaching in Developmental Education, 4*(2), 5-17.

Kiewra, K.A., DuBois, N.F., Christensen, M., Kim, S. & Lindberg, N. (1989). A more equitable account of the note-taking functions in learning from lecture and from text. *Instructional Science, 18*, 217-232.

Kiewra, K.A., DuBois, N.F., Christian, D. & Mcshane, A. (1988). Providing study notes: Comparison of three types of notes for review. *Journal of Educational Psychology, 80*(4), 595-597.

Kiewra, K.A., DuBois, N.F., Christian, D., Mcshane, A., Meyerhoffer, M. & Roskelly, D. (1991). Note-taking functions and techniques. *Journal of Educational Psychology, 83*(2), 240-245.

Kiewra, K.A., Mayer, R.E., Christensen, M., Kim, S. & Risch, N. (1991). Effects of repetition on recall and note-taking: Strategies for learning from lectures. *Journal of Educational Psychology, 83*(1), 120-123.
Kozma, R. B. (1987). The implications of cognitive psychology for computer-based learning tools. *Educational Technology, 27,20.

Matott, G. (1987). Linear composing, discourse analysis, and the outline. *Journal of Advanced Composition, 4*, 33-45.

McMahon, T.A., Carr, A.A. & Fishman, B.J. (1993). Hypermedia and constructivism: three approaches to enhanced learning. *Hypernexus: Journal of Hypermedia and Multimedia Studies*, 3(2), 5-10.

McCarthy, R. (1989). Multimedia: What the excitement is all about. *Electronic Learning*, 8(3), 26-31.

Novak, J.D. (1990). Concept maps and vee diagrams: Two metacognitive tools to facilitate meaningful learning. *Instructional Science*, 19, 29-52.

Novak, J.D. & Gowin, D.B. (1984). *Learning how to learn*. New York: Cambridge University Press.

Novak, J.D., Gowin, D.B. & Johansen, G.T. (1983). The use of concept mapping and knowledge vee mapping with junior high school science students. *Science Education, 67*(5), 625-645.

Palmatier, R. A. (1971). Comparison of four note-taking procedures. *Journal of Reading, 14*, 235-240.

Peper, R.J. & Mayer, R.E. (1986). Generative effects of note-taking during science lectures. *Journal of Educational Psychology, 78*(1), 34-38.

Reder, L.M. (1980). The role of elaboration in the comprehension and retention of prose: A critical review. *Review of Educational Research*, 50(1), 5-53.

Rickards, J.P. (1979). Notetaking: Theory and research. *Improving Human Performance Quarterly*, 8(3), 152-161.

Rieber, L.P. (1992). Computer-based microworlds: A bridge between constructivism and direct instruction. *Educational Technology: Research and Development, 40*(1), 93-106.

Risch, N.L. & Kiewra, K.A. (1990). Content and form variations in note taking: Effects among junior high students. *Journal of Educational Research, 83*(6), 355-357.

Schwier, R.A. & Misanchuk, E.R. (1993). *Interactive multimedia instruction*. Englewood Cliffs, NJ: Educational Technology Productions.

Shimmerlik, S.M. & Nolan, J.J. (1976). Reorganization and the recall of prose. *Journal of Educational Psychology, 80*, 776-786.

Slater, W., Graves, M. F., & Piche, G.L. (1985). Effects of structural organizers on ninth grade students' comprehension and recall of four patterns of expository text. *Reading Research Quarterly, 20*,189-202.

Snyder, V. (1984). A comparison of the SQ3R method with two other traditional methods for studying college textbook chapters. *Reading Improvement, 21*, 244-248.

Stordahl, K.E. & Christensen, C.M. (1956). the effect of study techniques on comprehension and retention. *Journal of Educational Research, 49*(8), 561-570.
Tenny, J.L. (1988). A study of the effectiveness of computer-mediated and rereading study procedures. Unpublished doctoral dissertation, University of Oregon, Eugene.

Todd, W. & Kessler, C.C. (1971). Influence of response mode, sex, reading ability and level of difficulty on four measures of recall of meaningful written material. *Journal of Educational Psychology, 62, 229 - 234*.

Tuckerman, B.W. (1993). *The coded elaborative outline as a strategy to help students learn from text*. Paper presented at the annual meeting of the American Educational Research Association (Atlanta, GA, April 12-16, 1993). (ERIC Document Reproduction Service, ED 357075).

Wade, S.E. & Trathen, W. (1989). Effect of self-selected study methods on learning. *Journal of Educational Psychology, 81*(1), 40-47.

Weinstein, C.E. (1982). Training students to use elaboration learning strategies. *Contemporary Educational Psychology, 7*, 301-311.

Weinstein, C.E. & Mayer, R.E. (1986). The teaching of learning strategies. In M.C. Wittrock (Ed.). *Handbook of Research on Teaching* (3rd ed.) NY: Macmillan.

Wittrock, M.C. (1974). Learning as a generative process. *Educational Psychologist, 11*, 87-95.

Wolf, F.M. (1986). Meta-analysis: Quantitative methods for research synthesis. Beverly Hills, CA: Sage Publications. Wolf, F.M. (1990). Methodological observations on bias. In K.W. Wachter & M.L. Straf (Eds.), *The future of meta-analysis*. New York, NY: Russell Sage Foundation.

## AUTHORS

Eileen E. Schroeder is Assistant Professor, Educational Foundations, University of Wisconsin-Whitewater, Whitewater, WI 53190.

Richard F. Kenny is Assistant Professor, Faculty of Education, University of Ottawa, 145 Jean-Jacques Lussier, Ottawa, ON KIN 6N5.
Engagement as a Design Concept for
Multimedia

Richard Jacques
Jenny Preece
Tom Carey

**Abstract:** Developers of educational multimedia software aim to design presentations that will encourage and facilitate students' learning. The paper describes how an understanding of "engagement" can be applied to aid in the design process. Engaging interactions make multimedia systems more attractive in ways that support learning goals. We provide a detailed explanation and a taxonomy for engagement based on practical work from our laboratories. We give examples of interactions that are found to be engaging, suggest methods for evaluating their impact on the learner and practical hints for incorporating them into multimedia system design.

**Résumé:** Les concepteurs de logiciels pédagogiques multimédia doivent élaborer des présentations qui encouragent et aident les étudiant à apprendre. Cet article décrit comment la compréhension du terme «engagement» peut aider au processus de conception. En encourageant les interactions, on rend les systèmes multimédia plus attrayants tout en favorisant l’apprentissage. Nous apportons ici des explications détaillées et une taxinomie du terme «engagement» basées sur le travail pratique fait dans nos laboratoires. Nous apportons des exemples d’interactions que nous considérons comme «engageantes» et nous faisons quelques suggestions sur les méthodes à utiliser pour en évaluer l’impact sur l’apprenant. Nous offrons également des conseils pratiques pour leur incorporation au processus de conception des systèmes multimédia.

INTRODUCTION

Throughout our lives certain things, people and events attract us and hold
our attention more than others. Sometimes we can explain why and other
times we cannot. This paper argues that a greater understanding of the
reasons why some things we encounter "engage" us more than others will help
developers of educational multimedia systems to produce more successful
designs. Specifically designing for *engaging* interactions has been shown to
encourage and facilitate learning. Adelson (1992) for example, used "evoca-
tive agents" in a multimedia application to provide an "engaging but not
distracting environment for learning to analyze and argue with individuals from a foreign culture" (p.356). Hsi & Agogino (1993) engaged students with their multimedia system, using "interactive pop-up *think* questions...to stimulate reflection and critical thinking" (p.257).

To date, there has been little attempt to define, analyse or evaluate *engagement* with computer-based learning materials. Skelly (1991) suggests that an *engaging* computer interface is “seductive” as it will “draw in the user and make interaction with the computer a fulfilling experience” (p.3). Laurel (1991) recognises the benefits of making interactions “pleasurably engaging”, saying it is a “desirable — even essential — human response to computer mediated activities” (p. 112).

This paper aims to extend the current understanding of *engagement* by discussing examples of work carried out in our laboratories. Both laboratories design and evaluate the usability of systems with a human-computer interface. The paper provides a definition and a taxonomy of engagement, as well as practical hints on how designers can evaluate *engaging* interactions and incorporate them into their work.

ENGAGEMENT AND MULTIMEDIA

Learners' motivation to use computer systems can stem from two sources: *intrinsic* and *extrinsic* goals (Malone, 1980). If their reason for use comes from external influences such as obtaining good grades or peer pressure, they are *extrinsically motivated*. If the reason originates from their own willingness or desires, they are said to be *intrinsically motivated*.

While good multimedia system designers will appeal to both kinds of motivation, this paper is concerned with engagement arising primarily from intrinsic motivation. That is, systems which encourage and support users' attention without external pressure. Inherently rewarding interactions increase learners' intrinsic motivation to continue, which can lead to a state of *flow* a condition in which "people are so involved in an activity that nothing else seems to matter" (Csikszentmihalyi, 1992, p.4). However, too much engagement can be a disadvantage. Games for example, can be very compelling (Shotton, 1989) and distract users from their initial learning goal. Conversely, systems that become unattractive during an interaction and do not help to reach learning goals, can lose users' attention, cause them to become *disengaged* and perhaps even conclude their involvement.

Well designed educational multimedia is balanced; it should provide the content and functionality to satisfy learners' needs in a manner that is attractive, yet not distracting. The next section describes some practical examples of design features affecting participants' motivation and engagement.
PRACTICAL EXAMPLES OF ENGAGEMENT

The following examples come from four practical studies conducted in our laboratories.

Study 1 (Jacques, 1994a) asked ten novice multimedia users to interact with a multimedia CD-ROM presentation called Great Wonders of the World (GWW). GWW was a software innovation award winner in 1992. It provides history and travel information about ten Wonders of the World using a variety of media, including video, audio, photographs, graphics and hypertext. Figure 1 is a typical menu page for a Wonder of the World and shows the different categories of information and media available to users.

The participants were individually asked to explore GWW by browsing at will and for as long as they wanted. Throughout their interaction they were asked to speak out aloud their thoughts (i.e. provide a "verbal protocol"; Ericsson & Simon, 1993). If they were quiet for too long, the observing Investigator asked them, "What are you thinking?", although avoided entering into dialogue. The study was video recorded to aid the analysis of their comments later on.

<figure><img src="image_12.png" /><figcaption>Figure 1. The Pyramids menu page in Great Wonders of the World</figcaption></figure>

Study 2 (Jacques, 1994b) also used the software Great Wonders of the World. Eighteenvolunteersparticipated, whose terms was rated as "average" on a five point Likert scale from "none" to
"extensive". They were individually asked to spend up to fifteen minutes on each of three tasks common to hypertext type systems (Marchionini & Shneiderman, 1988): browsing (for an overview of the subject content), closed search tasks (find a single answer) or open search tasks (find as many constituents to an answer as possible). After each task, they were asked to complete a questionnaire about their subjective reactions. The questionnaire was mainly in a "check the appropriate box" type of format.

Study 3 (Nonnecke et al, 1995) examined the use of video clips for a future multimedia system. Seven video clips (from 12 to 90 seconds duration) were taken from a six minute video, with the intention of capturing the salient aspects of the storyline. Participants in the study were asked to run and control each clip in sequence and to write down any questions they had about them. Eighteen individuals, typically with "a little" or "average" knowledge of the domain subject, computer software design, participated.

Study 4 (van Aalst et al, 1995), evaluated the learning potential of a multimedia teaching aid recently developed in the laboratory at Guelph. The multimedia module is called FLUID and provides a Framework for Learning User Interface Design. It combines a concept table, tutorials, design work-bench, library and case studies using text, audio, video and graphics.

Six students with classroom experience of systems analysis and design were given a learning goal and asked individually to meet it using FLUID. Afterwards, they were given short written questions. Throughout the study, they were observed by two investigators who briefly interviewed them afterwards to determine their subjective reactions to its usability and those design features which they did, or did not like.

All the participants' comments and investigators' observational notes from the four studies have been collated. By a process of comparison, it is possible to categorise each of them into one of the following three groups: content, task or media. The distribution into each category is not of significance for this paper as each study had a different objective. For example, Study 2 aimed to establish participants' preferences for different task types, so the content and media were commented upon less. While in Study 1, the participants undertook only one task, browsing, so their comments were directed more towards the content and the media. It is therefore not possible to conclude that one of the three categories is of greater significance to design than the others. An explanation of the three categories is given below using examples from the studies most pertinent to them.

## Content

The term content is used to categorise participants' comments about the subject material of the multimedia system. Such comments were particularly prevalent in Study 1, in which participants were invited to browse at will through the multimedia presentation, Great Wonders of the World. Participants would frequently justify their style and choice of interaction in terms of the subject material. For example, "I like the Pyramids section best because I am interested
in Ancient Egypt". Likewise, the information categories within each World Wonder (e.g. Figure 1) were frequently chosen and judged in terms of their content. A typical comment was, "I'm enjoying this because I like History". The method of presenting the subject material was also a factor; for example, "I like the facts; they're interesting". Educational content is commonly provided in the form of facts, concepts and skills.

In terms of the content, one World Wonder was not largely more popular than another. This is probably because participants' reactions to content are varied and often diverse, based on personal and individual preferences.

## Task

The type of task participants' undertook in our studies was found to have an impact on their opinion of a particular multimedia system they were using. The purpose of Study 2 was to discover their preferences among three tasks; browsing (to gain an overview of the subject content), closed searching (find a single answer) and open searching (find as many constituents to an answer as possible). The results indicate browsing to be the most popular, with typical reasons being "more freedom" and "less pressure"; although some complained they were bored and felt unsure about what they were doing. The closed search task was the next most popular. Some said they preferred it over browsing because it offered more of a challenge. The open search task was the least popular, as participants said they were unsure whether they had all the answers and when to finish.

A few participants commented that if they were to use the same multimedia system again their reactions would be different. Browsing, they said was good for a first time interaction, but with subsequent use they would prefer more of a specific goal. No-one remarked that a particular task was not suitable to Great Wonders of the World, indicating that it was a reasonable choice of software for the study.

## Media

Participants' engagement with the media used in a particular system falls into three divisions: *type*, *presentation* and *control*.

The influence of the *type* of media available to participants was demonstrated

three of the studies. In Study 1, participants reported a strong preference for visually based media such as video, photographs or animations. Our observations support this, as they clearly chose these media more frequently and for longer than the text or audio based ones, even though they were free to use the system at will. In Study 2, however, the participants said they preferred text based media for the search tasks. They explained that although they enjoyed the video and slides, they were often too slow to appear on the screen, sometimes of poor quality and lacked the control they would have liked. When undertaking a search, they said they found it easier to skim back and forth through text, looking for relevant information rather than replay audio or visual sequences. A similar result was found in Study 3, where many participants said they would like to have the information from the video clips available in an alternative
format. Popular choices were on-line text and books. Although we acknowledge this is dependent on the content and the quality of video, it does suggest that the inclusion of video in multimedia presentations is not always an engaging feature. It does however, seem a popular choice when participants have "more freedom" and "less pressure" to browse through the system at will.

The *presentation* of the media, such as the style of the typeface, use of colour in graphics and quality of sounds were found to influence participants' opinions of the system in all the studies. In Study 4 for example, FLUID uses many short video sequences (of up to 2 minutes) of head and shoulder views of "experts" to help the user reach their learning goal. In the early development of FLUID, the video was presented in "pop-up" QuickTime windows at the center of the page. In subsequent versions, this was changed by removing the window surrounding the video image and placing the experts elsewhere (see Figure 2). The experts then became fully integrated into the text and could interact with it. The expert in the bottom left hand corner of the page in Figure 2 moves in front of and behind the text, then points at words or buttons to illustrate what she is saying. Throughout the evaluation stage, participants commented on how much they liked the integrated video, often replaying them to specifically see the sequence again. They found these video clips engaging.

<figure><img src="image_13.png" /><figcaption>**Figure 2.**
The Person in the Bottom Left-Hand Corner of this Page from FLUID is Part of a Motion Video Interacting with the Text Behind</figcaption></figure>
The effects of presentation can equally disengage and demotivate learners. In Studies 1 and 2, some participants complained of poor video quality and refused to use it, saying that the QuickTime window was "too small and grainy". This is a technical constraint and is one that designers must consider carefully; it may be better not to use video at all, than to use it poorly.

Throughout the practical work, we noticed that the multimedia presentations affording a poor degree of control and navigational support for the user caused many complaints. The issue of control presents itself to users early on. If they do not feel the system is navigable or are uncomfortable with its structure, they will not be sufficiently engaged to continue using it. The slides' category in GWW illustrate this point. Each category consists of around 15 narrated slides, all having forward, reverse and return buttons in the bottom right hand corner (see Figure 3). When each narration is complete (after 10 to 40 seconds), the program automatically advances to the next slide. This caused many participants to comment and they were divided as to whether they liked it. Some said they enjoyed being able to sit back from the computer and allow the slide show to run, while others were frustrated and wanted to view each one in their own time. Like most engaging or disengaging features, their reactions are dependent on other factors related to their motives for use. In this instance, it seems that the participants were influenced by their task; they preferred the automatic slide sequence if they were undertaking a free browse, but when searching for a specific piece of information they wanted to control it themselves.

<caption>Figure 3.</caption>

*A Slide from Great Wonders of the World Showing the Forward, Reverse and Return Buttons in the Bottom Right Hand Corner*

<figure><img src="image_14.png" /></figure>
engaging in an unproductive way: students who encountered it enjoyed it immensely and went on to try the patience of the other experts to see if they had similar responses. This wasted time that could have been spent on the task; it also wasted the contextual advice the experts video clips offered at that point in the exercise.

The most assured way to produce successful educational multimedia is to make the design user centred and the process of its evaluation iterative. Evaluating for learners' engagement is an important component in this formula. Engagement describes their intrinsically motivated attraction and is expressed in cognitive, behavioural and affective terms. We suggest that it is difficult and impractical to consider them all at one instance and the most rewarding approach is to ask users for their subjective reactions. Popular methods to be considered are verbal protocols (Ericsson & Simon, 1993) and observation during interaction, and interviews or questionnaires afterwards. It is not possible to truly determine levels of engagement by examination of learners' knowledge: users who know they are to be examined may become sufficiently motivated extrinsically to learn the subject matter, without having found the interaction engaging at all. Additionally, a measurement of time spent on the interaction is not a good determinant, as some users take longer but are equally engaged, while some take longer because they are bored and not paying attention.

<figure><img src="image_15.png" /><figcaption>**Figure 4.**
A Taxonomy of the Factors that Engage Learners with Educational Multimedia</figcaption></figure>
Evaluators have a further decision: either to examine learners' engagement with particular features of the multimedia system such as the presentation style or interest in the content, or take a holistic approach and analyse their subjective reactions to the system overall. We recommend at least part of the evaluation in the iterative process be holistic as the "effect of multimedia is more than just the sum of its parts" (McKerlie & Preece, 1993). Consideration must also be given to the fact that features engaging learners the first time, may do so because they are novel; in subsequent interactions they may lose their appeal.

## SUMMARY

Learners are "engaged" with educational multimedia when it holds their attention and they are attracted to it for intrinsic rewards. If their engagement is in alliance with learning goals, then the pedagogical potential of the system is increased. To facilitate this, designers should consider the *tusks* users will undertake, the *content* they need and the effectiveness of the media available to them. A user centred design and an iterative approach for evaluating these factors both individually and collectively is recommended. Well designed educational multimedia systems will draw learners in, motivate interaction and help them accomplish learning goals without distraction.

## REFERENCES

* Adelson, B. (1994, May). Evocative agents and multimedia interface design. *Proceedings. of ACM CHI 92: New York: ACM Press, 351-356.*

* Csikszentmihalyi, M. (1992). *Flow: the psychology of happiness*. Harper & Row, New York.

* Ericsson, K. & Simon, H. (1993). *Protocol Analysis: Verbal reports as data*. MIT Press, MA.

* Hsi, S. & Agogino, A. (1993, June). Creating excitement and motivation in engineering design: Developing and evaluating student participatory experience in multimedia case studies In H. Maurer (Ed.), *Educational multimedia and hypermedia annual*, 1993. *Proceedings. of ED-MEDIA 93*, Charlottesville, VA: AACE Publications.

* Jacques, R., Nonnecke, B., Preece, J. & McKerlie, D. (1993). Current designs in hypercard: What can we learn? *Journal of Educational Multimedia and Hypermedia, 2*, (3), 219-237.

* Jacques, R. (1994a). *Subjective reactions to a hypermedia system: An evaluation study* (Technical Report), London, U.K.: South Bank University, Centre for People and Systems Interaction.

* Jacques, R. (1994b). *Subjective reactions to different tasks with a hypermedia system: An Evaluation Study* (Technical Report), London, U.K.: South Bank University, Centre for People and Systems Interaction.
Laurel, B. (1991). *Computers as theatre*. Reading, MA: Addison-Wesley Publishing Co. Inc.

Laurel, B., Oren, T. & Don, A. (1992). Issues in multimedia interface design: Media integration and interface agents. In M. Blattner & R. Dannenberg (Eds.), *Multimedia Interface Design* (pp. 53-65). New York: ACM Press.

Laurillard, D. (1993). *Rethinking university education: A framework for the effective use of educational technology*. London: Routledge.

Malone, T.W. (1980). *What makes things fun to learn? A study of intrinsically motivating computer games* (Cognitive and Instructional Science Series, Vol. CIS-7, SSL-80-11). Palo Alto, CA: Xerox.

Marchionini, G. & Shneiderman, B. (1988). Finding Facts vs. Browsing Knowledge in Hypertext Systems. *IEEE Computer*, 21(1), 70-80.

McKerlie, D. & Preece, J. (1993). The hype and the media: Issues concerned with designing hypermedia. *Journal of Microcomputer Applications*. 16 33-47

Nonnecke, B., Jacques, R., McKerlie, D. & Preece, J. (1995). Video-based hypermedia: Guiding design with users' questions. Manuscript submitted for publication.

Shotton, M. (1989). *Computer addiction? A study of computer dependency*. London, U.K.: Taylor & Francis Ltd.

Skelly, T. (1991, April). *The Design of Seductive Interfaces*. Tutorial presented at ACM CHI 91.

van Aalst, J., Carey, T. & McKerlie, D. (1995, May). *Design space analysis as training wheels for user interface design*. Paper presented at the meeting of ACM CHI 95, Denver, CO.

## AUTHORS

Richard Jacques is a PhD student at the Centre for People and systems Interaction, South Bank University, London, SE1 0AA, U.K.

Jenny Preece is Professor, C.P.S.I., South Bank University, London, SE1 0AA, U.K.

Tom Carey is the Director, HCI Lab, Department of Computing and Information Science, and Director, Learning Technologies, Teaching Support Services, at the University of Guelph, Guelph, ON, N1G 2W1

## ACKNOWLEDGEMENTS

Great Wonders of the World: Man Made, Volume 1 is copyright of InterOptica Publishing Ltd, San Francisco and Hong Kong. QuickTime is a trademark of Apple Computer, Inc.
Using Video-On-Demand for
Educational Puposes: Observations
from a Three Month Experiment

Pierre C. Bélanger
Sébastien Clément

**Abstract:** Mosf recent technological developments in the field of communica-
tion and telecommunication seem to relate, one way or another, to the much
heralded Information Superhighway metaphor, this electronic Infobahn of the
future which promises to link us all to a universal global village. One such
technology is Video-On-Demand (VOD), a technology that allows viewers to
watch an audiovisual document of their choice, at their convenience. Among
the companies advocating the introduction of Video-On-Demand is Stentor
Resource Centre Inc., a consortium of Canada's lorgest telephone companies.
In January 1994, Stentor and Bell Canada launched an experiment involving
the transmission of video content over telephone lines. The trial was conducted
with the collaboration of Carleton University und the University of Ottawa. At
both universities, students used the technology as part of an official course
curriculum. This paper describes the quasi-experimental conditions in which
17 students from the University of Ottawa were involved. Over a three-month
period, users' reactions toward the system were examined. The results indicate
a definite occurrence, over time, of the Hawthorne effect, as students rated
the technology much more positively at the beginning of the trial than they did
at the conclusion. The VOD system as tested holds some interesting potential
in many applications. Future developments of the technology must be sensitive
to the needs of the user in order to maximize its educational merits and
competitiveness.

Résumé: Le concept d'autoroute électronique est rapidement devenu un véritable paradigme en voie de conditionner une bonne partie de lu recherche et du développement reliés aux secteurs des communications et des télécommunications. Il semble en effet que le village global, loin de ressembler aux villages romantiques d'antan, sera vraisemblablement irrigué pur une infrastructure électronique comparable aux INFOBAHNS sur lesquelles on peut filer sans limitation de vitesse. Encouragées pur les gouvernements qui ont avalisé le projet, plusieurs entreprises se bousculent pour revendiquer une priorité d'accès à cette fameuse voie rapide de l'information. Les principales compagnies canadiennes de téléphonie, regroupées sous l'égide du consortium Stentor Resource Centre Inc., participent à la course. Le système Vidéo-à-lu-demande, une technologie qui permet à l'usager de visionner à son gré les documents audiovisuels de son choix, constitue l'un de ces premiers véhicules actuellement mis à/''essai. Stentor et Bell Canada lancèrent la foute première expérience canadienne de transmission vidéo entièrement numérique pur fils téléphoniques en janvier 1994. L'opération a été
conduite dans la région de la capitale nationale en collaboration avec l'Université Carleton et l'Université d'Ottawa. Des étudiant-e-s inscrit-e-s à chacune de ces institutions partenaires ont été invité-e-s à utiliser le système dans le cadre de cours qui faisaient partie de leur programme respectif. Nous faisons état ici de l'expérience menée à l'Université d'Ottawa où les réactions de ces premiers utilisateurs ont été observées durant une période de trois mois. Les résultats confirment clairement l'effet d'Hawthorne selon lequel l'enthousiasme original se dissipe progressivement au fur et à mesure de la fréquentation d'une nouvelle technologie. Si le prototype présente d'indéniables atouts sur le plan des applications éducationnelles, il n'en demeure pas moins que ses promoteurs devront être attentifs aux besoins des clientèles potentielles s'ils veulent se maintenir dans le peloton de fête.

## INTRODUCTION

Judging by the nature of the discussions that took place among the Group of Seven industrialized countries at their February 1995 summit in Brussels and at the Canadian Radio-television and Telecommunications Commission public hearings in Hull, QC the following month on the structuring of the Canadian Information Highway, the roads of the future will rest upon intricate networks of fiber optics, microwaves as well as cable, telephone and cellular networks (Industry Canada, 1994b). It appears as though every new technological development in the field of communication and multimedia areas is connected in one way or another to this infrastructure, which is supposed to lead us into the 21st Century. Ostensibly, any company with an interest in communication technology will try its best to get a part of the multibillions of dollars that have been promised to be injected in this global telecommunications project. The current situation in the Canadian long-distance telephone market is a most eloquent example. It is not surprising that Bill Clinton emphasized the Information Highway concept in his last Presidential campaign. In the United States and in most other Western countries, where thousands of jobs are being lost, foreign and local investors are frugal, and most sectors of the economy are stagnant. As an incentive to restore confidence in the economy, the Clinton administration proposes a global infrastructure project that is akin to the building of the interstate highway network of the '60s (Information Infrastructure Task Force, 1993). The Information Highway project has been touted as a way to stimulate the economy, and it is already showing signs of paying off. Major corporations are spending billions of dollars in an attempt to place their wares on the expressway (Stentor, 1994). In order to be the first to "put some rubber" on the Infobahn¹, the big multimedia companies are entering into strategic alliances with infrastructure providers (cable companies, telecommunications companies).

Aside from deciding which enterprises will provide the services, communication regulatory bodies are also trying to determine what kinds of services are to be offered, and at what price. Possible services include home-shopping, telebanking, video games, information retrieval and dissemination services, electronic mailing, and à-la-carte entertainment (Industry Canada, 1994a). In this plethora of multimedia services, Video-On-Demand (VOD) consistently
figures inmost blueprints. Such a system would allow users to watch any movies at their convenience. VOD, in its current state of development, offers ample possibilities for situations where the use of audiovisual documents is commonplace. A variety of applications can be devised around VOD systems and this partly explains why so many promoters are including this type of product in their first Information Highway development plans. One such corporation is Stentor a consortium comprising Canada's nine largest telephone companies. In collaboration with Bell Atlantic in the United States and their respective research and development sibling organizations, Stentorpioneered a new prototype technology which allows the transmission of an audiovisual signal through standard telephone lines.

In Canada, as well as in the United States, telecommunications providers have always been forbidden to transmit video signals and own cable systems. In July of 1992, the US Federal Communications Commission (FCC) ruled that telecommunications providers should be allowed to transmit video signals on a trial basis. The service, which the FCC referred to as Video Dialtone had to be offered in a nondiscriminatory manner to any content provider. This ground-breaking ruling immediately triggered a series of trials conducted by different phone companies, most of them using fiber optics for the transmission of their video signals. Bell Atlantic, which is currently the only company in North America to use the same technology as Stentor, is now running a trial project among 300 of its employees in Northern Virginia. The trial has been running since May of 1993 and the video service is scheduled to be offered commercially in late 1994 or early 1995. Meanwhile, the Canadian Radio-Television and Telecommunications Commission (CRTC) startedapublicconsultationprocess on the matter in March 1994 (Public notice 1994-33) and a final decision is still pending.

It was this more relaxed regulatory context which allowed Stentor to position VOD trial as falling under the telecommunications jurisdiction. From January to April 1994, Stentor held phase 1 of the trial at two Ottawa-area universities: Carleton University and the University of Ottawa. The study described in this paper was conducted during the trial at the University of Ottawa. The system Stentor was testing was based on a technology jointly developed by two research and development companies, Bell-Northern Research in Montreal and Bellcore in the United States. The access technology called Asymmetric Digital Subscriber Loop (ADSL), uses state of the art modulation technologies to provide a bandwidth of 1.5 million bits per second, i.e. approximately 95 pages of text can be transmitted every second, over standard telephone wires. Coupled digital video compression standard called MPEG-1 (Motion Pictures Expert Group), ADSL allows the transmission of a VCR-quality signal (as opposed to broadcast-quality) over traditional twisted-pair copper wires. The difference in quality between the VOD signals and cable television signals lies mainly in the resolution of the picture. For phase 1 of the trial, Bell-Northern Research devised a computerized interface which allows users to select the movie they want by choosing from a list presented on the computer screen. The VOD
software features VCR-like controls (fast-forward, rewind, pause, play, and stop) which considerably increase the flexibility of the viewing session. Since the navigation program was conceived in the Microsoft Windows environment, students involved in the project could use other concurrent software, namely word processing, to take notes on the movie while they were watching.

## VIDEO-ON-DEMAND AS A PEDAGOGICAL ASSISTANT

There are two primary ways in which VOD could be used in a classroom as a learning aid: either as a teaching resource to present videotaped material or as an independent learning system, either showing linear videos or interactive video segments. While numerous experiments have been conducted with the use of video for instructional purposes, be it with the use of television, VCR or the computer, the unique contribution of the trial held at the two Ottawa universities is the fact that for the first time, users had complete and instantaneous control not only over the content they wanted to view but also over the pace of their learning session.

In a study conducted at Ball State University, Fissel (1993) reported that the use of video material to teach first year university classes had contributed positively to students' acquisition of some of the course's materials. The system used at Ball State University provided networked distribution of video signals in a number of rooms on campus. Teachers videotaped computer graphics and audiovisual material and then showed them in class. Students said the use of computer graphics helped them synthesize what was important and allowed them to do more efficient note-taking. The use of video material seemed to make the textbook more understandable and gave visual cues that students were able to remember more easily both during study and exams.

Aside from its use as a teaching resource, VOD can also be used as a tool for interactive video learning. Through the years, many educators have used video material in class to facilitate various aspects of learning, in particular in the general field of media education (Buckingham, 1990). As a forerunner to VOD, the development of videodisc technology has opened up novel learning applications in which students get complete control over the playback of interactive video modules. Although with the latter technology students are limited by the range of materials encoded on a single videosdisc, its educational potentials and similarities with VOD are nonetheless worth noting.

Studies show that the use of video material in which the learner controls the pace of the viewing session can be a great learning aid. Switzer and Switzer (1993) present the results of a study using courseware designed for university-level biology teaching, stating that "students who used the interactive video system scored significantly higher on a reliable post-test than students who received the instruction from classroom lecture and textbook.
The average total study time required by the videodisc group was 37 percent less than the time required by the classroom group" (p. 314). Another study sponsored by IBM Corporation compared student retention in a traditional lecture-based classroom to retention in an interactive video environment. " Students using interactive video demonstrated a 50 percent greater retention of the material. The percentage of students reaching mastery (80% on the post-test) increased over 300 percent with the use of interactive video" (Switzer & Switzer, p. 315).

Switzer and Switzer (1993) further list the benefits of interactive video as: "participation increases learning, feedback from the computer aids learning, repetition increases learning, information taken in through more than one sense is more readily learned" (p. 314). As mentioned before, many of these characteristics were present in the VOD experiment conducted at University of Ottawa, including the number of times a segment was shown.

In agreement with the definition provided by Sweeters (1994), the VOD trial described in this article can be categorized as a tutorial learning tool, in the sense that it "can be used to replace a lecture, for self-learning, or for remediation" (p.48). The tutorial model presented by Sweeters in Figure 1 relies on Robert Gagnes Events of Instruction (Gagne, Briggs, & Wager, 1992). Gagne is a scholar generally credited with having spelled out the necessary functions of a learning system. The standard tutorial model developed by Sweeters is instructive for our discussion of VOD as a learning tool in that it helps visualize precisely where VOD differs from other traditional models.

<figure><img src="image_16.png" /><figcaption>**Figure 1.**
VOD's Tutorial Model</figcaption></figure>

Because of the specific documentary nature of the contents that were used in the University of Ottawa's experiment, some components of Sweeters' model do not apply to VOD. Such is the case with the introduction, summary and test units of the model, that are better suited for structured, didactic material and which were generally covered by the professor responsible for the VOD trial. As we see in the figure above, what VOD adds to a tutorial model is a feedback loop that allows the learner to review any given segment at will. This feature of VOD that favors a flexible, accurate and user-friendly to-and-fro movement between the different sections of a document makes it a learning system where user control is at the forefront.

However, as the "Overall Assessment" section onp. 72 will indicate later, the biggest challenge in developing learning tutorials of this type consists in
matching the teaching objectives with the specific learning requirements of a
variety of users. Our results confirm Sweeters' (1994) observation that " when
used extensively, tutorials sometimes exhibit standard patterns of design which
students find repetitious or boring" (p. 48). Whether the contents are offered on
demand or not, the fact remains that redundancy in the presentational aspects
can have detrimental effects on both the evaluation of the system and on the
propensity to use it.

A word of caution is in order at this point with regards to positioning VOD,
and similar recent technological developments, as being a truly interactive
system. As far as the system's hardware is concerned, Heeter (1989) proposes
6 criteria upon which we can establish the relative degree of interactivity of
a given system. These criteria are:

¹ the complexity of choices presented to the user;

· the amount of effort needed to access information;

· the degree to which a media responds effectively to the user;

¹ the monitoring capacity of the system;

· the possibility for the user to add information to the system;

· the capacity for the system to facilitate communication between its
users.

Analyzed under this specific set of criteria, we can see that the VOD system
tested at the University of Ottawa cannot be considered purely interactive, as the
last two items in the above list have not been implemented in this system. It must
be noted that interactivity lies not only in the technology, but also in the user's
behavioural patterns. Interactive television systems were studied by Jun (1986)
who came up with two fundamental aspects of interactivity: a technical aspect
which corresponds to the specific media, and a motivational factor linked to the
willingness of the user to get involved in the system's bidirectional communica-
tion paths. Chen (1984) offers similar observations: " we begin to see that
passivity and interactivity are qualities of individuals making use of media, not
the media themselves " (p. 284). It is thus fundamental to evaluate whether the
system addresses a specific need for which the user would be ready to become
active in the communication process.

If we use the word "interactive" with a more restrictive notion of interac-
tion, then there are many cases in which the use of interactive video seems the
perfect alternative to traditional teaching. For instance, when the subject is of
an abstract nature, video documents may help by providing visual cues to the
learner. Interactive video is also very useful in cases where hands-on experience
with the subject taught would be costly or dangerous. Students can become more
responsible for their learning if they can work when they want, thus adding
a flexible dimension to their acquisition of knowledge in the process.

This form of self-controlled learning has been primarily restricted to
videodisc systems which allow students to interact on an individual basis with
the content of the video material. Lately, some software programs, such as
QuickTime, have allowed the development of multimedia products running on CD-Roms. VOD holds promise as a replacement or as an extension of the videodisc. The VOD system tested in the present study allows users to select the document which is viewed and to control the pace of the viewing session. The use of an interactive video learning system such as VOD allows for simultaneous access to dozens of video service providers, whereas videodiscs must be purchased in advance and must physically reside in every location where they are used. VOD has another major advantage over videodiscs. Different users of VOD can access the same document at once at their individual workstations and independently screen the film as they please, whereas there must be one videodisc for every concurrent user. In addition, content providers can instantaneously upgrade their material on a VOD system, a flexibility that is not possible with videodiscs, since new videodiscs must be produced when the content is revised. Since VOD systems are still in the early phases of their development, it is still premature to comment with any reasonable certainty about their level of reliability and possible uses. There is no question, however, that they are unlikely to replace videodisc systems, at least in the immediate future.

## CONTEXT OF EXPERIMENTATION

Carleton University's participation in the experiment was conducted through the Department of Philosophy. Over the semester, 36 modules of 45 minutes, each dealing with symbolic logic, were videotaped and made available to students through four computers. Three hundred and forty students (340) were registered in the course, and about 90 of them (26.5%) actually used the VOD system at least once over the duration of the course. The users had total control over when they chose to attend their classes and the duration of their viewing.

At the University of Ottawa, the trial was held under the auspices of the Department of Communication. For the Department, VOD represented an opportunity to integrate the trial in a course dealing with the social impacts of new communication, information and entertainment technologies. After negotiating an agreement with the National Film Board of Canada (NFB), the Department secured permission to use, free of copyright fees, some 40 films from the National Film Board catalogue. From the NFB standpoint, this type of partnership was conceived as an occasion to explore new ways to promote its catalogue of documentaries and fiction movies. In addition, the joint venture with Stentor and the University of Ottawa was considered by NFB authorities as a high profile experiment that held encouraging potential for the advancement of audiovisual distributionsystems. Because of storage and cost constraints imposed by the trial's video server, the computer which stores the digitally encoded video material, the University of Ottawa was allocated approximately 48% of the server's total capacity (32 gygabytes). This capacity enabled up to 32 hours of NFB video content to be stored on the server for the
duration of the trial. The selected material for the trial at the University of
Ottawa addressed a variety of topics, such as society, media, women in media,
new technologies and work, children and the media, and journalism. Most of the
titles chosen dealt, to some degree, with the impacts of new technologies on
different aspects of contemporary life. The videos were accessed in the computer
program through a tree-like structure splitting the titles first into French and
Englishh entries, and subsequently into various categories.

Seventeen students (17) were enrolled in the course CMN 4515: New
Communication, Information and Entertainment Technologies and Private
Life. From a methodological standpoint, our inquiry would be best described as
quasi-experimental. In the official course outline distributed to students at the
outset of the semester, they were informed that the course would contain an
experiment to test the pedagogical potency of VOD. As part of their assignments,
students were required to watch one movie per week and submit a one-page
critical essay on the subject matter dealt with in the movie. This weekly
assignment was designed to ensure a regular use of the VOD technology so that
students could make an informed assessment of it. Students also were required
to make two class presentations using excerpts from the movies available on the
system. This protocol allowed them to experiment with the VOD system as an
independent learning vehicle as well as a teaching resource that would serve to
illustrate certain key points in the students' exposes.

Another Mass Media communication class of thirty students also had
sporadic access to the system. They used VOD primarily to experiment with
the technology within their course curriculum. These students were asked to
consult two documents made available through the system. We collected data
on their experience for comparative purposes.

METHODOLOGY

The purpose of this study was to assess the educational potential a new
communication technology like VOD might have in the marketplace. More
specifically, we wanted to see how users, in this case students, would react to
the introduction of this device in the classroom. Although that specific
perspective had very little, if any, bearing on the way the actual experiment
was conducted, one must nevertheless acknowledge the system provider's
point of view in this trial. Stentor was, naturally, very concerned with how the
system behaved and responded to the frequent modifications that an experiment
inevitably entails. As we mention in the next section, this state of affairs cannot
be discounted as it might have affected the way some students perceived the
system.

A series of three questionnaires was administered to students enrolled in
both the New Technologies and the Mass Media courses at the University of
Ottawa. A first questionnaire was given to students before they had any contact
with Stentor's VOD system. This questionnaire was designed to provide a profile
of student characteristics including age, sex, former experience with technologies, current use of communication technologies and their preconceptions about technology in general. The second questionnaire was distributed one month after the introduction of the system. It asked students about their overall experience using the system: user-friendliness of the software, relative satisfaction and thoughts about how the system could be used and improved. At the end of the trial, in April 1994, the final questionnaire was distributed. This questionnaire was similar to the second one, except that it included a series of questions on the user's vision of the future for the VOD technology.

This series of three questionnaires was complemented by two focus group sessions which were held in the classroom. In these sessions, students were asked about their feelings regarding the VOD technology. They were encouraged to voice their comments for and against the system, and were asked to describe the system's main problems and how they thought these problems could be solved. Finally, an on-screen questionnaire was used. Throughout the three months of the trial, every third student using VOD would get to answer a brief on-screen questionnaire asking about their appreciation of the system. By presenting the questionnaire to every third student, a random sample was obtained.

## DESCRIPTION OF THE EXPERIMENTAL GROUP

The results presented here are from two different pools of respondents. Unless otherwise stated, all of the information discussed in this section originates from the class where VOD was tested. In some cases, data obtained from another communication class will be brought up to accentuate certain tendencies or conversely, to signal dissonant results. To avoid any confusion between the two groups, we will refer to this latter class as the "comparison group". It is worth repeating that in both these groups, VOD was construed as an emerging technology whose pedagogical efficacy was the prime element of investigation.

It must be emphasized that the trial involved a limited number of students in a particular setting. The trial group included 17 students, most of them in their third or fourth year of a bachelor's degree in Communication Studies. The New Technologies course was set up much like a seminar. Students were asked to discuss their weekly readings on new communication technologies and how they impacted on the user's everyday life in particular and on society in general. The course's goal was to foster the development of a critical approach toward new technologies. Therefore, our respondents came to adopt a predominantly skeptical and probing stance vis-a-vis the various technological developments that were examined. Another determining factor in the student's assessment of VOD was the perception they had of the trial environment. Because it was a Canadian first for the testing of a fully digital VOD system, the trial received substantial media coverage and many journalists came during or
around class time to obtain students' comments. A Stentor representative was often on the site during lectures to check progress and to ensure that the system was working adequately. This situation clearly gave students a double message: on the one hand, they felt somehow flattered to be receiving so much media attention as they were part of a select group of students fast becoming experts on VOD. On the other hand however, some expressed a certain degree of annoyance at having frequent visits from external parties for what was perceived to be a "check call" to verify the performance of a major financial and public relations investment.

With this experimental context in mind and the limited number of users (n=17), this study purported to evaluate the nature of the interactions that users established with the first-generation of the VOD system and to assess the educational potential of using this technology in a classroom setting. The scope of the results presented here is possibly limited by a host of factors: the relatively small size of the sample; the limited choice of video documents users could access; and the fact that respondents were fully cognizant of being part of an experiment on a technology that was being fine-tuned before going to market. Hence, the numbers presented below are to be construed as indications of the dominant tendencies expressed by the first educational users of VOD and must be limited to the group under study.

Our prime² sample group (n=17) was mostly composed of women (59%), and 76% of these respondents were under 25 years of age. The two important parameters that define this sample group are the interest students show toward new communication technologies and how they perceive themselves vis-a-vis the technologies. On both questions, 95% of the students said they had either some interest (60%) or a lot of interest (35%) towards new technologies (Figure 2), while the same percentage said they were either a moderate technophile (80%) or an inveterate technophile (15%) (Figure 3).

<figure><img src="image_17.png" /><figcaption>Figure 2. Interest Toward New Technology</figcaption></figure>
Our comparison group out shares basic demographic descriptors with the experimental group. The main area on which the two groups differ is the relative level of interest shown towards technology, where students from the comparison group expressed having either some interest (63%) or a lot of interest (11%) (see figure 2), whereas our experimental group's combined total in these two categories was 95%. Whenasked to describe themselves withregard to technology, only 77% of the students in the comparison group consider themselves either moderate or inveterate technophiles, while it was 95% for the experimental group (Figure 3). Another distinction between the experimental group and the comparison group was that the course the latter group was taking is a compulsory course, whereas students in the experimental group had freely chosen to enroll in that course. For students who elected to take the New Technologies course, it is safe to assume that they did so primarily because they had a keen interest in the subject matter, and were possibly more committed toward owning, using and reflecting upon new technologies.

<figure><img src="image_18.png" /><figcaption>**Figure 3.**
*Propensity Toward Technology*</figcaption></figure>

We wanted to assess to what degree our respondents were familiar with various communication and information technologies, thus giving us some background information on their predispositions toward yet another communication innovation. Most of the respondents were rather familiar with technology, with 88% of them subscribing to cable, owning a VCR and a computer³. Granted that owning a technology does not equate to using it, the fact that we are dealing here with university-level communication students more or less warrants a functional knowledge of both VCRs and computers as part of the general training they obtain during the course of their education.

Since the generation of VOD system that was tested at University of Ottawa is based on a computer, it is interesting to observe that 80% of the students that own a computer use it at least 4 hours per week, with 27% of them using it for
more than 13 hours per week. This propensity toward interacting with comput-
ers should contribute to facilitate the navigation procedures within the VOD
architecture. Among those who had a computer (n=15), 40% had a modem and
47% had a fax. In addition, 82% of the students owned a compact disc player, 94%
had an answering machine and 47% subscribed to one of Bell Canada's
specialized telephone services, with call waiting being by far the most common
service subscribed to within that group (87.5%). When asked about their
entertainment habits, a little over a third of the students (40%) said they watch
a movie in a theater twice a month or more. A large majority of them (88%) spend
less than $20 a month in expenses related to video entertainment, and 65% rent
a movie twice or more per month.

From a transmission standpoint, since VOD has the technical potential to
compete with what is currently offered by cable companies, respondents were
asked what they thought of what is typically shown on television. A little over
half of the sample (60%) thinkthat the choice ofprogramspresentedon television
is adequate, while 94% of respondents find that the quality of the programs
shown on television is good.

OVERALLASSESSMENT

We also wanted to assess whether VOD had had an influence on the user's
learning over the course of the semester. When first asked about it after one
month of usage, 77% of the students said that it had a positive influence on their
learning. At the end of the semester, however, the percentage went down some
36 points to 41% (Figure 4). This seems to indicate that the initial enthusiasm
that students experienced towards the use of a new technology faded over the
three months of use, and that a more critical position was taken at the conclusion
of the experiment, a situation fairly common in research known as the
Hawthorne effect. In all fairness to the technology though, one cannot discount
the "fatigue factor" which certainly affected negatively the evaluation of VOD.
For the majority of the users, the novelty of going to a laboratory to view an NFB
document on a 14 inch computer screen rapidly wore off. In retrospect, based on
the comments expressed by the participants, had larger monitors been used or
if students could have accessed the system from their home, this fatigue factor
could have been lessened. Similarly, had VOD had a wider variety of documents
from which students could choose, their evaluation of the system might have
been different. On the other hand, it must be noted that students in the
comparison group gave somewhat more mitigated answers, with 63% indicating
that VOD had no effect on their learning after one month of use, and only 19%
claiming that it had a positive influence. This can most likely be explained by
virtue of the fact that students in the comparison group had had a rather sporadic
contact with VOD and hence had not been given adequate opportunities to assess
thoroughly the relative merits of the system compared to other more traditional
means of learning
<figure><img src="image_19.png" /><figcaption>**Figure 4.**
Effect of Video on Demand on Learning</figcaption></figure>

Another factor that affected the students' assessment of the technology is the physical setup in which the trial took place. The four computers made available to students were grouped in two rather small rooms. The chairs were normal classroom chairs and the confined space forced the students to sit not only very close to the screen but also to each other. At the end of the trial, one out of three (33%) students answered negatively when asked if the physical environment was adequate for viewing and/or learning. The discussion during the focus groups often centered on the physical environment, as many students expressed dissatisfaction regarding the chairs, proximity of the screen from the viewer relative smallness of the room and discomfort from the headset. "I can't listen to a movie on a chair with headphones not loud enough. I don't find it comfortable, I don't find it pleasing." is a typical example of the comments heard. Ostensibly, some of the ergonomic elements of the system seem to have contributed negatively to the overall assessment of VOD as well as detracted from a more frequent and beneficial use. With regard to the ideal location where they would prefer to access the system, students listed their home (100%), in a library (50%), and in other places (83%) -including at the office (Figure 5).

The overall tendency towards the manifestation of the Hawthorne Effect and the attenuation of the level of enthusiasm via-a-vis VOD that surrounds it, is confirmed by many variables. In all questions that pertain to the user's appreciation of the system, we see a generalized shift towards a more neutral position. In a question where students are asked about their overall satisfaction with the system so far, the answers went from 88% saying they were generally satisfied (82%) or very satisfied (6%) after one month of use to 66% at the end of the semester (Figure 6). Those results must not be interpreted as a disenchantment with VOD as much as a somewhat natural tendency toward a more balanced appraisal over a three-month long exposure to it. Although it is
impossible to identify the individual contribution of the contents in the overall assessment of VOD, one must nevertheless recognize that the limits imposed by the sole selection of NFB entries could have contaminated somehow the overall evaluation of this VOD trial by creating in the minds of users a strong bond between the technology and the paucity of film offerings at that particular stage.

<figure><img src="image_20.png" /><figcaption>**Figure 6.**
Overall Satisfaction Toward the System</figcaption></figure>

<figure><img src="image_21.png" /></figure>
USERINTERFACE

The way the user interface was designed caused many problems and may account for some of the lower scores on the satisfaction scale on the final questionnaire. The users felt the interface was cluttered and that too many windows were "floating around", making it difficult to give commands to the program and organize the windows on the screen. During focus groups, many students expressed their dissatisfaction with the intuitiveness of the interface: "I think they should improve the interface a bit. There are too many little screens popping up. They should make it smaller and more intuitive". We thus see the need for engineers to take time to reassess the user interface and the potential user's reception of it at the early stages of the development of a technology. Users wished to have every command integrated in one window, but the program was designed in such a way that the separate modules each have their own window. The user interface should not be compromised to compensate for delays in the implementation of a trial program since the technology will inevitably be assessed upon its interface, which is the part the user is in contact with the most and the only part the user really is knowledgeable about.

It is relevant to mention at this point that, subsequent to the end of the trial, Stentor used the feedback provided by students at both universities tore-design the interface. The changes resulted in minimizing the number of windows present at any one point on the screen as well as integrating all commands, such as VCR controls, audiovisual controls, access to word-processing, etc., in a single integrated menu toolbar at the top of the screen.

With regard to their competence at handling the system, users seemed to gain a better familiarity with the controls throughout the trial period. On a question dealing with the entry/exit procedures of the system where users were asked to quantify the intuitiveness of those procedures on a scale going from 1 (very hard) to 5 (very easy), 64% of the respondents answered either 4 or 5 after one month of use. Furthermore, that number climbed to 84% at the end of the trial (Figure 7). Although users became more acquainted with the commands and controls, half of them (50%) said, after three months of use, that the controls in the program were less easy to use than those on their VCR. We must note however that despite a 15 point-drop from the results obtained on that same question in January, this remains a relatively high percentage of dissatisfaction. This fact can be explained by the problems that were experienced with the use of the rewind, pause and fast-forward functions, which sometimes showed erratic behaviour when invoked. Here again, Stentor has since addressed this problem.
<figure><img src="image_22.png" /><figcaption>**Figure 7.**
Intuitiveness of Entry/Exit Procedures</figcaption></figure>

When studying users' reactions to the various control functions, it seems that a majority of users (65% at the beginning of the semester, 100% at the end of the semester) find that the “jump to” command is very useful. This command allows the user to select any point in the movie from which to start the viewing session. At any time during the session, the user can also modify the current location and jump to another part of the movie. There again, the higher score obtained at the conclusion of the trial period can be attributed to users becoming more comfortable with the system's components. These results show one of the main advantages VOD has over traditional VCRs: complete control over the viewing sequence and in addition, a much faster and more precise access to any given segment of the document. While traditional videocassettes are generally viewed in a linear sequence and are not designed to easily locate any given sequence on a tape, VOD video segments can be viewed in any order, instantly jumping from one segment to another. This feature makes VOD a potentially powerful learning tool that can be used to present video sequences in an order chosen by the learner and/or instructor. When reflecting upon the way VOD was used during the trial period, they did not really perceive how it differed from what they could already accomplish with their VCR. Students acknowledged the potential of the technology but thought, at this present stage, that they were underutilized.

One exception is worth mentioning. Notwithstanding the nature of the consensus that users arrived at when comparing the potentials of VOD and a VCR, one of the most technologically inclined students in the group managed to exploit the referential potential of VOD in an original manner. When submitting his weekly critical essays on the NFB title that he had seen that week, the student used the “time bar” in much the same way quotations are traditionally used in a paper. The student would indicate that a given passage in his essay related to the segment shown” at the 12 minutes 26 seconds mark “in the video.
This indexing capability of VOD can indeed have a high degree of pertinence for instructional purposes.

Concerning future developments of the technology, almost 6 out of 10 students (58%) thought that the system would benefit from more interaction between the user and the video. The VCR-like controls were thought to be an important feature for 69% of the users. As for content, all considered multimedia documents (documents that combine text, sound, film, graphics and pictures much like CD-ROMs), as being very important to be provided by VOD (100%) whereas two thirds considered learning material (67%), or informative documents (64%) should be treated as priorities for such a system. We can see that the students in our group are rather technologically inclined as they are familiar with the concept of multimedia and would like to benefit from the possibilities the use of the computer as an interface brings forth.

At the end of the trial, users seemed to appreciate the potential VOD has as a learning tool. On the first questionnaire, only 12% of the students said they would prefer accessing VOD servers through a computer screen versus through a television set. On the second questionnaire, one third (33%) of the students said a computer screen would be their preferred mode. It appears users began to realize the potential of a well-designed interface that would allow bridging the power of the computer with the power of a video distribution system like VOD. This openness toward what the system can potentially achieve is visible in the fact that, at the conclusion of the trial period, close to six respondents out of ten (58%) stated they would be willing to spend between 3 and 5 hours a week using the VOD system versus only 12% at the beginning of the trial (Figure 8). This more favorable perception of VOD is, however, toned down by some remarks that were voiced during the focus group sessions pertaining to the power of the computer as well as the quality of the interface. Indeed, the lack of pixel resolution on a computer screen seems to hinder the time one would like to watch a video document on a screen to a maximum of a couple of hours at a time. This factor is however directly dependent on the type of computer and monitor used to both run and display the video material.
<figure><img src="image_23.png" /><figcaption>Figure 8.
Maxirnum Number of Hours Spent Watching Video on Demand During One Week</figcaption></figure>

STUDENTS' PERCEPTIONS OF NEW TECHNOLOGIES
AND VIDEO-ON-DEMAND

The questionnaire that students filled out at the end of the semester included a series of statements about new technologies, the information highway, and VOD. The 17 students that took part in the trial were asked if they agreed or disagreed with each statement. The statements that received the highest percentage of agreement (i.e. combining the scores of "somehow" and "totally in agreement") are illustrated in the following table 1: "I like to be able to select what I want to watch when I want to watch it." with which 100% of the users agreed, "VOD is a technology in search of an application" (92%), "I cannot imagine viewing a film for more than an hour on a PC screen (84%)," "VOD will give people a better control on their viewing habits" (83%), and "The CRTC will do everything in its power to facilitate the diffusion of this technology" (64%).
<caption>TABLE 1</caption>

Statements which Received the Highest Percentage of Agreement (n=17)

<table><thead><tr><th></th><th>Partly<br>in agreement</th><th>Totally<br>in agreement</th></tr></thead><tbody><tr><td>I like to be able to select what I want to watch when I want to watch it</td><td>75</td><td>25</td></tr><tr><td>Video-On-Demand is a technology in search of an application</td><td>67</td><td>25</td></tr><tr><td>I cannot imagine viewing a film for more than an hour on a PC screen</td><td>42</td><td>42</td></tr><tr><td>Video-On-Demand will give people a better control over their viewing habits</td><td>75</td><td>8</td></tr><tr><td>The CRTC will do everything in its power to facilitate the diffusion of this technology</td><td>46</td><td>18</td></tr></tbody></table>

The statements which received the highest percentage of disagreement (i.e. combining the scores of “partly” and “totally disagree”) are listed in table 2: “The cable companies must have a monopoly over video broadcasting” (100%), “I would like to attend classes through VOD” (100%), “With the advent of VOD, I would cancel my cable subscription” (91%) and “VOD will help me save money on my entertainment expenses” (73%).

The results presented in table 2 indicate a persistent degree of attachment toward traditional means of entertainment and education. The participants in this trial clearly do not conceive of VOD, at least, *at this current stage of its development*, as a replacement technology that could eventually supplant what they are used to get via their cable service provider. Nor do they think that because of what VOD can offer, they would have less of a need to either go out to see a film, play or concert, or rent a video and in the process, end up spending less for their entertainment activities. As for the educational prospects of VOD, significant improvements will need to be implemented on both the ergonomic and content aspects of the system before students begin to show any firm inclination toward using VOD as an alternative to physically attending a lecture.
<table><caption>**TABLE 2**
*Statements Which Received the Most Disagreement (n=17)*</caption><thead><tr><th></th><th>Partly disagree</th><th>Totally disagree</th></tr></thead><tbody><tr><td>The cable companies must have a monopoly over video broadcasting</td><td>58</td><td>42</td></tr><tr><td>I would like to attend classes through video-On-Demand</td><td>33</td><td>67</td></tr><tr><td>With the advent of Video-On-Demand, I would cancel my cable subscription</td><td>45</td><td>46</td></tr><tr><td>Video-On-Demand will help me save money on my entertainment expenses</td><td>46</td><td>27</td></tr></tbody></table>

CONCLUSION

From the results collected for the present study, we can project a series of observations from the VOD system to a number of other recent technologies. The main conclusion to be drawn is the risk involved in evaluating a technology in the preliminary stages of its commercial development. The evaluation process addresses not only the technology in itself but also a collection of external variables, often out of the direct control of the developer and/or promoter, which affect the user's perception. In the specific case of the VOD trial, those factors include the physical environment in which the trial tookplace, the content made available through the system, the relative technical unreliability of a system still in the early stages of its development, and the fact that the user interface was not corrected over the three-month period according to the user's "wish list".

There is a tendency that follows the introduction of new technologies for the public to see the new tool as a new " toy ", a glimpse of the future that they would really like to experiment with. The initial amazement gradually wears off when the innovation becomes part of the public's everyday life and it becomes what it was designed to be: yet another technology. In the Stentor VOD case, this phenomenon is confirmed by the results obtained. The novelty effect wore off when students came to see the VOD system as just another study tool rather than as an innovation that could significantly affect the way they go about collecting information and knowledge. The external factors cited previously contributed to the dissipation of the novelty effect, particularly the fact that users did not see any significant changes in the technology over the semester. The content stayed the same and limited the possibilities for the users to experiment with the