CANADIAN JOURNAL OF EDUCATIONAL COMMUNICATION

Volume 19, Number 1, Winter 1990
ISSN 0710-4340

Conception d'un système conseiller intelligent dédié à la
modélisation conceptuelle des données

Marie-Michèle Boulet

Examination of Cognitive Style FD/FI as a Learner Selection
Criterion in Formative Evaluation

Chris A. Chinlen

Protecting Expression in Teleconferencing: Pseudonym-Based
Peer Review Journals

David S. Stodolsky

Computer-Mediated Communication and School Administrators:
A Case Study of a University In-Service Course

Janice Ahola-Sidaway

Margaret MacLean

Jack Treuhaft

An AMTEC Publication

<figure><img src="image_1.png" /></figure>
## EDITORIAL BOARD

Cheryl Amundsen
McGill University

Gary J. Anglin
University of Kentucky

Jon Baggaley
Concordia University

Robert M. Bernard
Concordia University

F. Barry Brown
University of Saskatchewan

Bruce Clark
University of Calgary

Dan Coldeway
Athabasca University

Bernard Dubreuil
Concordia University

D. J. Engel
University of Alberta

Garfield Fizzard
Memorial University of Newfoundland

George L. Geis
OISE

Michael Hannafin
Florida State University

William Hanson
Calgary Board of Education

Denis Hlynka
University of Manitoba

W. J. Hunter
University of Calgary

Robert Jones
Sheridan College of Applied
Arts and Technology

Lorne Koroluk
University of British Columbia

James J. LaFollette
University of Alberta

Richard F. Lewis
University of Windsor

David A. Mappin
University of Alberta

Earl R. Misanchuk
University of Saskatchewan

Ronald Owston
York University

Lauran Sandals
University of Calgary

Louise Sauve
Tele-Universite

Richard F. Schmold
Concordia University

R. J. Schmidt
Strathcona County Board of
Education

Marlela Tovar
Concordia University

Ronald Viaw
University de Sherbrooke

Laura Wlner
APO Quebec

Clayton R. Wright
Grant MacEwan Community College

## AMTEC BOARD OF DIRECTORS

**President**
Mary Kennedy
Memorial University

**Past President**
Wayne Blair
Alberta Department of Education

**President Elect**
Bruce McLean
Vancouver Community College

**Secretary / Treasurer**
Al LeBlanc
Separate School Board Sault Ste. Mar/e

**Director**
Margaret Sadler
University of Alberta

**Director**
John Godfreyson
School District #68 B.C.

**Director**
Dave Bieman
Huron County Board of Education
Canadian Journal
of Educational
Communication

Volume 19, Number 1
Winter 1990

Editor

Richard A. Schwier

Editorial Assistant

Patricia Nickel-St. Onge

Production Manager

Mary Genova, WRITEWORKS

ISSN 0710 - 4340

The Canadian Journal of Educational Com-
munication is published by the Association
for Media and Technology in Education in
Canada; Separate School Board, 395 Korah
Road, Sault Ste. Marie, Ontario P6B 1X5; At-
tention: Mr. Al LeBlanc. Secretary/Treasurer.
Notification of address change should be
sent to the above. All articles are copyright
by AMTEC and may be reproduced for non-
profit use without permission provided credit
is given to CJEC. Back issues of CJEC are $10
Canadian and may be obtained by con-
tacting the Editor. CJEC is indexed in the
Canadian Education Index and ERIC.

Second Class Mail Registration No. 6956

ARTICLES

Conception d'un systeme conseiller 3
intelligent dedie a la modelisation
conceptuelle des donnees

Marie-Michele Boulet

Examination of Cognitive Style
FD/FI as a Learner Selection
Criterion in Formative Evaluation

Chris A, Chinien

19

Protecting Expression in
Teleconferencing: Pseudonym-
Based Peer Review Journals

David S. Stodolsky

41

Computer-Mediated
Communication and School
Administrators: A Case Study
of a University In-Service Course

Janice Ahola-Sidaway

Margaret MacLean

Jack Treuhafi

53

MICROWARE REVIEW

BOOK REVIEWS

All correspondence should be
addressed to:

Dr. Richard A. Schwier, Editor
Communications/Continuing
& Vocational Education
College of Education
University of Saskatchewan
Saskatoon, Saskatchewan
Membership Form

AMTEC

Media ResourceCentre
67 McMeeken Street
Saulte Ste. Marie, ON
P6A 3L6

name _______________ title _______________

address _______________

city _______________ prov/postal code _______________

phone (w) _______________ phone (h) _______________

Fax _______________ Netmail _______________

I am willing to have pertinent information from the above included in the next Members Handbook

ALL RENEWALS COME DUE ON EITHER JANUARY 1ST OR JUNE 1ST

If this is a renewal then please indicate your Membership Number _______________

Student of Educational Technology $20.00*

Individual $50.00

Institutional $75.00

Commercial $125.00

*Students must have this form signed by your Department Head: _______________

Three issues $40.00

Special Interest Areas:

Media Production

Media Management

Telecommunications

School Libraries

Distance Education

Media Literacy

Other _______________

Micromputer Applications

Instructional Development

Mass Communications

Educ. Tech. Research

Curriculum Implementation

Interactive Video

Work Environment

Elementary

Secondary

College

University

Other _______________

Business

Government

Health Care

Please duplicate this form.

CJEC is typeset on an Apple Macintosh Plus™ in PageMaker 3.0™. Galley proofs to authors and final camera-ready impressions are output on an Apple LaserWriter Plus™.

Concordia University Printing Services

Acknowledgement

The Canadian Journal of Educational Communication is supported by a grant from the Social Sciences and Humanities Research Council of Canada.
# Conception d'un système conseiller intelligent dédié à la modélisation conceptuelle des données

Marie-Michèle Boulet

Résumé: Dans cet article, nous traitons du développement d'un système conseiller intelligent dédié à la modélisation conceptuelle des données. Nous exposons en premier lieu les caractéristiques du domaine d'application soit la modélisation conceptuelle des données. Les principales composantes du système sont ensuite décrites soit la base de connaissances, le modèle de l'usager, le module d'acquisition des connaissances et l'interface avec le monde extérieur.

## INTRODUCTION

Pour être en mesure de conseiller une personne sur la démarche à suivre pour résoudre un problème dans un domaine spécifique, il faut non seulement connaître le domaine, mais aussi pouvoir déterminer où la personne se situe dans la hiérarchie de préalables associée ou, en d'autres termes, quel est le niveau de ses connaissances. En fait, le conseiller doit posséder des connaissances ayant trait au processus d'apprentissage et au domaine spécifique tel l'investissement immobilier, la composition musicale ou la modélisation conceptuelle des données (Carroll et McKendree, 1987). En conséquence, un conseiller intelligent se définit comme étant un système informatique, accessible à l'intérieur d'un logiciel d'application et servant à répondre aux questions de l'usager, à lui poser des questions ou à lui enseigner les concepts du domaine dans lequel il oeuvre et à l'intérieur duquel les connaissances sont représentées en termes de structures de préalables. Le conseiller que nous avons développé intervient dans le cheminement d'un usager effectuant une tâche complexe, celle de concevoir un modèle conceptuel de données suivant la méthode Merise à l'aide du logiciel d'application graphique Windows.

Dans un premier temps, nous présentons brièvement le domaine d'application du système conseiller décrit ici soit la modélisation conceptuelle des données. Nous exposons ensuite les éléments nécessaires au fonctionnement d'un système conseiller. Par la suite, nous décrivons comment nous avons
procédé pour tenir compte de ces éléments lors de la conception de la base de connaissances, du modèle de l'usager, du mécanisme d'inférence, du module d'acquisition des connaissances et de l'interface avec le monde extérieur.

La modélisation conceptuelle des données suivant la méthode Merise

Un cours portant sur la modélisation conceptuelle comporte une partie théorique au cours de laquelle les divers concepts et règles propres au domaine sont exposés et une partie pratique au cours de laquelle les étudiants doivent appliquer les concepts théoriques et les règles lors de la résolution de cas décrivant diverses organisations et leurs données. C'est pour aider les étudiants lors de la résolution de ces cas en laboratoire que le système conseiller décrit plus après a été conçu.

Un modèle conceptuel de données est en quelque sorte une photographie de la mémoire d'une organisation. C'est un plan de mémoire servant à représenter l'ensemble des informations nécessaires aux opérations de l'organisation objet d'étude ainsi que les liens entre ces informations. Un modèle conceptuel de données se conçoit à partir des éléments d'information véhiculés dans l'organisation et de sa mission. Dans un modèle conceptuel de données, on illustre à l'aide d'un diagramme des choses identifiables et des associations entre ces choses. Suivant la méthode Merise, ces choses identifiables sont des individus et les associations entre ces choses identifiables sont des relations. Ces individus et ces relations possèdent des propriétés. Les règles d'utilisation des relations sont indiquées par des cardinalités.

Bien que chaque méthode propose des règles de modélisation, la décision à savoir comment représenter l'actualité particulière d'une organisation est une question de perception. A titre d'exemple, prenons le concept de mariage. Il peut être représenté de plusieurs façons, le tout dépendant du besoin de l'organisation pour laquelle le modèle est conçu.

Le concept de mariage peut être représenté à l'aide d'un seul individu-type MARIAGE possédant comme liste de propriétés-types NO ACTE DE MARIAGE, DATE DU MARIAGE, NO. ASS. SOC. EPOUSE et NO. ASS. SOC.

(Figure 1). Si on le représente tel qu'illustré à la figure toute cérémonie de mariage est une nouvelle occurrence de l'individu-type MARIAGE. Il ne peut donc y avoir qu'une seule date de mariage. Celaindique que l'organisation

<table><caption>Figure 1.</caption><tr><td>MARIAGE</td></tr><tr><td>* No acte de mariage</td></tr><tr><td>Date du mariage</td></tr><tr><td>No. ass. soc. épouse</td></tr><tr><td>No. ass. soc. mari</td></tr></table>

Concept de mariage représenté suivant Merise par un seul individu.
ne veut et n'a besoin de mémoriser que les cérémonies et non pas avec qui une personne est mariée.

Cependant, si le besoin de l'organisation consiste à vouloir connaître avec qui une personne est mariée, la représentation du concept de mariage présentée à la figure 1 est inadéquate. On devra alors utiliser la représentation de la figure 2.

<figure><img src="image_2.png" /><figcaption>Figure 2.
Concept de mariage représenté par un individu et une relation.</figcaption></figure>

Comme on peut le constater, la décision de représenter la réalité de l'organisation par un individu-type ou une relation-type dépend des besoins des preneurs de décision, c'est-à-dire de quel accès aux données ils ont besoin dans la base de données. Cet exemple nous a permis d'illustrer les difficultés que pose l'enseignement de la modélisation conceptuelle des données.

## Délai de réponse d'un système conseiller

Un système conseiller doit être en mesure de réagir de façon prompte à toutes les fois que les interactions usager/logiciel d'application le requièrent. Pour ce faire, il doit continuellement être en mesure de recevoir des données ayant trait à ce que l'usager a fait, fait à un moment précis dans le temps et fera potentiellement dans le futur. Il doit analyser ces données afin de prendre la meilleure décision en regard de ces données et de donner le meilleur feed-back au moment où il est opportun de le faire (Genesereth, 1982; Hill et Miller, 1988; Kaemmerer et Allard, 1987). Parmi les moyens utilisables pour rendre le conseiller capable de fournir le feed-back approprié au bon moment, on trouve la separation des environnements de conception et d'exécution, l'utilisation de code compilé et de mécanismes de contraintes implantés dans le processus d'inférence permettant d'éviter une explosion combinatoire lors de la recherche. Un des moyens proposés dans la littérature est le découpage de la base de connaissances en parties fonctionnelles (McCalla et coll., 1986). Ce découpage étant fait, le processus d'inférence possède une façon de déterminer quelles parties sont pertinentes et lesquelles ne le sont pas. Cette façon de représenter les connaissances nécessite une connaissance préalable à l'utilisation du système ayant trait au quand et comment une portion de connaissance est utilisée. Ainsi, l'utilisation du savoir par le système est déterminée lors du développement et non pas lors des interactions usager/système-à-base-de-
connaissances (Rowan, 1987). C'est la voie que nous avons suivie et dont nous traiterons plus loin dans le présent article.

## Qualité des décisions prises

Un système conseiller étant un outil d'aide et de conseil pour tout usager au moment où il interagit avec un logiciel d'application, non seulement doit-il pouvoir agir dans un délai souvent très court mais aussi avec fiabilité. Pour ce faire, le système doit posséder, en plus d'un algorithme efficace d'inférence, une bonne connaissance du domaine pour lequel il conseille (Rodriguez et Rivera, 1986). Cela comprend les connaissances suivantes:

* des heuristiques utilisées par les experts du domaine;

* des informations sur la structure de préalables nécessaires à la réalisation de la tâche;

* des correctifs appropriés pour chaque faiblesse détectée chez un usager; et

* des comportements attendus de la part de l'usager.

Chacun de ces aspects a été considéré lors du développement du système conseiller dédié à la modélisation conceptuelle des données.

## Gestion des ressources disponibles

Le système doit être en mesure de gérer adéquatement ses ressources. En effet la vitesse de calcul et la taille de la mémoire d'un ordinateur sont des ressources finies. Le conseiller doit donc être en mesure de déterminer à tout moment ce qu'il est pertinent de conserver en mémoire, de calculer, de vérifier à un moment donné dans le temps la qualité de son diagnostic sans pour autant ignorer les tâches courantes réalisées par l'usager. Ainsi, les actions à prendre doivent être sélectionnées par le système en fonction de certains objectifs. Le mécanisme d'inférence à ce titre joue un rôle important. En ce qui a trait à la gestion de l'espace mémoire, il est essentiel de permettre au système d'éliminer et d'archiver toute donnée pertinente (Leitch, 1987).

## DONNÉES ET CONNAISSANCES NÉCESSAIRES AU SYSTÈME

Pour fonctionner adéquatement, le système conseiller doit, avant tout, posséder des connaissances. Le système doit en conséquence pouvoir gérer plusieurs bases de connaissances qui peuvent être facilement élaborées, élargies, modifiées et entretenues (Fisher et coll., 1985). Ainsi, il est fortement recommandé de disposer d'outils destinés à l'acquisition des connaissances visant à assister les experts lors la saisie des connaissances, à vérifier et valider ces connaissances nouvellement entrées et à les stocker à l'intérieur de la base de (Sauer et Walsh, 1983). Notre système conseiller dispose d'un
tel module. Le système conseiller doit également être en mesure d'effectuer
certaines prédicitions concernant le futur à partir des données passées et
présentes ayant trait à l'usager (Milne, 1987). Pour ce faire, il doit posséder
certaines connaissance de la dynamique du domaine en fonction de diverses
considérations (Khanna et Moore, 1986). Nous avons analysé le domaine de la
modélisation conceptuelle des données en termes de structures de préalables
afin de doter le système conseiller de ces possibilités.

Ainsi, un système conseiller doit posséder une base de connaissances, un
modèle de l'usager, un mécanisme d'inférence, un module d'acquisition des
connaissances et une interface avec le monde extérieur. Nous présentons
sommairement dans ce qui suit l'implantation de chacune de ces composantes.

Base de connaissances, modèle de l'usager et mécanisme d'inférence

La structure d'un domaine de connaissances est représentée par la descrip-
tion de différents objets, composantes, entités ou concepts d'intérêt pour le
système et les liens entre ceux-ci (Hayes-Roth et coll., 1983). Pour représenter
cette structure, nous avons organisé les connaissances en termes de plusieurs
structures arborescentes reliées les unes aux autres et représentant une
hiérarchie de préalables conformément à l'hypothèse de l'apprentissage
cumulatif. Nous acceptons donc que la condition la plus importante pour avoir
l'assurance qu'un apprentissage a été effectué est la maîtrise des capacités
préalables (Gagné, 1968; Gagné et coll., 1962). Cela veut dire, qu'en accord avec
ces auteurs, nous considérons que chaque personne approche une nouvelle
tâche avec une collection différente d'apprentissage d'habiletés préalables et
que pour être efficace, un programme d'apprentissage ou de conseil doit
prendre en compte ce que la personne sait faire et ce qu'elle ne sait pas faire.
Lorsque l'on enseigne, les capacités qu'une personne possède sont évaluées à
l'aide de tests diagnostiques (évaluationformative) qui permettent de produire
un profil de l'apprenant en fonction d'une structure de préalables. Dans un
contexte d'enseignement, le point le plus important est d'enseigner les préal-
ables à une habileté intellectuelle que la personne ne maîtrise pas et de lui
donner la possibilité de revoir ces préalables à n'importe quel moment. Dans
un contexte de conseil, que ce soit un professeur qui reçoit un apprenant pour
un dépannage ou un système conseiller intervenant au moment auquel un
usager éprouve certaines diffkultés avec un domaine donné, il faut que
l'intervenant humain ou informatisé puisse positionner une question posée
par ou à l'usager, ou une erreur de ce même usager, à l'intérieur d'une structure
de préalables. L'intervenant doit ensuite inférer quelles diffkultés pourront
potentiellement en découler, en vue d'éliminer de sa mémoire toute donnée non
pertinente. Pour implanter, doter le système conseiller de tels mécanismes,
nous nous sommes inspirés d'un des outils de l'évaluation formative, le schème
à facettes multiples (Scallon, 1988) pour construire en mémoire d'ordinateur
un portrait du profil de l'usager, soit le modèle de l'usager, en ce qui a trait à
ses difficultés et ce, afin d'être en mesure de lui prodiguer un conseil ou un
enseignement correctif. Comme le système peut anticiper les problèmes
potentiels de l'usager, les explications nécessaires sont chargées en mémoire, évitant ainsi un trop long délai de réponse et une attente trop longue pour l'usager.

Le schème à facettes multiples offre la possibilité d'évaluer de façon précise le degré d'atteinte d'un objectif d'apprentissage de type habileté intellectuelle. Il est conçu en vue de la décision à prendre soit de faire progresser un élève vers d'autres objectifs, soit de l'impliquer dans des activités correctives spécifiques aux difficultés identifiées. Ce modèle permet, à l'aide de facettes, de définir un ensemble d'items et d'en arriver à un tableau de spécifications à deux ou plusieurs dimensions. Par facettes, il faut entendre toute variation introduite au niveau des items servant à évaluer le rendement de l'élève. Au lieu d'utiliser le principe du schèmes à facettes multiples pour produire des questions pour un test, nous l'utilisons pour rattacher chacune des interventions du système conseiller, permettant ainsi de les situer par rapport à d'autres préalables. Lors de l'acquisition des connaissances pour notre conseiller, nous nous sommes inspirés de tels schèmes pour représenter les connaissances relatives à chaque hiérarchie d'apprentissage établie lors de l'analyse du domaine de la modélisation conceptuelle des données.

Lorsque l'usager élabore un modèle conceptuel de données à l'aide d'un logiciel d'application graphique le système conseiller recueille d'abord un ensemble de données relatives au modèle. Ces données sont emmagasinées suivant les principes de design pédagogique de Gagne (1977). Le but visé par le système conseiller est de permettre à toute personne utilisant un logiciel d'application graphique de recevoir de l'aide lors de la conception d'un modèle conceptuel des données. Ce but se découpe en quatre objectifs d'unité suite à l'analyse du domaine effectuée par Boulet (1988a, 1988b,1988c) et Boulet et Barbeau (1989). A titre d'exemple, le premier objectif d'unité (O.U.1) identifié par Boulet (1988a) consiste à permettre à un usager de démarrer un processus de modélisation conceptuelle de données. Le deuxième objectif d'unité consiste à permettre à un usager d'identifier et de représenter à l'aide du formalisme Merise les individus d'un modèle conceptuel de données, leur identifiant et leur liste de propriétés. Le troisième objectif d'unité vise à développer les mêmes capacités que celle énoncées pour le deuxième objectif mais cette fois pour les relations d'un modèle conceptuel de données. Le quatrième objectif d'unité consiste à permettre à un usager de poser adéquatement toutes les cardinalités d'un modèle conceptuel de données. Chaque objectif d'unité se décompose en objectifs terminaux. Ainsi, toujours selon Boulet (1988a), l'objectif d'unité 1 se découpe en deux (2) objectifs terminaux. Ces objectifs visent à développer la capacité d'expliquer ce qu'est un modèle conceptuel de données et celle d'établir la liste des propriétés. Chacun des objectifs terminaux est ensuite découpé en objectifs intermédiaires. Le premier objectif terminal mentionné plus avant comporte quatre objectifs intermédiaires (Boulet, 1988a): l'usager doit premièrement définir ce qu'est un modèle conceptuel de données, deuxième-ment énoncer la philosophie à la base de la modélisation conceptuelle des données, troisièmement énoncer le rôle du modèle conceptuel de données, et
finalement en énumérer les éléments fondamentaux. Possédant la structure de préalables inhérente à la capacité de concevoir un modèle conceptuel de données, il a été possible d'énoncer les explications associées aux éléments les plus fins de la hiérarchie, c'est-à-dire aux objectifs intermédiaires. Pour ce faire, le principe de séparation des connaissances et des traitements à appliquer a été utilisé. Ainsi, les diverses questions posées par ou à l'usager et les explications fournies sont situées par le système conseiller dans les matrices correspondant à la hiérarchie de préalables associée. Avant de répondre à la question posée par l'usager ou de procéder à l'analyse de la réponse faite par un usager à une question du conseiller, le conseiller examine les valeurs de chacune des matrices et, en fonction du nombre de fois auquel l'usager a préalablement référé au même contenu, décide de répondre directement à la question ou de plutôt lui suggérer la revision d'un ensemble de concepts. Pour l'instant, les seuils de réussite servant à déterminer le nombre de fois que le système conseiller répondra à une même question ont été fixés à 80% suivant la théorie de la maîtrise des apprentissages (Block et Anderson, 1975). L'étape de validation nous permettra d'ajuster ceux-ci.

Illustrons ceci à l'aide d'un exemple simplifié puisqu'il serait très long d'expliciter les changements effectués dans chacune des matrices. Supposons un usager utilisant un logiciel d'application graphique pour élaborer un modèle conceptuel de données suivant la méthode Merise. Cet usager questionne le conseiller sur les cardinalités qu'il a établies. Au fur et à mesure qu'il progresse, un certain nombre de matrices se construisent. Nous illustrerons une partie de celles concernant l'objectif "Associer adéquatement à un contexte un couple de paires de cardinalités".

Tout d'abord, le système construit en mémoire la matrice présentée à la figure 3 puis calcule le nombre total de couples de paires de cardinalités utilisés dans le modèle conceptuel de données en additionnant chacune des cellules de la matrice. Aux fins de notre démonstration, il y en a 60. Le système utilise cette valeur pour déterminer le nombre de consultations permises au total pour les couples de paires de cardinalités soit 16 en vertu du seuil de réussite 80% mentionné précédemment.

<table><caption>Figure 3.
Partie d'une matrice se construisant en mémoire concernant le nombre de couples de paires de cardialités dans le modèle élaboré par</caption><thead><tr><th rowspan="2">Paire sur</th><th colspan="4">Datte droite</th></tr><tr><th>0,n</th><th>1,n</th><th>1,1</th><th>(1,1)</th></tr></thead><tbody><tr><td>patte</td><td>0,n<br>1,n</td><td>12<br>20</td><td>16<br>12</td><td></td></tr><tr><td>gauche</td><td>1,1<br>(1,1)</td><td></td><td></td><td></td></tr></tbody></table>
Il peut être observé à la figure 3 que le système possède également des données sur la fréquence de chaque couple de paires de cardinalités soit 12 couples 0,n - 1,n, 16 couples 0,n - 1,1, etc. A partir de ces données une autre matrice, présentée à la figure 4, servant à déterminer le nombre de consultations permises pour chaque couple de paires de cardinalités en fonction du seuil préétabli se construira parallèlement.

<caption>Figure 4.</caption>

P artie de la matrice ayant trait aux consultations permises à l'usager.

<table><thead><tr><th rowspan="2" colspan="2">Paire sur</th><th colspan="3">patte droite</th><th rowspan="2"></th></tr><tr><th>0,n</th><th>1,n</th><th>1,1</th><th>1,1</th></tr></thead><tbody><tr><th rowspan="2">patte</th><th>0,n</th><td></td><td>3</td><td>3</td><td></td></tr><tr><th>1,n</th><td></td><td>4</td><td>3</td><td></td></tr><tr><th rowspan="2">gauche</th><th>1,1</th><td></td><td></td><td></td><td></td></tr><tr><th>1,1</th><td></td><td></td><td></td><td></td></tr></tbody></table>

Toujours à partir de la matrice dont la forme est rappellons-le inspirée du schème à facettes multiples, le nombre total de paires de cardinalités utilisées dans le modèle conceptuel de données est calculé en faisant la somme des lignes et colonnes. La valeur calculée pour la paire 0,n sera 28 et ainsi de suite. En conséquence, les consultations permises pour la paire de cardinalités 0,n considérant le seuil de réussite sera de 6. Deux autres matrices non illustrées ici et dont la forme s'inspire encore une fois du schème à facettes multiples seront élaborées pour d'une part, mémoriser le nombre de cardinalités minimum 0,1, c et n, le nombre de cardinalités maximale 1, c, n, et d'autre part, mémoriser le nombre de consultations permises pour ces éléments à un niveau plus bas de la hiérarchie de préalables.

Avant que ne soit déclenché le processus global d'activation du niveau inférieur, soit les connaissances relatives à l'objectif d'unité 3, l'usager pourra effectuer au total 12 consultations compte tenu que son modèle comporte 60 couples de cardinalités. En fait, ce niveau inférieur réfère à une autre hiérarchie d'apprentissage non illustrée ici.

L'usager utilise 12 fois dans son modèle le couple 1,n - 1,1. Ainsi, en vertu du seuil de réussite fixé à 80%, il a droit à 3 consultations concernant spécifiquement ce couple sinon, on le réferera au concept de paires de cardinalités. Supposons que notre usager demande au conseiller de lui aider à vérifier si un couple de cardinalités 1,n - 1,1 est valable par rapport à ce qu'il perçoit de la réalité. Toutes les matrices représentant le cheminement de l'usager seront mises à jour tel qu'illustré dans ce qui suit.

En premier lieu, les matrices ayant trait aux consultations effectuées pour chaque couple de paires de cardinalités seront construites; il s'agit du modèle
de l'usager dont un extrait est présenté à la figure 5. Nous nous en tenons ici à la présentation d'une partie de la matrice concernant les couples de paires de cardinalités.

<figure><img src="image_3.png" /><figcaption>Figure 5. Partie d'une des matrices ayant trait aux couples de paires de cardinalités correspondant au modèle de l'usager.</figcaption></figure>

Ainsi, une lecture effectuée par le système lui permet de constater qu'une seule consultation a été effectuée au total pour tous les couples de cardinalités (somme de toute la matrice) et pour le couple 1,n - 1,1 (valeur de l'intersection 1,n - 1,1)

Comme c'est la première fois que l'usager réfère à cet élément de connaissance, le conseiller lui présentera une explication reliée directement à la question posée. Supposons maintenant que le même usager demande à trois autres reprises de valider des couples 1,n - 1,1; même une personne humaine s'interrogerait sur sa compréhension. Une mise à jour continuelle du modèle de l'usager étant effectuée, la matrice représentant ce dernier contiendra les valeurs présentées à la figure 6.

<figure><img src="image_4.png" /><figcaption>Figure 6. Actualisation de la matrice correspondant au modèle de l'usager.</figcaption></figure>

Les résultats des divers calculs effectués par le système lui permettra de se rendre compte que le nombre de consultations effectuées (4) est plus grand que le nombre permis (3) pour le couple 1,n - 1,1 mais que les autres seuils ne
sont pas atteints. Il prendra la décision de diriger l'usager vers la partie de la base de connaissances portant sur les paires de cardinalités, à un niveau plus bas de la structure de préalables. Notons que d'autres matrices vont contin- uellement se construire et être mises à jour tout au long du cheminement de l'usager.

L'usager peut également décider lui-même et au moment où il le juge utile de consulter une autre portion de connaissances. Afin de lui permettre ces consultations, nous avons conçu une structure d'inspiration Hypertext pour les explications présentées à l'usager. Cette structure d'inspiration Hypertext consiste à afficher en relief les mots correspondant à des notions préalables à la compréhension de l'explication; l'usager peut, à l'aide de la souris, se positionner sur un mot dont il a oublié la signification et appuyer sur le bouton de la souris. Le conseiller à ce moment chargera en mémoire les matrices correspondant aux structures de préalables en cause, vérifiera les valeurs des matrices et affichera les explications pertinentes. Un extrait de contenu d'une intervention du conseiller est présenté dans ce qui suit:

> "15% de votre modèle conceptuel de données comporte des couples de paires de cardinalité 0,n - 0,n. Cela signifie que les deux individus-types (ici le conseiller affiche le nom des individus-types impliqués dans la relation. Nous les appellons ici A, pour l'individu de gauche et B, peur l'individu de droite) A et B peuvent exister dans la mémoire sans que la relation (ici le nom de la relation du modèle conçu par l'usager apparaît. Nous l'appellons R) R n'existe. Par contre, lorsque la relation R existe, les deux..."

Il peut être observé qu'à l'intérieur de l'explication présentée plus haut et référant à un objectif intermédiaire de l'objectif d'unité 4 soit à un couple de paires de cardinalités 0,n - On, les concepts d'individus-types et de relation sont impliqués. Le concept d'individus-types fait référence à l'un des objectifs intermédiaires de l'objectif d'unité 2 et le concept de relation à l'un des objectifs intermédiaires de l'objectif d'unité 3.

## Module d'acquisition des connaissances

Un des buts visés par la conception du module d'acquisition des connaissances est de permettre la collecte de cas réels. Ces cas seront par la suite utilisés lors des interventions du conseiller; ces cas visent à permettre à l'usager de mieux saisir les explications théoriques puisqu'elles seront applications d'exemples référant au domaine pour lequel le modèle conceptuel des données est élaboré. Ainsi, un autre des buts visés est de permettre la diffusion de l'expertise ayant trait à l'élaboration du modèle conceptuel des données d'un secteur d'activité particulier.

La décision d'élaborer un module d'acquisition des connaissances de la part d'experts en modélisation conceptuelle des données a rendu nécessaire la détermination d'une structure de présentation des interventions du conseiller. En effet, puisque les experts seront des personnes différentes, s'exprimant
différemment et de surcroit non spécialisées en théorie de la communication, nous devions être en mesure de les encadrer lors de la formulation de leurs exemples, ceci en vue d'assurer une présentation uniforme des explications présentées par le système conseiller. Nous avons en conséquence consulté la littérature ayant trait à la communication dans un cadre d'enseignement et d'apprentissage en vue notamment de fournir aux experts un encadrement optimal.

Les théoriciens de la communication s'entendent sur le fait que, pour qu'un message exprimé par un émetteur, dans notre cas le module d'intervention du conseiller informatisé, soit reçu et enregistré par un récepteur, dans notre cas l'usager du système conseiller en modélisation conceptuelle des données, l'émetteur et le récepteur doivent utiliser le même code. Cependant, l'application de cette théorie est plus complexe lorsque le but de l'émetteur est de faire acquérir de nouvelles notions au récepteur; c'est justement le but de l'émetteur "système conseiller". Il faut à ce moment distinguer dans le vocabulaire du système conseiller les mots du vocabulaire courant connus de l'usager utilisés pour lui transmettre des concepts nouveaux des mots correspondants à l'apprentissage et en conséquence nouveaux pour l'usager et que l'on désire lui faire apprendre.

Les spécialistes de la communication nous recommandent donc de choisir parmi les mots les plus usuels ceux qui serviront à formuler les explications d'un phénomène (Richaudeau, 1975). C'est pour respecter cette théorie issue de diverses recherches que nous avons décidé de rendre possible la présentation par le conseiller informatisé d'exemples ayant trait au domaine pour lequel le modèle conceptuel de données est conçu et, en conséquence, de développer le module d'acquisition des connaissances dont il est question ici.

Les mots dits nouveaux sont ceux spécifiques à chaque matière comme par exemple les termes individu, relation, cardinalité, occurrence, contrainte d'intégrité fonctionnelle utilisés dans le domaine de la modélisation conceptuelle des données. Pour les mots nouveaux, il ne s'agit plus d'adapter le vocabulaire à l'usager mais plutôt de lui apprendre un nouveau vocabulaire tout en tenant compte de ses capacités psychologiques et linguistiques à retenir ceux-ci. Ici encore, les théoriciens de la communication nous recommandent de suivre certaines règles.

1. Il faut doser l'apparition des mots nouveaux et veiller à ce que chaque intervention n'en comporte qu'un nombre réduit.

2. Il faut bien définir chaque mot nouveau, même s'il s'agit de mots usuels. Les spécialistes de la communication signalent que pour les mots usuels, leur insertion dans une phrase peut suffire à en faire deviner la signification au récepteur; ils précisent qu'il vaut cependant mieux sous-estimer cette faculté du récepteur et s'astreindre soit à faire précéder le nouveau mot usuel d'un mot synonyme, soit le définir en utilisant dans le texte des formules du type "c'est-à-dire" ou "ce qui veut dire.*
3. Il faut veiller à répéter chacun de ces nouveaux mots au cours de la même intervention (si cela est possible) et au cours des interventions qui suivent.

En fonction de ces considérations, les fonctions du module d'acquisition des connaissances se décrivent comme suit:

1. Le module d'acquisition des connaissances permettra de balayer complètement un modèle conceptuel des données entré par un expert dans le but de recueillir les informations néces saires à la production d'exemples réels et collant à la réalité que l'usager tente de modéliser.

2. Dans le but d'assurer l'uniformité des exemples, le module d'acquisition des connaissances obligera les experts à fonctionner à l'intérieur d'un cadre. Nous décrivons dans ce qui suit cette forme d'encadrement.

En ce qui a trait aux informations utilisées dans les exemples, le premier type sert à situer le contexte du modèle (Figure 7). Ce contexte se présente selon l'ordre suivant:

1. Sorte d'organisation;

2. Sorte de clientèle;

3. Sorte de transaction;

4. Description de la transaction et de ce que l'organisation retient de cette transaction; et

5. Spécimen de la transaction si applicable.

<figure><img src="image_5.png" /><figcaption>Figure 7.
Exemple d'une fenêtre du module d'acquisition des connaissances.</figcaption></figure>
Le contexte étant précisé, la partie visant le transfert des connaissances théoriques à leur application pratique est ensuite présentée. Cette partie est composée de phrases utilisées lors des explications théoriques présentées en réponse à certaines questions auxquelles sont ajoutées des phrases ayant trait au contexte. Nous présentons dans ce qui suit, un exemple de structure de ces phrases.

*Explication numéro 21170.* Tout d'abord, on procède à la présentation du contexte organisationnel selon le principe 1 énoncé plus avant. Par la suite, si un spécimen de transaction effectuée par l'organisation est disponible, il sera graphiquement saisi par le module, et le texte suivant sera par la suite affiché:

L'on peut observer que:

* [...] est une occurrence de la propriété-type [...]

* [...] est une occurrence de la propriété-type [...]

et ainsi de suite.

Signalons que ce qui est placé entre crochets est précisément ce que l'expert devra compléter.

### *Interface avec le monde extérieur*

La figure 8 (voir page suivante) présente la fenêtre de l'outil de modélisation apparaissant au démarrage. L'installation du système conseiller provoque un ajout à la barre de menu située au haut de la fenêtre soit l'option CONSEILLER. Dans ce qui est illustré, on peut remarquer que l'usager a débuté la conception d'un modèle conceptuel de données. Trois individus ont été créés à l'aide de la palette d'outils située au haut à gauche de l'écran, soit ETUDIANT, ADRESSE ET COURS ainsi qu'une relation SUIT et une autre relation A. Par ailleurs, la figure 9 (voir page suivante) présente l'un des menus présenté à l'usager après qu'il ait indiqué au conseiller qu'il désirait obtenir des informations ayant trait aux cardinalités de la relation SUIT.

## CONCLUSION

Dans cet article, nous avons présenté les grandes lignes des principes suivis lors de la conception d'un système conseiller intelligent. Dans le futur, d'autres modules seront ajoutés au système. Ainsi, un module d'explication des faiblesses décelées chez l'usager sera ajouté; le système conseiller sera en mesure d'expliquer à l'usager les raisons pour lesquelles il a pris certaines décisions ayant trait, par exemple, à la sélection des explications qui ont été présentées. Une prescription d'apprentissage correctif sera également générée par le système, en fonction du modèle de l'usager et sera présentée à l'usager en termes d'impacts futurs sur sa capacité effectuer correctement et rapidement un modèle conceptuel de données. Un autre module visant à gérer la présentation des explications à l'écran sera également ajouté; pour l'instant,
<figure><img src="image_6.png" /><figcaption>Figure 9. Partie d'une fenêtre permettant à l'usager d'obtenir des explications.</figcaption></figure>

<figure><img src="image_7.png" /></figure>
nous en sommes à la revue de littérature ayant trait aux interfaces personne/
machine pour les systèmes conseillers.

RÉFÉRENCES

Block, J.H., et Anderson, L.W. (1975). *Mastery learning in classroom instruction*. New York, NY: MacMillan.

Boulet, M.M. (1988a). *Système conseiller: Analyse de l'objectif d'unité 1*
"Démarrer un processus de modélisation conceptuelle des données."
Rapport de recherche DIUL-RR-8815. Département d'informatique.
Faculté des Sciences et de génie, Université Laval.

Boulet, M.M. (1988b). *Système conseiller: Analyse de l'objectif d'unité 2*
"Identifier et représenter à l'aide du formalisme Merise les individus d'un
modèle conceptuel de données, leur identifiant et leur liste de propriétés."
Rapport de recherche DIUL-RR-8816. Département d'informatique.
Faculté des Sciences et de génie, Université Laval.

Boulet, M.M. (1988c). *Système conseiller: Analyse de l'objectif d'unité 3*
"Identifier et représenter à l'aide du formalisme Merise les relations d'un
modèle conceptuel de données, leur identifiant et leur liste de propriétés."
Rapport de recherche DIUL-RR-8817. Département d'informatique.
Faculté des Sciences et de génie, Université Laval.

Boulet, M.M., et Barbeau, L. (1989). *Cardinalités: Document d'analyse de l'utilisation descardinalités suivant le formalisme Merise en vue d'élaborer le module d'intervention d'un système conseiller*. Rapport de recherche à paraître. Département d'informatique. Faculté des Sciences et de génie. Université Laval.

Carroll, J.M., & McKendree, J. (1987, January). *Interface design issues for advice-giving expert systems*. *Communications of the ACM*, *30* (1), pp. 14-31.

Fisher, G., Lemke, G., & Schwab, T. (1985). Knowledge-based help systems. *CHI'85 Proceedings*, pp. 161-167.

Gagné, R.M. (1968, November). Learning hierarchies. *Educational Psychologist*, *6*, pp. 1-9.

Gagné, R.M., Mayor, J.R., & Garstens, H.L., Paradise, N.E. (1962). Factors in acquiring knowledge of a mathematical task. *Psychological Monographs*, 76 (7), pp. 1-19.

Genesereth M.R. (1982). The role of plans in intelligent teaching systems. In D. Sleeman & J.S. Brown (Eds.) *Intelligent tutoring systems*. London: Academic Press Ltd.

Hayes-Roth, F., Waterman, D.A., & Lenat, D.B. (1983). *Building expert systems*. Addison-Wesley Publishing Company Inc.

Hill, W.C., & Miller, J.R. (1988, May). Justified advice: a semi-naturalistic study for advisory strategies. *Proceedings of CHI'88: Human Factors in Computing Systems.*, Association for Computing Machinery, pp. 185-190.
Kaemmerer, W.F.,&Allard, J.R. (1987). An automated reasoning technique for providing moment-by-moment advice concerning the operation of a process. *Review of American Association for Artification Intelligence*, pp. 16-28.

Khanna, R., & Moore, R.L. (1987, July). Expert systems involving dynamic data for decisions. Second *International Expert Systems Conference, Association for Computing Machinery*, pp. 34-40.

Leitch, R.R. (1987, July). Modelling of complex dynamic systems. *The Institute of Electrical and Electronics Engineers Proceedings, 134*, (4), Pt. D.

McCalla, G.I., Bunt, R.B., & Harms, J.J. (1986). The design of the SCENT automated advisor. *Computational Intelligence, 2* (2), pp. 13-27.

Milnes, R. (1987, July). Artificial intelligence for online diagnosis. *The Institute of Electrical and Electronics Engineers Proceedings, 134* (4), Pt. D.

Richaudeau, F. (1975). Les *secrets de la communication efficace*. Paris: RETZEPL.

Rodriguez, G., & Rivera, P. (1986, July). Apractical approach to expert systems for safety and diagnostics. *InTech*, pp. 9-16.

Rowan, D.A. (1987). On-line fault diagnosis: initial success and future directions. *Proceedings of the ISA'87 International Conference and Exhibit*, pp. 1211-1218.

Sauers, R., & Walsh, R. (1983). On the requirements of future expert systems. *International Joint Conference on Artificial Intelligence, American Association for Artificial Intelligence*, pp. 306-320.

Scallon, G. (1988). L'évaluation *formative des apprentissages: L'instrumentation*. Presses de l'Université Laval: Québec, QC.

## AUTEUR

Marie-Michèle Boulet est une professeure agrégée au Département d'information, Université Laval, Québec, QC G1K 7P4.

REMERCIEMENTS: Ce projet de recherches beneficie d'une subvention du Centre Canadien de Recherches sur l'Informatisation du Travail (CCRIT) et d'un support technique de IBM-Canada. Il est une partie d'un projet plus global dont les autres partenaires sont le Conseil National de Recherches Canada, l'Université de Leeds, Price Waterhouse et XA system. Les étudiants gradués Luc Barbeau, Sergio Slobodrian et Jacques ont contribué à la programmation du système.
Examination of Cognitive Style FD/FI as
a Learner Selection Criterion in
Formative Evaluation

Chris A. Chinien

**Abstract:** The selection of students to provide feedback for reviewing and revising instructional materials, during the developmental stages is critical. However, there is little empirical evidence regarding the most appropriate students' selection criteria for the process. This paper describes the experimental phase of a study designed to examine the effects of using the cognitive style construct field-dependent/field-independent (FD/FI) as a student selection criterion for evaluating prototypical materials. A prototype version of a computer assisted instruction was evaluated and revised on the basis of feedback generated by field-dependent/independent students. Results for achievement indicated significant main effects for treatment and cognitive style. The revised version of the instructional material was more effective than its prototype. Mean achievement scores of FD and FI students exposed to the prototype were significantly different. However, the evidence observed did not support a differential effect of the revised instructional material for FD and FI students. No significant main effects of treatment and cognitive style were found for study time and attitude.

INTRODUCTION

In spite of considerable progress made in the field of instructional devel-
opment, through the adoption of systematic instructional design process,
practitioners still have difficulty in producing efficient and effective first draft
products (Dick & Carey, 1985; Gagne & Briggs, 1979). Conscious of this
inherent difficulty, and recognizing that the design process is not foolproof
instructional developers have included a formative evaluation component in
their models (Baggaley, 1986; Geis, Weston, & Burt, 1984; Weston, 1986). The
purpose of formative evaluation is to provide instructional developers with an
opportunity to identify errors and problems within a set of instructional
materials while they are still in a developmental stage (Baker & Alkin, 1984;
Dick & Carey, 1985; Gagne & Briggs, 1979; Geis, 1987; Sanders & Cunningham, 1973). Nathenson and Henderson (1980) note that the formative evaluation of instructional products is an essential activity in the design and
development of instruction, due to a lack of a comprehensive theory of learning
to guide practice. Two broad questions are addressed by formative evaluation. The first relates to content and technical quality of the material, and the second pertains to how easily it can be learned. The evaluation of content and technical quality is addressed through expert verification and revision, whereas it is generally believed that students are most qualified for providing feedback data to assess ease of learning.

Dick and Carey (1985) suggest a three stage sequential approach for evaluating instructional materials using students' feedback data:

1) one-to-one;

2) small groups; and

3) field testing.

The one-to-one evaluation occurs in the earliest phase of product development. The purpose is to try out draft materials with individual students in order to identify and revise most obvious errors and problems. The small group evaluation is performed with small groups of students when the materials are almost in the final form. The intent is to: (a) validate modifications made to the materials during the one-to-one evaluation, and (b) identify and correct any additional errors and problems in the materials. When the materials are in semi-final form they are field tested to verify the effectiveness of previous verifications and revisions performed during earlier phases of evaluation. Field testing is conducted in a setting which approximates the actual context for which the materials were designed (Weston, 1987). The process helps to ascertain if the materials will function smoothly, and whether they will be accepted by students, teachers, and administrators.

Although the importance of formative evaluation is well documented in the literature, the state-of-the-art is still an underdeveloped and under-conceptualized field of inquiry. There is a paucity of research in formative evaluation, and many aspects of the process still need the illumination of sound research efforts (Baker & Alkin, 1984; Cambre, 1981; Geis, 1987). One aspect of formative evaluation which has received little research attention is the student selection criteria for participation in the process (Dick, 1980; Wager, 1980). Dick (1980) argues that the characteristics and profiles of students participating in earlier stages of formative evaluation are critical for improving instructional products. Only two studies have investigated the student selection criteria for the one-to-one formative evaluation. Berthelot (1978) studied the student's level of verbalization, and Wager (1980) examined the student's ability level as selection criteria.

The cognitive style construct field-dependent/field-independent (FD/FI) constitutes an important dimension of individual differences among students, regarding their preferences for various modes of gaining, storing, processing and using information (Messick, 1985). This construct should be an important student selection criterion, with promising potential for the one-to-one formative evaluation process. However, it has so far been overlooked by researchers
and practitioners in the field of instructional design and development (Ausburn & Ausburn, 1978). The purpose of this study was to determine the effectiveness of the cognitive style construct FD/FI as a student selection criterion for the one-to-one formative evaluation. More specifically, this study attempted to answer the following questions:

1) Is an instructional material revised on the basis of feedback generated by FD and FI students in the one-to-one formative evaluation process more effective than its original prototype? and

2) Is an instructional material revised on the basis of feedback generated by FD and FI students in the one-to-one formative evaluation process, effective for both FD and FI students?

## CONCEPTUAL FRAMEWORK

The conceptualization of the theoretical framework for this study emerged from the review of literature and research on the cognitive style construct field-dependent/independent, instructional material effectiveness, and paradigms of inquiry. The most significant literature and research in these three areas are briefly reviewed in this section.

### Cognitive Style FD/FI

A considerable body of knowledge that has been accumulated on cognitive style (Witkin & Goodenough, 1981), and various cognitive styles have been identified. One of the cognitive style dimensions which has been most extensively researched is the field-dependent/independent construct. FD/FI refers to the psychological differentiation which describes the source of experiences that is used when a person enters a new situation (Witkin, Moore, Goodenough, & Cox, 1977). It represents "a global versus an analytical way of perceiving. It entails the ability to perceive items without being influenced by background" (Kirby, 1979, p. 52). Research has shown the FD and FI students differ in important ways with respect to personality factors, perceptual patterns, and social interaction (Ausburn & Ausburn, 1978).

Field-dependent individuals are drawn to people and like to have people around them. They exhibit more non-verbal behaviors; prefer occupations which require involvement with others (e.g., social sciences); and demonstrate a preference for academic areas that are people oriented (e.g., teaching, selling). In contrast, relatively field-independent persons demonstrate a preference for impersonal and abstract school subjects (e.g., mathematics and physical sciences); they are more impersonal and prefer occupations in which interaction with others is less important (e.g., astronomy, engineering, mathematics, sciences and architectural domains) (Witkin & Moore, 1974). Relatively field-dependent persons have a global perception and take a long time to solve problems (Witkin & Moore, 1974). They use "external referents for self-
definition, make less use of mediators in the coding process of knowledge acquisition and prefer a spectator approach to concept attainment" (Caliste, 1985, p. 26).

Furthermore, individuals who are very field-dependent are extremely alert to social cues, have highly developed interpersonal skills, and like to study in groups. In addition, they are extremely sensitive to social criticism and are extremely influenced by others around them (Witkin & Goodenough, 1981; Witkin & Moore, 1974). In contrast the relatively field-independent persons are more abstract-analytical, tend to solve problems rapidly and learn better when content is abstract (Witkin & Moore, 1974). They use "internal referents for self-definition, make frequent use of mediators and engage in a hypothesis-testing approach to concept attainment" (Caliste, 1985, p. 26). The field-independent individuals tend to be "aloof, theoretical and not sensitive to others around them. They will restructure any random or non-hierarchically presented information for better retention and retrieval" (Wallace & Gregory, 1985, p. 22).

Earlier studies (Witkin, Lewis, Hertzman, Machover, Meissner, & Wapner, 1954) indicated that the cognitive style FD/FI was related to gender. Males appeared to be more field-independent than females. However, evidence from more recent studies indicates that gender differences in field-dependence is inconclusive (Naditch, 1976).

The cognitive style FD/FI appears to have important implications for formative evaluation. Ausburn and Ausburn (1978) state: "cognitive style is important as a learner characteristic relevant to instructional design because it influences the ability to process information or learning" (p. 344). Ragan & others, (1979) argue that since cognitive style determines the way information is acquired and processed, "an individual may encounter tasks that require the processing of information in a way that they are unable to accomplish, simply because their cognitive style restricts the availability of the processing technique" (p. 2). This argument suggests that instructional materials can be cognitive style based.

The cognitive style FD/FI has some other particular characteristics which makes it an important variable as a student selection criterion:

1) it is stable over time and across tasks;

2) it is value neutral and bipolar; and

3) it correlates with students' performance in various school subjects. In addition, valid and reliable instruments which can be easily administered and scored are available for assessing the construct (Witkin & Goodenough, 1981).

## Instructional Material Effectiveness

Instructional material effectiveness can be viewed within a framework encapsulating three major elements: 1) achievement; 2) study time; and 3) student attitude toward the material. Previous studies have shown that
formative evaluation can significantly improve achievement (Abedor, 1972; Baker, 1970; Gropper, 1967; Kandaswamy, 1976; Rosen, 1968; Silberman, Coulson, Melargno, & Newmark, 1964; VanderMeer, Morrison & Smith, 1965; Wager, 1980), and improve performance, while keeping study time constant (VanderMeer, 1964; VanderMeer & Montgomery, 1964; VanderMeer & Thorne, 1964; Sulzen, 1972). Abedor (1972) and Silberman et al., (1964), found that formative evaluation can also lead to improved achievement with a decrease in study time. Studies conducted by Abedor (1972), Stolovitch (1975), and Wager (1980) indicate that students tend to have a more positive attitude toward revised instructional materials as compared to prototype versions. Romiszowski (1986) however, argued that the novelty effect can confound measures of students' attitudes in formative evaluation. Additionally, Nathenson and Henderson (1980) indicate that it is difficult to quantitatively assess students' attitude toward an instructional product.

## Paradigms of Inquiry

The methodological difficulties commonly encountered in conducting research in formative evaluation can in part be attributed to the basic axiomatic assumptions of the rationalistic paradigm of inquiry (Berthelot, 1978; Dick, 1977; Geis et al., 1984). Shrock (1984) argues that the rationalistic paradigm of inquiry is constraining, and that instructional developers should consider alternate paradigms which are more facilitating. She strongly advocates a shift to the naturalistic paradigm of inquiry. The interest for the naturalistic inquiry has emerged because the rationalistic approach has apparently failed to provide enlightened information and explanations related to highly complex problems, involving human interactions and values (Lincoln & Guba, 1985).

The rationalist assumption regarding the relationship between knower and known is problematic and constraining in formative evaluation research. One inherent problem associated with this assumption is the reviser's effect. Critics of formative evaluation research argue that in the majority of these studies it is impossible to partial out how much of the revisions which contributed to the effectiveness of the materials were guided by students' feedback data, and how much by the evaluators' intuitions, finesse, perspective and insights (Baker, 1970; Baker & Atkin, 1984; Nathenson & Henderson, 1980). Additionally, the observation that different revisers can produce differentially effective revisions (Kandaswamy, Stolovitch & Thiagarajan, 1976), is also problematic under the rationalistic paradigm.

One-to-one formative evaluation is a highly interactive process between the evaluator, test subjects and instructional The rationalist assumption regarding the relationship of knower to known is not amiable to formative evaluation research. On the other hand, the naturalist claim for subject-object interrelatedness provides a better fit to process. Lincoln and Guba (1985) argue that: "the inquirer and the 'object' of inquiry interact to influence one another, knower and known are inseparable" (p. 37). The
inquirer cannot be removed from the phenomena, objects or subjects being studied. Guba (1979) argues:

> ...it is fruitless to act as though that interactivity were not there; a more intelligent approach requires understanding the possible influences of such interactivity and taking it into account. (p. 270)

Guba further indicates that the naturalistic inquirer can capitalize on the richness of interactivity to acquire a better understanding of the phenomena being studied.

Lincoln and Guba (1985) have also identified a set of strategies to guide naturalistic inquirers in establishing the trustworthiness of the inquiry process. These strategies are: member check, peer debriefing, triangulation, referential adequacy and audit checks.

# METHODS

## Instructional Material

The instructional material used in this study was a prototype version of a computer-assisted instruction (CAI), entitled Superelevation. This material was designed to teach the application of the principles of superelevation in road design, to third-year university students enrolled in a Landscape Architecture course.

This CAI package was designed as a self contained tutorial, consisting of 1) a set of objectives; 2) the linear presentation of subject matter content, and 3) a quiz on the content. All major concepts presented were supported by colour graphics and animations.

Several factors influenced the selection of Superelevation as an instructional treatment for this study. First, this material dealt with several abstract concepts. Research has shown FI students tend to outperform FD students in learning abstract concepts (Witkin et al., 1977). Secondly, the delivery medium was the computer assisted instruction. Research indicates that CAI appears to be more effective for FI students (Post, 1984). Thirdly, Superelevation is self-contained, therefore many extraneous variables were eliminated or controlled. Additionally, this material was reviewed and judged appropriate for this study by four expert judges.

## Research Design

A two-stage approach was used to test the hypotheses formulated for this study. During the first stage the prototype version of Superelevation was evaluated and revised on the basis of feedback generated by FD and FI subjects, using the naturalistic inquiry paradigm (Lincoln & Guba, 1985). During stage two, the revised version of Superelevation was validated to determine any gain in effectiveness over the original prototype.
A paradigm mix (naturalistic and rationalistic) was found to provide a good fit for this formative evaluation study. Because of the highly interactive nature (subject/evaluator) of one-to-one evaluation, the rationalistic paradigm of inquiry was found to be too constraining (Berthelot, 1978; Geis et al., 1984), and the naturalistic inquiry emerged as a paradigm of choice, which was more facilitating. However, the rationalistic paradigm appeared to be more amenable to the experimental phase of the study, which was designed to evaluate the overall effectiveness of the revised product.

The second research stage was conducted as an experiment. A two factor treatment by block design with an outside no-treatment control group (Keppel, 1982) was used. This design is depicted in Figure 1. The treatment factor had three levels: 1) revised version of Superelevation; 2) unrevised version of Superelevation; and 3) no treatment control condition. The second factor had two levels representing the subjects cognitive style dimension: 1) field-dependent and 2) field-independent. The factors were fixed and completely crossed.

<figure><img src="image_8.png" /><figcaption>Figure 1.
Treatment x Block Design.</figcaption></figure>

The dependent variables for this study were: 1) student achievement as evidenced by the group mean achievement scores on an objective type posttest; 2) study time operationalized as the mean time that each group spent interacting with the instructional treatment; and 3) student attitude toward the instructional material was measured as the mean scores on a Likert-type attitude questionnaire (1 - Strongly Disagree; 2 - Disagree; 3 -Agree; and 4 - Strongly Agree). The reliability of the posttest and attitude questionnaire was .78 (Spearman Brown) and .91 (Cronbach Alpha) respectively.
## *Research Hypotheses*

One research hypothesis was formulated for each of the three indicators of instructional material effectiveness identified in the literature: a) achievement; b) study time; and c) attitude. It was hypothesized that there will be significant differences between FD and FI students interacting with the prototype and a revised version of Superelevation (modified on the basis of feedback generated by FI and FD subjects) in: a) achievement; b) study time; and c) attitude toward the materials. These three hypotheses were tested at the .05 level of confidence.

## *Subjects*

Subjects for this study represented of the entire population of third-year university students (N = 46) enrolled in a Landscape Architecture course, during the Fall session, 1986. This population was treated as a sample in time of a broader population for which Superelevation was specifically designed. The Hidden Figures Test (HFT) (Ekstrom, French, Harman & Derman, 1976) was administered to the population to determine their cognitive style dimension. Subjects were classified as FD and FI using a median split technique.

A purposive sample consisting of 2 males and 2 females (one FD and one FI each) was selected from the population for the one-to-one formative evaluation stage. Subjects were selected on the basis of their extreme HFT scores. The remaining subjects were assigned to six equal groups (3 FD and 3 FI), using a randomized blocking procedure.

# PROCEDURE

## *Evaluation and Revision*

Formative evaluation data on Superelevation were gathered while each of the four subjects individually interacted with the material during the one-to-one tryouts. Subjects were requested to read aloud and to verbalize their thought process as they interacted with the content. This think aloud procedure was used to provide insights into the information processing demands that the material imposed upon the subjects.

Subjects were encouraged to ask questions whenever they had any difficulty. Furthermore when the researcher sensed that a subject was having difficulty which was not verbalized, he probed for additional feedback.

During the one-to-one evaluation sessions, data were also gathered through informal observation of non-verbal behaviours exhibited by the subjects while interacting with Superelevation. Additionally, all subjects were debriefed at the end of the evaluation sessions.

The researcher recorded all feedback provided by subjects. The feedback were interpreted and taken back to the subjects for confirmation and refinement. When errors and problems requiring minor revisions were identified, the researcher immediately made the necessary modifications on hard copy of
the corresponding screens. The revised segment was presented again to the subjects in order to confirm the correct interpretation of their feedback data, and to validate the revision decisions.

Each evaluation session was audio taped. These tapes provided vivid episodes of the evaluation. All tapes were also transcribed for content analysis.

Following each evaluation session the researcher wrote field notes and was debriefed by a peer, who in this instance was also the designer of Superelevation. Data collected were analyzed and translated into revision decisions for modifying Superelevation. The subjects' feedback data along with proposed revisions were taken to the peer for confirmation and validation. Appropriate modifications were made when any discrepancy was observed. The major themes evolving the analysis of feedback data as well as the corresponding revision decisions are outlined in Table 1.

The prototype material was revised by the designer. The revised version of the material, the student feedback data and the corresponding revision decisions were taken to an audit for a final confirmation and validation. A graduate student knowledgeable in the field of instructional design and with training in qualitative research methods was used as audit.

## Experimental Phase

The three experimental conditions (revised/unrevised versions of Super-elevation and no treatment control) were randomly assigned to each of the three groups of FD and FI subjects. The treatment was administered to all subjects under the instructional treatment conditions. Immediately following, the posttest and attitude questionnaire were administered to each subject. Study time data (time spent interacting with Superelevation) for each subject were recorded. No constraints were placed on study time since it was a dependent variable. One hour time limit was allowed for the completion of the posttest which was also administered to subjects in the two control groups. All data gathering instruments were collected and hand scored by the researcher.

# RESULTS

## Cognitive Style

Subjects' scores on the HFT ranged from .75 to 31, the maximum possible score being 32. No significant difference in performance on the HFT was found between males (M = 17.60) and females (M = 15.60) t (44) = .5456, p. < .05.

## Achievement

The means and standard deviations for achievement are presented in Table 2. The analysis for achievement revealed no significant treatment by cognitive style interaction, (F (2,36) = .77, p > .05, Table 3). However, main effects for both treatment (F (2,36) = 52.31, p < .001) and cognitive style (F (1,36) = 5.20, p < .05) were significant.
<table><caption>TABLE 1
Central Themes Evolving From Feedback Data and Corresponding Revision Decisions</caption><thead><tr><th>Source of Feedback<br>FD</th><th>Feedback<br>FI</th><th>Themes Generated From<br>Feedback</th><th>Revision Decisions</th></tr></thead><tbody><tr><td></td><td>X</td><td>Spelling errors.</td><td>Correct spelling.</td></tr><tr><td>x</td><td>x</td><td>Difficulty in relating<br>explanations in text<br>to graphics.</td><td>Label graphics.</td></tr><tr><td></td><td>X</td><td>Frustration when graphics<br>are not related to text.</td><td>Delete or replace<br>graphics.</td></tr><tr><td>x</td><td>x</td><td>Graphics do not<br>communicate.</td><td>Modify graphics.</td></tr><tr><td></td><td>X</td><td>Use arrows, colours and<br>moving objects.</td><td>Use attention getting<br>strategies, colours<br>and animation to<br>enhance<br>comprehension.</td></tr><tr><td></td><td></td><td>Clarity problems.</td><td>Modify to improve<br>clarity of<br>instruction.</td></tr><tr><td></td><td></td><td>Provide summary of main<br>points.</td><td>Summarize main ideas<br>at the end of each<br>unit of instruction.</td></tr><tr><td></td><td></td><td>Provide practice problems<br>and model answers.</td><td>Built in practice<br>and feedback at the<br>end of each block of<br>instruction.</td></tr><tr><td></td><td></td><td>Use of unfamiliar terms.</td><td>Define unfamiliar<br>terms.</td></tr></tbody></table>
<table><caption>TABLE 1 (cont'd.)
Central Themes Evolving From Feedback Data and Corresponding Revision Decisions</caption><thead><tr><th>Source of FD</th><th>Feedback FI</th><th>Themes Generated From Feedback</th><th>Revision Decisions</th></tr></thead><tbody><tr><td>X</td><td>X</td><td>Inaccurate assessment of entry behaviours.</td><td>Reassess prerequisite skills, provide additional instructions to compensate for deficiencies in entry behaviours.</td></tr><tr><td>X</td><td>X</td><td>Misconception and inaccurate interpretation of instruction.</td><td>Modify instruction to remove ambiguities and improve clarity.</td></tr><tr><td>X</td><td></td><td>Difficulty in processing information.</td><td>Modify instruction to reduce information processing demands and facilitate processing of the same.</td></tr><tr><td>X</td><td>X</td><td>Difficulty in relating previously learned concepts to new ones presented.</td><td>Provide review of previously learned concepts and transition to facilitate linkages with new concepts.</td></tr><tr><td>X</td><td>X</td><td>Provide meaningful examples.</td><td>Provide examples that can be associated with previous experiences.</td></tr><tr><td></td><td>X</td><td>Contradictory instruction.</td><td>Modify to remove contradiction and keep consistent.</td></tr></tbody></table>
<table><caption>TABLE 2
Means and Standard Deviations of Posttest Scores by Treatment Group and Cognitive Style</caption><thead><tr><th>Treatment Groups(a)</th><th>Mean(b)</th><th>Standard Deviation</th></tr></thead><tbody><tr><td>FD Control</td><td>7.00</td><td>2.76</td></tr><tr><td>FI Control</td><td>8.00</td><td>5.94</td></tr><tr><td>FD Unrevised</td><td>16.28</td><td>6.94</td></tr><tr><td>FI Unrevised</td><td>22.00</td><td>5.32</td></tr><tr><td>FD Revised</td><td>25.00</td><td>4.93</td></tr><tr><td>FI Revised</td><td>29.00</td><td>3.26</td></tr></tbody><tfoot><tr><td>TOTAL</td><td>17.88</td><td>9.62</td></tr></tfoot></table>

aNote: n = 7 for each group.

bNote: maximum score 38.

<table><caption>TABLE 3
Analysis of Variance of Posttest Scores</caption><thead><tr><th>Source</th><th>df</th><th>s s</th><th>MS</th><th>F</th><th>Eta Square</th></tr></thead><tbody><tr><td>Treatment</td><td>2</td><td>2695.19</td><td>1347.59</td><td>52.31**</td><td>0.70</td></tr><tr><td>FD/FI</td><td>1</td><td>133.93</td><td>133.93</td><td>5.20*</td><td></td></tr><tr><td>Treatment/ FD/FI</td><td>2</td><td>39.86</td><td>19.93</td><td>0.77</td><td></td></tr><tr><td>Error</td><td>36</td><td>927.43</td><td>25.76</td><td></td><td></td></tr></tbody><tfoot><tr><td>TOTAL</td><td>41</td><td>3796.40</td><td></td><td></td><td></td></tr></tfoot></table>

*p < .05, **p < .001.

Follow-up analysis (Tukey studentized range, Honest Significance Difference, Table 4) of the main effects for treatment revealed that the mean posttest scores of subjects interacting with the revised, and the unrevised versions of the material were significantly higher than that of subjects under the no treatment control condition, (p. < .05). The learning gains could therefore be attributed to the instructional treatments. Results also indicated that subjects exposed to the revised version of the material scored significantly higher on the posttest than those exposed to the prototype, (p. < .05). The null hypothesis for achievement was therefore rejected.
<table><caption>TABLE 4
Tukey Studentized Range (HSD) Hypothesis-wise Test on Poshest Scores by Treatment Conditions</caption><thead><tr><th>Treatment Group(a)</th><th>Mean</th></tr></thead><tbody><tr><td>Control</td><td>7.50*</td></tr><tr><td>Unrevised</td><td>19.14*</td></tr><tr><td>Revised</td><td>27.00*</td></tr></tbody></table>

*minimum significant difference = 4.6891, p < .05.

aNOTE: n=14 14 for each group.

Additional follow-up comparisons (Least Significance Difference, Table 5) of the cell means for the two instructional treatment conditions (revised and unrevised version) revealed the FI subjects scored significantly higher than the FD subjects, on the unrevised version (p < .05). However, no significant difference in achievement was found between FD and FI subjects exposed to the revised version of the material. The evidence observed did not support a differential effect of the revised instructional material for FD and FI subjects.

<table><caption>TABLE 5
LSD* Pairwise Comparison of Mean Posttest Scores by Treatment Group and Cognitive Style</caption><thead><tr><th>Group</th><th>Mean</th><th>Group</th><th>Mean</th></tr></thead><tbody><tr><td>FD Unrevised</td><td>16.28</td><td>FD Revised</td><td>25.00*</td></tr><tr><td>FI Unrevised</td><td>22.00</td><td>FI Revised</td><td>29.00*</td></tr><tr><td>FD Unrevised</td><td>16.28</td><td>FI Unrevised</td><td>22.00*</td></tr><tr><td>FD Unrevised</td><td>16.28</td><td>FI Revised</td><td>29.00*</td></tr><tr><td>FI Unrevised</td><td>22.00</td><td>FD Revised</td><td>25.00</td></tr><tr><td>FD Revised</td><td>25.00</td><td>FI Revised</td><td>29.00</td></tr></tbody></table>

*LSD = 5.5023, p < .05.

## Study Time

Table 6 (see next page) shows the means and standard deviations for study time. The analysis of data for study time revealed no significant treatment by cognitive style interaction *F* (1,24)= 0.08 (Table 7, see next page). In addition, no significant main effects for treatment (*F* (1,24) = 0.00), and cognitive style (*F* (1,24)= _________ were found. The mean study time for subjects exposed to the revised and unrevised versions of the material was not significantly different. Therefore, the null hypothesis for study time was retained.
<table><caption>TABLE 6
Means and Standard Deviations of Study Time by Treatment Group and Cognitive Style*</caption><thead><tr><th>Treatment Group (a)</th><th>Mean</th><th>Standard Deviation</th></tr></thead><tbody><tr><td>FD Unrevised</td><td>78.57</td><td>19.73</td></tr><tr><td>FI Unrevised</td><td>77.14</td><td>17.04</td></tr><tr><td>FD Revised</td><td>80.00</td><td>11.18</td></tr><tr><td>FI Revised</td><td>82.14</td><td>17.04</td></tr><tr><td>TOTAL</td><td>79.46</td><td>15.71</td></tr></tbody></table>

*All time data in minutes.

aNOTE: n = 7 for each group.

<table><caption>TABLE 7
Analysis of Variance of Study Time</caption><thead><tr><th>Source</th><th>df</th><th>SS</th><th>MS</th><th>F</th></tr></thead><tbody><tr><td>Treatment</td><td>1</td><td>0.89</td><td>0.89</td><td>0.00</td></tr><tr><td>FD/FI</td><td>1</td><td>72.32</td><td>72.32</td><td>0.26</td></tr><tr><td>Treatment x FD/FI</td><td>1</td><td>22.32</td><td>22.32</td><td>0.08</td></tr><tr><td>Error</td><td>24</td><td>6571.43</td><td>273.81</td><td></td></tr><tr><td>TOTAL</td><td>27</td><td>6666.96</td><td>369.35</td><td></td></tr></tbody></table>

## Attitude

There were little variations in the mean attitude scores between FD and FI students exposed to the revised and unrevised version of Superelevation (Table 8). Data analysis for the attitudevariable disclosed no significant treatment by cognitive style interaction, (F (1,24) = 0.05), (Table 9). Main effects for treatment (3' (1,24) = 0.16) and cognitive style (F (1,24) = 0.49) were not significant. The mean attitude scores of subjects interacting with the revised and unrevised versions of the material were not significantly different. Therefore, the null hypothesis for the attitude variable was retained.

## Students' Feedback Data

The analysis of FD and FI feedback data generated during the one-to-one evaluation sessions revealed some interesting findings. These subjects were able to identify major gaps (practice and feedback) in the events of instruction.
<table><caption>TABLE 8
Means and Standard Deviations of Attitude Scores by Treatment Group and Cognitive Style*</caption><thead><tr><th>Treatment Group (a)</th><th>Mean</th><th>Standard Deviation</th></tr></thead><tbody><tr><td>FD Unrevised</td><td>3.14</td><td><strong>0.43</strong></td></tr><tr><td>FI Unrevised</td><td>3.17</td><td><strong>0.26</strong></td></tr><tr><td>FD Revised</td><td>2.98</td><td><strong>0.66</strong></td></tr><tr><td>FI Revised</td><td>3.09</td><td><strong>0.33</strong></td></tr><tr><td>TOTAL</td><td>3.09</td><td><strong>0.43</strong></td></tr></tbody></table>

(a)NOTE: n = 7 for each group.

<table><caption>TABLE 9
Analysis of Variance of Attitude Scores</caption><thead><tr><th>Source</th><th>df</th><th>SS</th><th>MS</th><th>F</th></tr></thead><tbody><tr><td>Treatment</td><td>1</td><td><strong>0.03</strong></td><td><strong>0.03</strong></td><td><strong>0.16</strong></td></tr><tr><td>FD/FI</td><td>1</td><td>0.09</td><td>0.09</td><td>0.49</td></tr><tr><td>Treatment x FD/FI</td><td>1</td><td>0.01</td><td>0.01</td><td><strong>0.05</strong></td></tr><tr><td>Error</td><td><strong>24</strong></td><td><strong>4.81</strong></td><td><strong>0.20</strong></td><td></td></tr><tr><td>TOTAL</td><td><strong>27</strong></td><td><strong>4.95</strong></td><td><strong>0.34</strong></td><td></td></tr></tbody></table>

Their feedback also revealed inaccurate assessment of prerequisite skills, information processing difficulties, and the need for attention getting strategies. The think aloud process used during the one-to-one evaluation helped to identify misconceptions formed in the processing of information presented.

DISCUSSION

Results indicated that a revised version of the instructional material was more effective than its prototype. This observation supports earlier findings regarding the effectiveness of formative evaluation (for a thorough review, see Nathenson & Henderson, 1980). In this study, FI subjects exposed to the prototype of the CAI obtained a significantly higher mean posttest score than did the FD subjects exposed to the same material. This result is in agreement with findings of other studies (Post, 1984; Smith, 1985), which suggest that the FI subjects benefit more from CAI. However, in this study, no significant differences in achievement were found between FD and FI subjects exposed to
the revised version of the CAI program. It is likely that the revisions made to the material on the basis of FD and FI feedback helped to minimize differential learning gains.

Previous studies attempting to provide for individual differences in learning due to cognitive style have focused on matching instruction to students' cognitive styles (e.g., Greco & McClung, 1979; Elliot, 1976; Grieve & Davis, 1971). Results of this exploratory study suggest that a cognitive style (FD/FI) focused formative evaluation can significantly improve the effectiveness of an instructional material. The effects of revising instructional materials on the basis of feedback generated by FD and FI are likely to minimize differential learning gains due to cognitive style. It appears that a unique instructional material can be produced, that is likely to be effective in terms of achievement (at least not detrimental) for both FD and FI students. However, this finding must be interpreted with caution, until more conclusive evidence is obtained from additional research conducted with larger samples.

No significant difference in study time was found between subjects exposed to the revised and unrevised versions of the material. This result is consistent with findings from earlier research (VanderMeer, 1964; VanderMeer & Montgomery, 1964; VanderMeer & Thorne, 1964, and Sulzen, 1972) indicating that formative evaluation could successfully improve performance on posttest, while keeping study time constant. It is noteworthy in the present study that while study time for the revised and unrevised version of the material was not significantly different, the amount of material was substantially increased in the revised version. The revisions made to the instructional material have most likely facilitated the information processing, thus enabling the subjects to cover more material in approximately the same amount of time.

Analysis of the attitude data failed to reject the null hypothesis of no significant difference between subjects interacting with the revised and the unrevised version of the instructional material. Similar results were obtained by Berthelot (1978). Other researchers (Abedor, 1972; Stolovitch, 1975; Wager, 1980) have found that students tend to have a more positive attitude toward revised instructional materials as compared to unrevised prototype. Romiszowski (1986) notes that the novelty effect can confound findings related to student attitude. In this study, CAI was a novel experience for all subjects. Consequently, the novelty effect may have influenced the results.

## Implications of the Study

The cognitive style construct field-dependent/independent constitutes an important dimension of individual differences among students. These differences relate to the psychological and personality factors, perceptual patterns and social orientation. Cognitive style FD/FI also relates students' preferences for various modes of gaining, storing, processing, using information and solving problems. Differences in information habits between FD and FI students is a source of considerable variations in learning (Witkin, Moore, Goodenough & Cox, 1977). Some students cannot accomplish a task simply
because they lack the information processing demands imposed upon them by
a particular task (Ragan et al., 1979). This construct has some important
educational implications, yet it has so far been overlooked by instructional
designers.

The literature supports that theoretically the cognitive style FD/FI has
promising potential in formative evaluation. The question raised in this study
was whether the construct is an effective criterion for selecting test subjects for
the one-to-one tryout. Qualitative results indicate that the feedback data
generated by FD/FI test subjects can help to identify important deficiencies in
instruction. The subjects were able to identify major gaps in the event of
instruction, inaccurate assessment of prerequisite skills, information process-
ing difficulties and the need for attention getting strategies.

Quantitatively results indicated that the instructional material revised on
the basis of feedback generated by FD and FI test subjects was significantly
more effective than its prototype in terms of achievement. Results also
indicated a significant difference in achievement between FD and FI students
interacting with the prototype version of the material. However, the evidence
observed did not support a differential effect of the revised version of the
material for FD and FI students. It is possible that the effect of revising the
instructional material on the basis of feedback generated by FD and FI sub-
jects has reduced the differential learning gain between FD and FI students
without being detrimental to either group. However, more conclusive evidence
is required from further investigations to support such inference.

Qualitatively and quantitatively it appears the cognitive style construct
FD/FI is an effective criterion for selecting test subjects for the one-to-one
formative evaluation. However, these implications should be considered as
base line, hypothesis generating, and interpreted with caution, since results of
this study are limited by the instructional material, design, test subjects and
evaluator. Additionally, the observed non significant difference on posttest
scores between FD and FI students interacting with the revised version of the
material must be interpreted with caution as further revisions could have
perhaps pushed the scores higher.

Although the cognitive style FD/FI appears to be a viable student selection
criterion for formative evaluation, some problems associated with its implem-
tation by practitioners in real settings, must be considered. Firstly, most
cognitive style assessment instruments are still research based, and their use
is restricted to trained researchers. Secondly, the short turn around time
usually allowed for evaluation and revision may restrict the feasibility of
implementing the cognitive style variable in formative evaluation. Finally, the
difficulty in getting test subjects for participation in formative evaluation
activities may also be a deterrent to the use of FD/FI construct.

Hopefully, further studies will provide stronger support for the use of this
construct, which will by far offset some of the limitations.
SUGGESTIONS FOR FURTHER RESEARCH

The insights gained during this investigation have led to the identification of a number of related problems that warrant further research.

This study used a small population of university students and a computer
assisted instruction program as a treatment. Similar research in this area is
recommended with different populations, and different modes of delivery,
covering various subject matter. Results generated from such research may
provide more generalizable principles for formative evaluation. Additional
research is also recommended to compare the effectiveness of other cognitive
style variables, as student selection criteria.

Research has indicated that cognitive style affects the way teachers teach and students learn. No study investigating the impact of the designer's cognitive style on the effectiveness of instructional material was found. The instructional material used in this study was designed by an extreme field-independent individual. Results indicated a significantly higher achievement score for the field-independent students exposed to that prototypical material. Therefore, a study investigating the impact of designers' cognitive styles on the overall effectiveness of an instructional material is recommended. Since evaluators play a major role in the revision process, similar research should be conducted to investigate the impact of their cognitive styles in the evaluation process and on the effectiveness of the revised product.

REFERENCES

Abedor, A. J. (1972). Second draft technology: Development and field test of a model for formative evaluation of self-instructional multi-media learning systems. *Viewpoints, 48*(4), 9-43.

Ausburn, L. J., & Ausburn, F. B. (1978). Cognitive styles: Some information and implications for instructional design. *Educational Communication and Technology, 26*(4), 337-354.

Baggaley, J. (1986). Formative evaluation of educational television. *Canadian Journal of Educational Communication, 15*(1), 29-43.

Baker, E. (1970). Generalizability of rules for empirical revision. *AV Communication Review, 18*(3), 300-305.

Baker, E. L., & Alkin, M. C. (1984). Formative evaluation of instructional development. In R. K Bass & C. R. Dills (Eds.), *Instructional development: The state of the art, II*. (pp. 319-332). Dubuque, IA: Kendall Hunt Publishing Company.

Berthelot, S. (1978). *Niveau de verbalisation d'un sujet duns le cadre de l'application d'une evaluation L. V. R. d'un document audio-visuel par le mode tutoriel*. Unpublished doctoral dissertation, Universite de Montreal.

Caliste, E. R. (1985). The relationship between the group embedded figures test and selected performance variables. *Performance and Instruction Journal, 24*(9), 26-28.
Cambre, M. A. (1981). Historical overview of formative evaluation of instruc-
tional media product. *Educational Communication and Technology Journal*, 29(1), 3-25.

Crono, L., & Snow, R. E. (1986). Adapting instruction to individual differences. In Wittrock (Ed.), *Third handbook of research on teaching* (3rd edition). (pp. 605-629). New York: Macmillan.

Dick, W. (1977). Formative evaluation. In L. J. Briggs (Ed.), *Instructional design: Principles and applications*. (pp. 313-333). Englewood Cliffs, N.J.: Educational Technology Publications.

Dick. W. (1980). Formative evaluation in instructional development. *Journal of Instructional Development, 3*(3), 3-6.

Dick, W., & Carey, L. (1985). *The systematic design of instruction* (2nd edition). Glenview, IL.: Scott, Foresman and Company.

Ekstrom, R. B., French, J. W., Harman, H. H., & Dermen, D. (1976). *Manual for kit of factor-referenced cognitive test*. Princeton, N.J.: Educational Testing Service.

Elliot, C. A. (1976). *The effects of instructional designs matched to individual differences in cognitive styles on concept learning: A trait-treatment interaction study*. Paper presented at the AERA, San Francisco, California. (ERIC Document Reproduction Service No. ED 122 973).

Gagne, R. M., & Briggs, L. J. (1979). *Principles of instructional design* (2nd edition). New York: Holt, Rinehart and Winston.

Geis, G. L. (1987). Formative evaluation: Developmental testing and expert review. *Performance and Instruction, 3*(4), 1-8.

Geis, G. L., Weston, C. B., & Burt, C. W. (1984). *Instructional development: Developmental testing*. Unpublished manuscript quoted with permission of the authors, McGill University.

Greco, A. A., & McClung, C. (1979). Interaction between attention directing and cognitive style. *Educational Communication and Technology Journal*, 27(2), 97-102.

Grieve, T. D., & Davis, J. K (1971). The relationship of cognitive style to performance in ninth grade geography. *Journal of Educational Psychology, 53*,110-118.

Gropper, G. L. (1967). Does programmed television need active responding? *Audio-Visual Communication Review*, 15(1), 5-22.

Guba, E. G. (1979). Naturalistic inquiry. *Improving Human Performance Quarterly*, 8(4), 268-276.

Kandaswamy, S. (1976). *Learner verification and revision: An experimental comparison of two methods*. Unpublished doctoral dissertation. Indiana University.

Kandaswamy, S., Stolovitch, H. D., & Thiagarajan, S. (1976). Learner verification and revision: An experimental comparison of two methods. *AV Communication Review, 24*(3), 316-328.

Keppel, G. (1982). *Design and analysis: A researcher's handbook* (2nd ed.). Englewood Cliff, N.J.: Prentice-Hall.
Kirby, P. (1979). *Cognitive style, learning style and transfer skill acquisition*. Columbus, OH.: The National Center for Research in Vocational Education, The Ohio State University.

Lincoln, Y. S., & Guba, E. G. (1985). *Naturalistic inquiry*. Beverly Hills, CA.: Sage Publications.

Messick, S. (1985). Style in the interplay of structure and process. In N. Entwistle (Ed.). *New direction in educational psychology 1. learning and teaching*. (pp. 83-98). Philadelphia, PA.: The Falmer Press.

Naditch, S. F. (1976). *Sex differences in field dependence: The role of social influence. In Determinants of Gender Differences in Cognitive Functioning*. Symposium presented at a meeting of the American Psychological Association, Washington, D.C.

Nathenson, M. B., & Henderson, E. S. (1980). *Using student feedback to improve learning materials*. London: Croom Helm.

Post, P. E. (1984). *The effect of students' field independence / field dependence on computer assisted instruction achievement*. Doctoral dissertation, Purdue University.

Ragan, T. J., & others (1979). *Cognitive styles: A review of the literature*. Interim Report, Brooks, Texas: Lory Airforce Base, Colorado. (ERIC Document Reproduction Service No. ED 174 655).

Romiszowski, A. J. (1986). *Developing auto-instructional materials*. New York: Nichols Publishing.

Rosen, M. J. (1968). *An experimental design for comparing the effects of instructional media programming procedures: Subjective vs. objective revision procedures*. Palo Alto, CA.: American Institute for Research.

Sanders, J. R., & Cunningham, D. J. (1973). A structure for formative evaluation in product development. Review of Educational Research, 43(2), 217-235.

Shrock, S. A. (1984). Naturalistic inquiry: An alternative methodology for instructional development research. In R. K. Bass & C. R. Dills (Eds.), *Instructional development: The state of the art, II*. (pp. 333-340). Dubuque, Iowa: Kendal/Hunt.

Silberman, H., Coulson, J., Melaragno, R., & Newmark, G. (1964). Use of *exploratory research and individualized tutoring techniques for the development of programming methods and theory*. Final Report. National Defence Education Act Project 7-14-0000-181. Santa Monica, CA.: Systems Development Corporation.

Smith, C. L. (1985). Relationship of microcomputer based instruction and learning style. *Journal of Educational Technology Systems, 13*(4), 265-270.

Stolovitch, H. D. (1975). *From models to modules: Application of an adapted model for instructional development to the design, production and evaluation as audiovisual training modules*. Unpublished doctoral dissertation, University of Indiana.

Sulzen, R. H. (1972). *The effects of empirical revision and the presentation of specific objectives to learners prior to programmed instruction upon the*
*criterion behavior of military subjects.* Unpublished doctoral dissertation.
University of California, Los Angeles.

VanderMeer, A. W. (1964). *An investigation of the improvement of educational filmstrips and a derivation of principles relating to the effectiveness of these media: Phase II Revision of filmstrip - The Sun and its planets*. Study No. 1, University Park, PA.: Pennsylvania State University.

VanderMeer, A. W., & Montgomery, R. F. (1964). *An investigation of the improvement of educational filmstrips and a derivation of principles relating to the effectiveness of these media*. Study No. III. University Park, PA.: Pennsylvania State University,

VanderMeer, A. W., Morrison, J., & Smith, P. (1965). *An investigation of the improvement of educational motion pictures and a derivation of principles relating to the effectiveness of these media*. University Park, PA.: Pennsylvania State University.

VanderMeer, A. W., & Thorne, H. E. (1964). *An investigation of the improvement of educational filmstrip and a derivation of principles relating to the effectiveness of these media*. Study No. 1, University Park, PA.: Pennsylvania State University.

Wager, J. C. (1980). *An examination of aptitude level and stage in the formative evaluation of instructional material*. Unpublished doctoral dissertation, Florida State University, Tallahassee.

Wallace, S. Q., & Gregory, R. A. (1985). *Cognitive style: The unaccommodated variables in training decisions*. *Performance and Instruction Journal*, 24(4), 22-33.

Weston, C. B. (1987). The importance of involving experts and learners in formative evaluation. *Canadian Journal of Educational Communication*. 16(1), 45-58.

Weston, C. B. (1986). Formative evaluation of instructional materials: An overview of approaches. *Canadian Journal of Educational Communication*. 15(1), 5-17.

Witkin, H. A., & Goodenough, D. R. (1981). *Cognitive styles: Essence and origins, field dependence and field independence*. New York: International University Press.

Witkin, H. A., Lewis, H. B., Hertzman, M., Machover, K., Meissner, P. B., Wapner, S. (1954). Personality through perception. New York: Harper.

Witkin, H. A., & Moore, C. A. (1974). *Cognitive style and the teaching and learning process*. Princeton, N.J.: Educational Testing Service.

Witkin, H. A., Moore, C. A., Goodenough, D. R., & Cox, P. W. (1977). Field-dependent and field-independent cognitive styles and their educational implications. *Review of Educational Research*, 47(1), 1-64.

## AUTHOR

Chris A. Chinien is an Assistant Professor in the Department of Mathematics and Natural Sciences, Industrial and Vocational Education, University of Manitoba, Winnipeg, Manitoba R3T 2N2.
ADVANCED EDUCATIONAL TECHNOLOGY
RESEARCH WORKSHOPS AND SUMMER SCHOOLS

The Science Committee of NATO sponsors the organization of Advanced Research Workshops and Advanced Study institutes (Summer Schools) on topics in the field of educational technology. The aim of this special programme is to stimulate international collaboration among established and younger scientists in this field of R&D.

Areas of interest (among others) include task analysis, simulation, modelling and diagnosis, tutorial strategies, intelligent tutoring and other interactive technologies, microworlds and interface design, and evaluative methodologies.

Workshops typically involve 20 to 40 experts in theirfield in a 2 to 5 day meeting with the aim of reviewing the state-of-the-art in the field and establishing collaborative links for future research. Study Institutes are 2-week summer schools where 12 to 15 leading scientists impart knowledge and experience to a group of 60 to 80 younger scientists with the aim of giving them a solid footing in the field. Both types of activity favour professional contact among scientists in the spirit of international co-operation in furthering promising areas of educa-tional technology.

NATO does not organize these activities itself; rather, it sponsors them through funding. Scientists interested in organizing an Advanced Research Workshop or an Advanced Study Institute in their particular speciality should request an application form and information packet from the

Scientific Affairs Division
(Advanced Educational Technology)
NATO
B-1110 Brussels, Belgium

Applications for more modest funding to enable international travel for collaborative research among scientists from different countries, and for research, study, and lecture visits abroad are also available.
# Protecting Expression in Teleconferencing: Pseudonym-Based Peer Review Journals¹

David S. Stodolsky

**Abstract:** The social environments of educational systems are less than ideal because power differentials exist that can suppress the free exchange of ideas. One solution is to strengthen personal integrity with an anonymity shield. Many text-based conferencing systems permit anonymous contributions, but this often leads to irresponsible behavior. If people are limited to one and only one pseudonym, however, responsible behavior can be expected. This reputation preserving anonymity overcomes the problems with traditional systems. A reputation is developed through peer evaluation which is based on routinely elicited judgments. Evaluative judgments of a message by one person can be available to all other potential receivers of that message immediately. Evaluations can then be used to automatically select messages worth reading. This approach deals effectively with the problems of both information overload and irresponsible behavior while providing the highest possible protection of expression,

‘He told the truth and politicians and civil servants hated him for it.’
(Wright, 1987, p. 356)

The notion of an ideal speech situation has alonghistory. Both Plato in the *Phaedrus* and Habermas argue that if dialogue is-to lead to truth, an ideal speech situation is needed, which in turn presupposes an ideal social environment (Bernstein, 1978, p. 262; McCarthy, 1978). The social environments of educational systems are less than ideal because power differentials exist between students and teachers, and among workers within the educational establishment. One problem is that people may be punished if they speak in opposition to power holders. More often, persons censor themselves because they fear that speaking out will damage their career possibilities.

## Anonymity: Potential and Problems

One solution is to strengthen personal integrity with an anonymity shield. Karabenick (1987) has shown that this can have direct educational benefits, since people are more likely to seek help in an academic situation if they can do so anonymously. He argues, “Seeking help [which must be preceded by admission of inadequacy] when needed is an integral part of the learning process” (p. 69). Further, “in cases where anonymity (e.g., using pen names) is
permitted the reason [for increased help-seeking] is obviously the reduction in social stigma, or embarrassment.. especially when faculty have access to the conference" (p. 72).

More generally, it can be argued, based on operant theories of learning, that feedback from the environment is crucial for learning. High rates of response -that create learning opportunities - can be sustained when correct responses are rewarded and erroneous responses are not punished. This requires an environment in which the individual is protected from negative consequences resulting from errors made during the learning process. That is, the protected learning environment acts as a discriminative stimulus for production of responses that may yield rewards. These rewards can then shape the behavior in a desired direction.

Many text-based² conferencing systems permit anonymous contributions. This often leads to irresponsible behavior (Wilkerson, 1987). "Anonymity breeds irresponsibility" (Spragge, 1987, p. 98), was the conclusion of one system administrator. Abuse of anonymity has also been a long standing problem with scientific journals (Garfield, 1988). The current peer review system has been described as one in which power relations have become dominant (Michel, 1982). Inappropriately used, anonymity can generate power differentials as well as irresponsible behavior (Garfield: 1988). Another important limitation of protection based upon anonymity is the inability to reward persons individually for specific acts. This can be seen as a block to effective learning and to motivation. It is a recognized problem with scientific peer review and surely plays a role in academic learning when anonymity is employed.

## Pseudonymity

What is needed is reputation preserving anonymity, or pseudonymity, which overcomes many of the problems with traditional journals while ensuring individual integrity. If people are limited to one and only one pseudonym, responsible behavior can be expected. The person must protect their pseudonym from developing a bad reputation or others will not select messages or use judgments issued under that pseudonym. On the other hand, messages under a pseudonym with a good reputation will be read more often and judgments as to what is worth reading under that pseudonym will have a strong influence on dissemination of messages.

An often raised objection to this approach is that even if an author can not be traced through the system because of formal protections, the writing style and similar factors can be used to identify an author. In the system described here, judgments similar to votes play an important role, and these can provide complete protection even in relatively small groups. The system automatically collects judgments on messages after they are read and transmits them in a standard format. Completely protected judgments can then be used by others to automatically select messages worth reading. We assume that it is not feasible to read all messages.
## *Public versus Protected Communications*

An analogy between the proposed system and anonymous voting in parliamentary meetings can be extended, although it is not completely correct. In such meetings, communication occurs on two levels, public and anonymous. Substantive motions, procedural motions, and discussions occur on the public level: There is no attempt to hide source identity. Voting, however, is typically anonymous. Similarly, with the proposed system substantive messages and judgments directed against them can be considered as operating on two different levels. Especially in a smaller group, it may be difficult to effectively hide the source of a substantive message because of stylistic features. Judgments in a standardized form, however, have no such features and therefore offer significantly greater protection for author identity. Like anonymous votes, they can play a crucial role in decision making.

However, unlike anonymous votes, pseudonymous judgments can play an important role in reputation development. For instance, a person could develop a good reputation purely on the basis of judgmental responses: those responses that are highly protected. This would ensure that any substantive messages later contributed would immediately come to the attention of other group members, thereby maximizing the message's influence potential. With the pseudonym system there is also much greater flexibility in the treatment of substantive messages. One extreme could be to have them signed by their authors using their public names. Another would be to have substantive messages sent first to human or machine editors, who would remove stylistic features that could reveal authorship, translate them to another language or even rewrite them for clarity. Pseudonymous communication, then, offers many more possibilities for finding an effective balance between protection and accountability for authors. It also offers significantly more information to readers – actually their programmed message sorting systems, – who must decide which messages to read.

## *Basic Organization of the System*

In its simplest form, the system includes untraceable mail and digital signature capabilities. By untraceable mail we mean that a message can not be traced back to its sender by physical means or by analysis of the information transmitted with the message. An ideal broadcast system would have such a physical characteristic. In practical systems, a ring topology network can transmit untraceable mail at 25% efficiency as compared to normal mail. The author's identity can be unconditionally secure, that is, resistant to infinite computational power. Then, finding the source of the message requires cooperation of all parties except for the one being traced (Chaum, 1985). A less secure, but readily available system is the public-access telephone network. Many data networks can also provide adequate security for short connection times.

The digital pseudonym³ is required to be untraceable and unforgeable. A one-to-one mapping between persons and pseudonyms is required. This can be
implemented with public-key cryptography* using an independent registrar or
is-a-person organization. This organization is the only one that can engage in
pseudonym creation. An interaction with a potential user permits the authori-
zation for creation of a pseudonym to be issued to the user. The user then, at
a later time, returns by untraceable mail the actual pseudonym to be used. This
pseudonym serves as that individual's public key in a digital signature system.
Messages decrypted with that key could only have been sent by that individual.
Persons must be physically identified to obtain an authorization, thus each
person can acquire one and only one unforgeable pseudonym.

One objective of the pseudonym system is to focus a reader's attention as completely as possible on the content of a message. The pseudonym mechanism makes it very difficult to determine an author's identity, thereby discouraging giving any attention to this aspect of a message. In fact, the pseudonyms discussed here would not be in a form easily read or remembered by a reader, normally they would not be seen at all. Readers would train their computer systems as to which pseudonyms merited attention merely by giving evaluative responses to messages. Both the reputation of the author of a message and the reputations of previous readers of that message (assuming they offered judgmental responses to it) would be used to automatically rank the message in priority. This is meant to duplicate, in a more rigorous manner, the way we use recommendations of friends and colleagues to decide what is worth reading.

A common misconception about using pseudonyms is that the benefits
would be short-lived since once a pseudonym's reputation had been established
it would function just as a real name, prejudicing reader acceptance of
massages and reactions to them. It is true that only an anonymous message
system guarantees the evaluation of message content without any influence of
previous messages from that author. However, such a system also offers the
reader no basis for selecting messages to be read. In a properly functioning
pseudonym-based system, reputation information gives an unbiased estimate
of whether anew message is worth reading. This estimate is based partly upon
the content of previous messages from the same author. The author's institu-
tional position, the prestige of the author's institution, and other biasing
factors which typically influence readers are screened out. If these factors were
true indicators of message quality, then they would be correlated with reputa-
tions developed within the message system. In summary, a system based upon
anonymous messages treats each message equally, while a system based upon
pseudonymous messages treats each author equally.

Differential Competence and Reputation Management

The possibility of differing levels of competence in different subject areas
can be accounted for by allowing persons to have a different pseudonym in each
separate conference or journal. By use of a *credential mechanism* (Chaum,
1985), expertise developed in one conference can be transferred to another
without any loss of security- that is, without release of information that would
permit the association of different pseudonyms. If one of the names used was
the name by which the person was known to an educational institution, this
mechanism could be used to show that educational objectives had been
satisfied.

Similarly, the mechanism could permit reputational credentials developed
outside the message system to be moved, untraceable, into the system. Thus,
for example, a given message could be shown to be from someone who had
received a certain educational degree, to have achieved a certain academic
rank, or to be employed by a certain institution. The more detailed such
information was, however, the more constrained the set of possible authors and
the more limited the protection for those authors.

An essential feature of the credential mechanism is the ability to move
reputational information from one name to another untraceable. Given this
ability, even positive identification of the author of a given message would not
compromise the overall functioning of the system, since all pseudonyms could
be changed without a loss of reputational information. For instance, such a
procedure could operate very similarly to the double-blind peer review used by
many scientific journals. While a message was being evaluated, authorship
could remain hidden. Later, the author could claim the message and even the
referees could identify themselves publicly. If this was followed by an immediate
change of pseudonyms, then the next message from that same author could
be evaluated in an unbiased manner, since the association between the
message with the publicly identified author and the new message would be
untraceable. The credit for producing that publicly identified message would
be available to the author, however, thus ensuring that the new message was
widely distributed.

In an educational setting the exchange of pseudonyms might be necessary
if teachers or staff members were to retain their protection. Otherwise, the
continuing presence of their pseudonyms in a conference, while student names
were constantly changing could give away their identity. Also, there are
instances in which a person might wish to share a program or text file that
would reveal the author's identity. That is, the work in question might have a
known author or be found in a storage location that belonged to a single person.
However, once pseudonymous communication became the dominant mode of
interaction, expertise established under a pseudonym would be connected to
specific works also available under that name, thus the previously mentioned
situation would not require a change of pseudonyms.

Often it is helpful to have publicly known local experts available in order
to get quick answers to specialized questions. In a conferencing system
environment the element of geographical locality is eliminated. It is replaced
by content or subject matter locality permitting consultation with the leading
expert on the specific topic. Identification of such experts would be, at least in
part, automatic as a result of the system for reputation maintenance discussed
here.
## Review Messages

When an individual reads a message and makes a judgment of it, that judgment can be signed by the reader and broadcast to other potential readers. These judgments of a message can be used by those who have not yet read a message to rank it in priority. The standard format of the review permits the user to allow a program to compute the probable importance of a given message and automatically schedule messages for attention. A Bayesian estimation model can be used to combine the information about the author with the judgments of the previous readers. The user's own judgment upon reading a message can then be used as a basis for revising the probabilities in the model parameters that describe each person's judgmental competence and competence as an author.

## Evaluative Dimensions

Another elaboration of the basic system permits judgments to be given on multiple dimensions. These judgments establish different types of formal relations between messages. For instance, a scientific paper after having been judged *relevant* is most likely to be accepted for publication if it meets three criteria (Garfield, 1988). First, it should be *sound*. The author must have employed reliable data, drawn valid conclusions, and committed no flaws in logic. It should also be *original*. Finally it should be *significant*, meaning that it should contain some new perspective or observation of potential importance. Judgments on these dimensions could be combined to decide whether a message is worth reading.

While evaluative messages could give quantitative responses on various dimensions, this is not a requirement for system operation. Both the dimensions or categories for evaluation and the scaling of such evaluations would follow from agreements between the users of the system. Such agreements would permit more or less effective sorting of messages by computer software. With very high message volume, multi-dimensional and carefully scaled responses would be beneficial. However, if powerful natural language understanding software was available, then unstructured responses could be entirely adequate.

The need for evaluative information becomes much greater with the complex and opaque multi-media or hypertext documents now being developed (Stodolsky, 1987). With parts of such documents or with very short messages and more conversational interaction, often associated with voice messages, the types of judgments could be quite different (Stodolsky, 1984). The degree of impact on the priority relations among messages waiting to be read would be the crucial measure of quality for review messages.

If a reader finds a message to be lacking on a certain dimension, a substantive message may be offered to supplement the judgment given. A structured form of argumentation can then take place. Some authors would be attracted into the controversy and strive to gain credibility by issuing judgments referencing a given message. Others would prefer to wait until the
situation had stabilized, as calculated by Bayesian estimation, before reading any of the messages (Stodolsky, 1984). This latter strategy might be called the text or reference book approach to conferencing. The first might be called the meeting approach.

## Summary of the Approach

The security of the system reduces the effect of power relations on the interchange of information (Stodolsky, 1985). The judgment mechanism focuses attention on reasoning in the dialogue process. The formal relations among messages and quest for credibility attracts competent criticism. The overall system integrates the reliability of the scientific journal with the rapid response of informal dialogue, thereby creating a powerful educational technology.

## IMPLEMENTATION

A fully developed system of the type outlined here would be quite adequate to support a multiple journal publication program of a major scientific society. In fact, it would be an improvement over the best current practice. Significant educational objectives can be supported with much less elaborate procedures particularly if the demands for security, flexibility in registering of new participants, and transferability of credits are relaxed. This is appropriate in an educational setting where the intensity of assets is significantly reduced as compared to the professional environment. That is, the size of payoffs and therefore, motivations toward corruption are reduced in educational environments.

A core mechanism of the proposed teleconferencing system is pseudonym-based communication. While this can be implemented with the highly secure cryptographic techniques mentioned, it can also be accomplished with much simpler procedures in educational settings. For example, if a third party can be found who is trusted by all participants, then that person can simply be assigned the responsibility of seeing that each person receives one and only one pseudonym, that is, act as registrar. If that person also plays the role of computer system administrator, then standard controls and accounting procedures available with current conferencing systems can also be used. What is crucial is that the users of the system feel they are secure, so that educational benefits of pseudonymous communication can be attained.

When this level of security is inadequate, another procedure is available assuming participants can meet physically at the beginning of an educational program. Then the number of persons present can be determined and that exact number of paper slips with pseudonyms and computer passwords written on them can be placed in a hat. The hat is then passed with instructions that each person select one and only one slip. If this procedure succeeds that is, if each person gets a valid pseudonym and password, then protected
communication can occur on any conferencing system. The validity of the slips could be checked, in the first instance, by a signature or other unforgeable mark on the slips. A further validity check and security enhancement would occur when participants logged-in to the computer for the first time and changed their passwords. This would best be accomplished before participants dispersed. A second "log-in" to ensure that the new passwords were functioning correctly would avoid possible problems that commonly occur when inexperienced persons begin using computer systems.

While ad-hoc procedures such as this can give an adequate level of security, they have their limitations. Since pseudonyms and their computer accounts will most likely be terminated at the conclusion of the educational program, and since credits earned under these names cannot be transferred or preserved (unless protection is compromised), it is likely that concern with maintaining good reputation of pseudonyms will drop toward the end of the program. Under such conditions an increase in irresponsible behavior and in the number of abusive messages can be expected.

## *System Specification and Software Development*

A level of implementation complexity beyond the ad-hoc arrangements mentioned above assumes enhancements to software. The process can be divided into system specification and software development. Two packages of software and associated institutional arrangements would be included in specification of a complete system. First, a powerful communications software package, that includes a display interface manager, communications handler, reputation database, statistical estimation routines, mail sorting software, and cryptography subsystem would be specified. Second, organization of the registrar function and associated cryptographic security system would be specified. The independence and security of this function is critical to the success of the pseudonym system.

From a functional point of view, software development can proceed in three lines. First, a mechanism for selecting messages based upon both content and source can be developed. The SMART current awareness information system (Fox, could be used for content-based selection. For reputation-based selection, a reputation database, statistical estimation routines, and mail sorting software would be integrated. Selection using both content and source would require integration of the two selection mechanisms with a display interface manager and communications handler.

Second, the collection of evaluative responses is a relatively straight forward software development and human interface problem. Unless the software integration is very smooth and transparent, however, is unlikely that readers will use the facility. The incentives for contributing evaluations of messages is likely to develop when substantial volumes of material with conflicting points of view are exchanged.

A third line of software development includes the security mechanisms for constructing pseudonyms. As the incentives for contributing messages in-
creased, the need for protection and authentication would also increase. Substantial development work is now in progress to replace the current password-based authentication on computer systems with public-key based cryptographic authentication procedures. These authentication procedures use the same principles as the pseudonym mechanism needed for the proposed system. At least one public-key cryptographic system is currently available as a commercial product.

Implementation Strategy

These procedures assume that physical tracing of communication is not practical. On most computer systems it is quite easy to determine which participant is using a certain line into the computer. In the case of dial-up lines, however, there are significant legal as well as technical barriers against determining actual identities (Stoll, 1989). Dependable security, however, requires specially structured communication subsystems (Chaum, 1985).

The focus of efforts in implementation can be guided by the actual needs in a given educational environment. When a high volume of messages on different subjects is expected, attention might best be focused upon an adaptive sorting mechanism based upon topic categories. In the case of a high volume of messages of variable quality, the source evaluation mechanism would be very important. This peer reviewing of messages might also be seen as desirable because it is an effective means of providing feedback to learners. It can also serve as a tool to evaluate relative competence of participants assuming an adequate level of competence in the group as a whole. When power relations threaten the free exchange of statements and judgments giving adequate attention to the security questions could be crucial. Effective sorting of messages is limited by the quality of judgments directed to them. The quality of judgments is in turn dependent upon the degree to which the speech environment approaches the ideal of non-dominative communication.

NOTES

<footnote n="1">¹Earlier versions of this paper were presented August 11, 1988, at the Fourteenth World Conference on Distance Education, Oslo, Norway and electronically prepublished May 9, 1989 in *Communication Research and Theory Network*, No. 775. University Park, PA: The Pennsylvania State University, Department of Speech Communication.</footnote>

<footnote n="2">²*Text-based* or asynchronous (store-and-forward) conferencing systems are distinguished from real-time audio conferencing which requires simultaneous presence.</footnote>

<footnote n="3">³The pseudonym is a binary number of about two hundred digits. *cryptography* uses two different keys, one for encoding and one for decoding. The public-key can be widely distributed without risk of revealing the private-key that is used to decode messages and sign documents.</footnote>
This system makes key distribution practical when there are large numbers of users.

## REFERENCES

Bernstein, R.J. (1978). *The reconstruction of social and political theory*. Philadelphia, PA: University of Pennsylvania Press.

Chaum, D. (1985). Security without identification: Transaction systems to make big brother obsolete. *Communications of the ACM*, **28**, 1030-1044.

Fox, E. (1981). Implementing SMART for minicomputers via relational processing with abstract data types. *Sigsmall Newsletter*, 7(2), 119-129.

Garfield, E. (1988). Refereeing and peer review: Part one. In E. Garfield, *Essays of an information scientist: Towards scientography* (Vol. 9, pp. 230-238). Philadelphia, PA: ISI Press,

Karabenick, S.A. (1987). Computer conferencing: Its impact on academic help-seeking. *The second Guelph Symposium on computer conferencing*, (pp. 69-76). Ontario, Canada: Guelph University, Department of Rural Extension Studies.

McCarthy, T. (1978). *The critical theory of Jurgen Habermas*. London: Hutchinson.

Michel, F.C. (1982). Solving the problem of refereeing. *Physics Today*, 35(12), 9:82.

Spragge, J.G. (1987). If computer users are a community, the conference must be the town meeting. *The second Guelph symposium on computer conferencing*, (pp. 91-104). Ontario, Canada: Guelph University, Department of Rural Extension Studies.

Stodolsky, D. (1984, December). *Self-management of criticism in dialog: Dynamic regulation through automatic mediation*. Paper presented at the symposium Communicating and Contracts between people in the Computerized Society, Gothenburg University, Sweden.

Stodolsky, D. (1985, June). Telematic journals and organizational control: Integrity, authority, and self-regulation. *Abstracts of the 7th colloquium of the European Group for Organization Studies*, Saltsjoebaden, Sweden.

Stodolsky, D. (1987, April). Telematic journals [Abstract]. *Proceedings of the First STIMDI Conference on Man-computer Interaction*. Stockholm, Sweden: STIMDI (Sveriges tvaervetenskapliga intressefoerening foer maenniska-datorinteraktion).

Stoll, C. (1989). Stalkingthewilyhacker. *Communicationsofthe ACM*, **31**,484-497.

Wilkerson, I. (1987, April 18). Ethnic jokes in campus computer prompt debate. *The New York Times*.

Wright, P., with Greengrass, P. (1987). *Spycatcher*. Richmond, Victoria: William Heinemann Australia.
AUTHOR

David S. Stodolsky is a Guest Researcher in the Department of Psychology
Copenhagen University, Njalsgade 88, DK-2300 Copenhagen S, Denmark:

ACKNOWLEDGEMENT: The author gratefully acknowledges the assistance of the fol-
lowing persons for their comments on drafts of this article: Bill Hunter Kathleen
M. Carley, Terry Winograd, Gerald Phillips, Hedwig Miller, George Altomare, and
two anonymous reviewers.
INTERNATIONAL

LASERDISC

CONFERENCE 1990

LASER AND OPTICAL DISCS
IN INDUSTRY, HEALTH
AND EDUCATION

June 10, 11 & 12, 1990
The University of Calgary
Calgary, Alberta

<figure><img src="image_9.png" /></figure>

ACCESS
NETWORK

<figure><img src="image_10.png" /></figure>

THE
UNIVERSITY
OF CALGARY

<figure><img src="image_11.png" /></figure>

Alberta LaserDisc Committee

for further information: Margaret Anne Stroh,
Conference Office, The University of Calgary
2500 University Drive N.W.,
Calgary, Alberta T2N 1N4
Phone: (403) 220-6229 Telex: 03-821545
Fax: (403) 289-7287
Computer-Mediated Communication
and School Administrators: A Case
Study of a University In-Service Course

Janice Ahola-Sidaway
Margaret MacLean
Jack Treuhaft

**Abstract:** As computer-mediated communication (CMC) becomes more widely known in educational circles, the socialization of new users and job-related contextual factors will become increasingly important issues. This article reports on a qualitative, exploratory case study that focused on these two issues. The participants were a group of twelve experienced school administrators who were introduced to CMC as part of a non-mandatory university in-service leadership course. Data were gathered by means of online participant observation, questionnaires, and focus group interviews.

The socialization process of this 'new user" group was found to be very complex. Access to equipment and a local resource person, knowledge of the system, and the incentive/motivation structure emerged as the three most important factors influencing this socialization process, CMC findings that were related to the participants' professional role as school principals were of two types, those related to time implications and those related to actual and potential CMC relevance for the principalship.

Computer-mediated communication (CMC) — that is, typed computer exchanges between individuals (electronic mail) and among individuals (electronic conferencing) -is a relatively new phenomenon in the field of education and, as such, we know very little about its innovative educational applications. There is some limited empirical evidence that specifies the relative merits of CMC as a teaching-learning tool for graduate and undergraduate students (e.g., Mason, 1988; McCreary &Van Duren, 1987). There is also some evidence of its potential for distance education (e.g., Kaye, 1987). Many questions regarding its educational applications, however, still remain to be investigated.

One of the most striking elements of much of the CMC research published to date is the type of research samples that have been examined. Typically, these samples have consisted of groups of graduate or undergraduate students who have familiarity with computer technology and/or are interested in computer-related issues before getting involved in CMC activities (e.g., Harsim, 1987; Hiltz, 1986; McCreary & Van Duren, 1987; Siegel, Dubrovsky,
Kiesler, & McGuire, 1986). As a result of this focus on computer literate samples, the research has tended to gloss over the complexity of the issues associated with moving from non-user to more experienced CMC user. A second limitation of the research is the tendency to ignore contextual factors which might influence how CMC users, other than university students enrolled in credit courses, participate in CMC activities.

As CMC becomes more widely considered as a potential pedagogical tool for a broader segment of the educational community -whether as an adjunct to regular face-to-face meetings or as part of an integrated or stand-alone educational delivery system for university-based courses, distance education, or in-service education, the socialization of new users will become an increasingly important issue. As CMC expands into other educational settings - especially beyond the confines of credit-based courses- the issue of contextual factors influencing CMC use will also become an increasingly important topic.

## FOCUS OF THE STUDY

The present study was intended to identify some of the key issues associated with CMC socialization for a group of new users who were enrolled in a university in-service course for experienced, practising school administrators. More specifically, we were interested in monitoring the socialization process of moving from non-user to more experienced CMC user, and exploring the implications of CMC for the professional role of the participants.

## CONTEXT OF THE STUDY

The Faculty of Education at the University of Ottawa offers an Ontario Ministry of Education course entitled the Principals' Refresher Course (PRC). This is a non-mandatory, in-service course for experienced, practising school administrators. PRC '88-'89 ran from early November, 1988 until mid-May, 1989. Participants met once a month (on average) on campus to discuss various leadership themes presented by invited professors or practitioners. Each of these monthly meetings lasted approximately seven hours.

Twelve school administrators, representing seven school boards in Eastern Ontario, were enrolled in PRC '88-'89. All of the participants were male, with an average age of 45-50. Both as a group and individually, they had contributed many years - anywhere from 6 to 31 years - to administration of education. The average number of years they had served as a principal or vice-principal was 17, with approximately 15 of those years as principal. At the time of the study, eight of the participants were elementary school principals, three were secondary (two high school and one intermediate school) principals, and one was a high school vice-principal.

The CMC component of the course spanned the entire November-May
timeframe. Preliminary training and information sessions were held during the three on-campus meetings in November, December, and mid-January. For approximately 16 weeks, from mid-January to mid-May, participants became users of CMC, and a part of each PRC meeting during that timeframe was devoted to CMC discussion/data gathering activities in the form of descriptive questionnaires and focus group interviews.

Because participants were not told about the CMC study prior to the opening session of PRC and because they would be expected to use their own computer hardware and software, the CMC component of PRC '88-'89 was presented to the group as a non-mandatory, yet important part of the course. Because it was not mandatory, not all were actively involved in online activities — although all participated in large group training/information sessions and focus group interviews and all completed the questionnaires related to CMC.

The CMC system that was used in this study was “Chimo” -an electronic network specifically designed for use by educators (Wilton, 1988). In January 1989, there were approximately 300 Chimo users; by May 1989, approximately 500 educators were online. The Chimo system permits typed computer exchanges between individual users (mail messages) and conferences in which many individuals can share ideas on topics of common interest. Conferences are designated as “open” (anyone can join) or “closed” (restricted to those who have requested and are granted access).

During the sixteen-week hands-on CMC experience, participants were encouraged to learn and use mail messaging, closed conferencing, and open conferencing. For example, they were encouraged to discuss the relative merits of CMC vis-a-vis other more traditional means of communication, to comment on specific class lectures and readings, to make suggestions and receive information concerning upcoming class activities, and to exchange information on topics of common concern with other educators in Chimo.

## METHODOLOGY

The study used a qualitative, naturalistic research approach. In this section of the article we discuss our rationale for adopting this research stance. We also describe the basic design of the study and the specific techniques used to conduct the study.

### *Rationale for a Qualitative Research Approach*

Four compelling reasons underpinned our decision to use a qualitative, naturalistic research design. First, as noted earlier in this article, CMC in education is a relatively new phenomenon; and research on CMC applications in education is understandably in its infancy. To our knowledge, no published research has investigated CMC applications involving practising school administrators enrolled in a university in-service leadership course. It seemed
appropriate, then, to use an "exploratory," descriptive research design — a design where the emphasis is on discovery rather than on testing and refinement (see, for example, Everhart, 1988).

Second, the types of questions that we used to guide our inquiry also pointed to using a naturalistic approach. For example, we were interested in exploring the Chimo experience from the perspective of the PRC participants, what is typically referred to as focusing on "multiple constructed realities" (Lincoln & Guba, 1985). We were also interested in unraveling some of the mysteries associated with the group's transformation from complete non-users of CMC to more experienced users, that is, on the process of their socialization.

Third, we wanted to capitalize on the fact that we, too, were participants in the study. By doing so, we could turn such issues as subjectivity, observer bias, and reactivity into methodological strengths rather than limitations (see, for example, Lincoln & Guba, 1985).

Fourth, we were also sensitive to contextual factors. The group that we were studying was a special sub-group of educators — school administrators — and we believed that it was important to explore their Chimo experience in relation to this fact.

## Basic Design of the Study

The basic design of the study is best conceptualized using three overlapping circles, as depicted in Figure 1 (see following page).

The three large circles, in their entirety, represent the "world" of Chimo, the "worlds" of the three investigators, and the "worlds" of the 12 participants enrolled in PRC '88-'89. The unshaded areas of these three worlds (labelled A, B, and C) represent domains that were outside the scope of the study. Domain A is that part of Chimo that neither the investigators nor the principals entered (e.g., conferences that were never joined). Domain B includes work responsibilities of the investigators that were not part of this research (e.g., teaching, other research). Domain C represents the collective professional lives of the 12 principals that remained unknown to the investigators.

The overlapping shaded areas across the three worlds (Domains D thru H) depict the research contexts that comprise the study. Domains D thru G represent specific Chimo-world contexts. D represents the Chimo experience that was collectively experienced and directly observable by both the principals and the investigators, that is, the closed PRC conferences, several Chimo open conferences, and mail messages involving at least one PRC participant and one investigator. Domain E represents that part of the Chimo world that was explored by the investigators as participants and observers in the larger Chimo world. These included open conferences and the investigators' mail messages sent between each other or that were sent to or received from outside the PRC group. Domain F represents that part of Chimo that was explored by the individual principals but was not directly observable by the investigators, that is mail messages (which are private exchanges between sender and
<figure><img src="image_12.png" /><figcaption>Figure 1.
Design of the Study</figcaption></figure>

receiver) between two of the PRC participants or between a PRC participant and another Chimo user. Domain G represents a closed Chimo conference created specifically to allow the three investigators a private online meeting place, Domain H represents the part of the administrators' worlds that were outside of, yet linked to, Chimo that the investigators explored. Data from this domain were limited to questionnaires and focus group interviews.

## Specific Methodological Techniques

The research team, consisting of two principal investigators and one research assistant, fulfilled multiple roles within the study. Although we presented ourselves as a research team and all three of us fulfilled a research role, each investigator also served a specific “public” role. One investigator served primarily a co-ordinating role; the other principal investigator primarily fulfilled the public role of “researcher;” and the research assistant served as the local computer consultant, offering assistance on-campus, by telephone,
and online. Our respective public personas were recognizable both online and during face-to-face course sessions. All three of us also participated in and observed, to varying degrees, online activities.

The stages involved in conducting the study included gaining commitment from the administrators, maintaining rapport, data collection, data analysis, and final reporting.

As noted, "gaining commitment," rather than the more typical term of "gaining entry" has been used here to describe the first field work stage. This distinction is important. In many important respects, we did not "enter" the social system that we were investigating as much as we "created" the opportunity for the principals to do so. Their commitment to participate was therefore essential. Keeping in mind that the methodological literature frequently refers to the importance of "reciprocity" in building and maintaining respondent commitment (e.g., Adler & Adler, 1987), we attempted to offer tangible benefits. For example, we appealed to their sense of professionalism by pointing out the importance of the research for the educational community and by telling them about other types of CMC initiatives (see also subsequent discussion "Incentive/Motivation Structure"). All 12 administrators agreed to participate in the study, although two of the twelve were unable to participate in the online activities.

Maintaining trust and rapport with the principals was relatively easy to accomplish, for at least two reasons. First, we — both the principals and ourselves-were all new to the Chimo system, so we could honestly and openly state that we had a common bond with them. This circumstance also made it relatively easy, we felt, for the administrators to admit to problems and to offer candid data about their experience. Second, because the Chimo experience itself rested on communication, participating in the study, we believe, served to enhance rapport within the group.

Data were collected prior to, throughout, and at the conclusion of the study by means of online participant observation, descriptive questionnaires, and audio-taped focus group interviews. As is typical with naturalistic studies, data analysis and interpretation were also ongoing activities throughout the investigation, with the investigators meeting regularly both face-to-face and online via a special Chimo conference and electronic mail. These activities helped us to keep track of the unfolding experience and to shape the investigation according to emerging questions and needs (see also discussion of Figure 1).

## FINDINGS AND DISCUSSION

We were interested in monitoring the socialization process of moving from non-user to more experienced CMC user and identifying the contextual factors influencing participants' experiences with and perceptions of CMC. This section of the article discusses our findings related to these two issues.
## Socialization Process

Our group was unique in that it consisted of school principals who were not experienced computer users. An initial survey revealed that while all but one had computers in their schools for administrative and instructional purposes, they were not necessarily directly involved themselves in using the machines. The survey results also indicated that four of the 12 participants had a computer at home, using it mainly and occasionally as a word processor. Six of the administrators rated their keyboarding skills as weak or non-existent; six rated themselves reasonably competent at the keyboard. None had had prior experience with CMC.

The group, then, represented a true sample of novice CMC users with whom it would be possible to monitor the process of becoming more expert CMC users.

A key finding of the study was the complexity of the process of moving from non-user to more experienced CMC user. This complexity is reflected in the fact that we found three important factors influencing the participants' use of the Chimo system: access to equipment and a local resource person, knowledge of the CMC system, and the incentive/motivation structure operating during the CMC experience.

## Access to Equipment and Local Resource Person

It is important to note that the CMC system (Chimo) that was used in this project was very flexible in that a wide variety of computers, modems, and related software could be accommodated. Such flexibility meant greater choice over equipment selection, yet it also meant more questions about and non-standardized answers to equipment decisions. Added to this were such issues as difficulty in setting up equipment, problems with faulty modems, deciding whether to install a second telephone line, blocking time to use CMC within the constraints of a busy daily schedule, and deciding on the best permanent location of the equipment (at home or at work). These latter two points are highlighted in this comment by Bob:

> "Even though I have set up in my office, I still have to wait till the secretary's finished and try to sneak her computer away, and by the time I get to it, I'm tired...or come in early in the morning and do it...So it's awkward. Then you have to set it all up and then as soon as it's time for the secretary, get it all back, set it all up [for her]...It's easy to spend two hours on it without much trouble."

As the CMC literature suggests, two of the most compelling reasons for using CMC are time and place independence (e.g., Harasim, 1989). This means that users can login and use the system at any time of the day or night and from any location that provides a connecting telephone line. These very strengths however, pose serious challenges for new CMC users, who are faced with the
difficulty of selecting, acquiring, and setting up equipment and then acquiring the necessary skills and knowledge to use the system independently.

One of the options that the group discussed as a solution to the problems of limited access to equipment was to have a computer at home. However, having a computer at home was not necessarily an easy solution to increasing access to CMC. Bob commented: "I'd rather [have it] at home, but I'd be dragging my computer back and forth each day." The home option also meant no secretarial assistance (useful for typing in particularly long messages). Thus, the home option could further contribute to the awkwardness of using the system. Furthermore, Bill, who opted for the home setup, found that although he was able to schedule access to the machine more easily at home than he might have from his office, he felt quite isolated in the sense that he had no direct access to a computer resource person. As he put it, "I'm 100% on my own."

As implied in the above comment, access to a local computer resource person was also an important factor influencing the group's socialization. Because the participants were not expert computer users, and because they would be setting up and using their equipment at their own locale, we encouraged them, right from the beginning of the project, to link with a computer resource person within their board who would be able to help them choose and set up equipment and use the system. It should be noted, however, that we did not rely on this individual as a key socialization agent; but instead, we incorporated a broad range of socialization strategies involving lectures, group discussions, demonstrations, exercises, personal manuals, and access to experts (see also "Knowledge of the System").

As the CMC experience unfolded, it became clear, however, that the computer resource person within the board could play a pivotal role in diminishing an individual's frustrations by helping set up equipment, demonstrating how to use specific terminal software, and re-explaining how to get online. For those participants who had access to an on-site person, such frustrations were greatly reduced as the following comment by Andy indicated:

"I just handed it over to my computer resource person because we have him in the school and said, 'Fix it up. I don't have time to screw around. I wanna get in and out of that machine.' He came back a day later and said, 'this is what you've got to do.'"

This arrangement was ideal in the sense that not only was the computer person on-site and therefore easily available to Andy, but also, the person was knowledgeable about CMC and prepared to be supportive in helping Andy become a more experienced user.

For many others, however, the local infrastructure was not so well-established. Having a computer resource person on-site was not the norm. A more typical arrangement was limited access to the board computer resource person who made a quick visit to the school to set up equipment. As John
explained, help tended to be restricted to the initial stages:

> "The computer person with the board came in and...set all the hardware up, [then] said 'You've got the manual, congratulations, read it, and away you go'."

These two comments, then, highlight the importance of having easy access to a local resource person who can support the new user as she/he gains more experience in using CMC. This is especially important for the new user who does not have much computer experience. Technical and conceptual support is important to help new users feel comfortable and confident in using the system independently.

## Knowledge of the System

The second factor influencing participants' use of CMC was their knowledge of the system. As relatively inexperienced computer users and new users of CMC in general and Chimo in particular, the group faced a much greater challenge than the current literature on CMC would lead one to believe. The learning process involved understanding and mastering a myriad of unfamiliar tasks and concepts. These included:

* gaining a conceptual understanding of CMC;

* learning how to operate the computer/modem/printer/communications software program;

* acquiring some basic, but essential, typing skills;

* learning the correct login procedure;

* understanding the different levels of Chimo activity;

* becoming familiar with the Chimo structure of mail, open/closed conferences, and topics;

* mastering Chimo commands to send/receive mail, learn about different conferences, join conferences, read/reply to conference messages, search conferences, find user identifications; and

* learning advanced skills such as edit/upload/download text and word processing programs.

In addition to these specific skills for using the system, participants also had to develop strategies to distinguish among problems associated with hardware, with software set-up, with their ignorance of the system and with the telephone line or host computer. The following comment by John highlights some of the frustrations of the learning process for a new user:

> "As a first time user, ... I found that I wasn't sure whether I was making a mistake. . . I worked on something for approximately 12 hours before I gave Jack a call and said 'what the hell am I doing wrong?'. . . He said 'you've got a problem somewhere out of Toronto...Call your consumer;
rep' ...[The rep] walked me through the problem...and in about 30 seconds he had the problem solved."

What's more, because they did not have much experience with computers or the Chimo system, participants often had weak trouble-shooting skills. As a result, they found that they were spending considerable time trying to deal with minor problems which a more experienced user could have easily resolved. Frank commented on how his lack of experience limited his ability to resolve problems easily:

"...all the frustration ...not only the technical stuff, which I know nothing about and don't care to know anything about, but if something goes wrong with the computer, they [computer experts] know which buttons to push, who to call to get it corrected, [and whether] it is correctable on-site."

In order to help participants cope with the complex task of learning and mastering the system, we provided a number of different support structures both online and offline. These included an overview of CMC before the project began; an introduction to Chimo at the initial training session; a limited hands-on session demonstrating correct procedures for logging in; periodic demonstrations for individuals or small groups who were experiencing specific problems; telephone access to the computer consultant for the project; online access to local and Chimo level "experts;" manuals for iNet and Chimo; occasional reviews of essential commands; and regular debriefings to discuss current concerns and to exchange information about technical and conceptual problems. Also, participants sometimes informally assisted each other. During the early hands-on stage, two of them arranged a session together on the same computer. Others sometimes telephoned each other or discussed specific problems informally during on-campus meetings.

Despite the difficulties in learning and mastering the system, participants did manage to get online and explore various aspects of the system. The following section highlights what incentives motivated them to persevere with the CMC experience.

## *Incentive/Motivation Structure*

Incentive/motivation structure was the third factor that emerged from the data as important in terms of influencing the group's participation in the Chimo experience. This factor can be thought of in terms of two types of motivation: intrinsic (rewards related to outcomes mediated within the individual) and extrinsic (rewards provided by other people).

Questionnaire data revealed that intrinsic factors such as stubbornness, curiosity, and the challenge of a new experience were important factors in stimulating participation. In addition, participants were predisposed to learning about CMC. Even before they had any extensive 'hands-on" experience
with CMC, the principals were enthusiastic about getting involved in the project. Among the more popular reasons that they cited for their positive attitudes were the following:

* opportunity to explore the potential of this emerging technology;

* opportunity to improve personal knowledge of computers;

* possibility of interacting with people in the course; and

* exposure to other educators.

There were, however, few formalized extrinsic benefits to warrant time spent online. In other CMC projects, participants were often required to be online in order to receive information or to meet course requirements. In the present study, however, the CMC experience was non-mandatory and formed only a part of the non-credit, in-service course. Thus, no sanctions could be applied to compel people to be online. Instead, we offered ongoing support in several ways. As indicated in the previous section, we provided a number of online and offline support structures to help the participants learn and master the system. Occasionally, the local computer resource person provided critical technical and moral support. In addition, as an initial symbolic incentive, we offered a free car wash to the first person to complete the first technical exercise (this also became an ongoing sharedjoke that seemed to enhance interpersonal relationships). We also encouraged and persuaded participation via online course exercises and messages, suggestions to read specific messages in open conferences, and quick feedback to incoming mail. During face-to-face sessions, we also appealed to their sense of professionalism (see also "Specific Methodological Techniques").

To some extent, too, -both the investigators and the participants themselves — served as “cheerleaders” by acknowledging tiny successes on the system. Online conference exchanges were occasionally humourous, which seemed to simultaneously acknowledge frustrations yet inject a sense of fellowship. The following set of conference exchanges between two participants provides a noteworthy example of this peer support system:

Message 1: "..I have no idea if you will ever read this [conference] message, which is just as [sic] well because i stillxD [sic] don't know what the hell I'm doing..."

Reply by peer: "Yes....your message came through clear as mud...[I'll] look for your next message with great anticipation."

Message 2 (Titled "Don't worry, be happy!"): "COME IN FRANK ! DO YOU READ ME? TEN FOUR OVER AND OUT OF MY MIND. ##$%"&*()*&A% so there!"

Reply by same peer: "...I think we are on board and have this conference format well in hand"...

Later comment by same peer: "...nice to see that you have mastered this... "

While there were few explicit extrinsic rewards in the project, one of the positive features that was mentioned by participants in debriefing sessions
was the fact that participating in online activities represented no financial cost to the participants. As a result of this no-cost feature, one could suggest that the project represented a low-risk opportunity in terms of financial considerations for exploring the potential of a new communication medium. Thus, participants were likely motivated to use the Chimo system for the duration of the project in part because it would not cost them anything in terms of their own school or board budget.

It is indeed noteworthy, then, that despite the challenge of access, the complex learning process associated with CMC mastery, and the relative absence of explicit extrinsic motivating factors, most participants devoted many out-of-class hours to the project, and, by the end of the project, all but one of the ten online participants had mastered the basic elements of CMC. At the same time, it is also important to note that, by the end of the study, only four had judged themselves capable of using (sometimes with outside help) the more sophisticated aspects of the system that seem to us necessary for meaningful online problem-solving interactions (e.g., uploading and downloading text to and from the system). What's more, only one of those four had reached the stage where he was able to easily download and upload information to and from his computer disk and the system.

## Key Contextual Factors

Our second set of findings relate to the contextual factors influencing the participants' experience with and perception of CMC. Because the 12 participants of the Chimo study represented a special sub-group of educators — school administrators — we were interested in exploring with them their experience with and perceptions of CMC in relation to their position as school principal (as noted earlier, eleven of the twelve were practising principals and one was a vice-principal). Our findings can be grouped into two broad categories: time implications and the issue of relevance.

## Time Implications

In a recent thoughtpiece on the role of the principal, Fullan (1988) made the point that today's principal faces the problem of demand overload. Citing a recent study involving 137 principals and vice-principals in the Toronto Board of Education, he noted, for example, that in the last five years

> time demands ... were listed [by the respondents] as having increased in dealing with parent and community groups...trustee requests... administration activities ... staff involvement and student services...social services ... [and] Board initiatives. (Fullan, 1988, pp. 1-2).

It is understandable, then, that the participants of the present study viewed CMC in terms of its real and potential impact on their time.
As already noted in this paper, moving from new-user to more experienced user of CMC was a challenging, complex process. The administrators also found the process very time consuming, as a re-reading of several already cited comments would reveal. Finding time at school to use CMC was also a challenge, for several reasons. Time online meant "shutting down office time" because a telephone line reduced or, in the case of some elementary schools, temporarily severed all telephone access. When equipment was shared it could also mean added set-up time and/or interfering with a secretaj's routine. When they did attempt a login, they sometimes experienced frustrating and time-consuming system problems, beyond their control, that bumped them offline. Once logged in, they were also sensitive to how they used their time. In their view, using the system for more social, as opposed to purely professional exchanges, was wasteful, as suggested by this comment by Joe:

"My time is budgeted.. I don't have time to CHAT. I have time to share information and hopefully to obtain information."

They also pointed out the potentially heavy time commitment and low personal payoff for a peer willing to share a good idea, as suggested here by Bob's comment:

"...I could see another cost — the cost [to] the person who has something good that everybody wants to hear about . . . a really interesting project...So where does he get the time to answer? I know he can put it on a bulletin board, but he's got to upload a lot of stuff. I'm wondering, where does he get the inclination? . . .Why should he invest all that time to pump out that information..."

Finding time to use CMC at school may also be complicated by the fact that principals do much of their work outside their office. In a recent well known study by Morris, Crowson, Porter-Gehrie, and Hurwitz (1984) both elementary and secondary principals were found to spend less than half of their workday in their inner office. In the present study, Andy made the same point this way:

"[Your staff] want to see you in your school. They want to see you visible. They want to see you walking in and out of classrooms and knowing what's happening in terms of instruction and where the school is going...I read the [school] announcements, and I get into the halls before the bell rings..."

Thus, even under the most ideal school conditions - a separate phone line and a computer-and-modem within arms' reach - finding time to log in could still prove problematic.

On the positive side, the administrators also noted a number of ways that CMC could potentially be used to reduce the time constraints associated with
the principalship. After experiencing the electronic mail facility of Chimo, they quickly saw the potential for eliminating frustrating telephone tag, providing of course that others would be hooked into the system. Another advantage to mail, as opposed to conference messages, and, of course, the traditional telephone message, was that it was possible to easily see whether or not a person had read your message. In addition, mail messages could be handled in the same way as phone messages might be handled, by simply "stacking" them (a term used by one of the participants) and then dealing with them at a particular time of day.

Perhaps the most important point they made with regard to using CMC to solve the time dilemma facing principals has to do with its perceived relevance to the job, as highlighted here at two different times by Phil:

'Create a need. . .somehow provide to administrators a service, information, [something] that is going to have value to them to the extent that they are willing to sacrifice of their current time, whatever the cost...' [Later he adds]: "I'd [use it then] because I'd become more knowledgeable. [with] more information [I] could make decisions quicker, put things together faster. In fact, you'd have more time!"

We turn now to this issue of relevance, the second and final contextual factor influencing the principals' experiences with and perceptions of CMC.

## CMC Relevance to the Principalship

A second key contextual theme emerging from the data was the extent to which CMC was or could be a relevant tool for principals. As already noted, CMC could be both time consuming and time saving, a particularly relevant issue for an already busy administrator. Also, as suggested above, CMC's relevance to principals hinged on its utility as an information-gathering and decision-making aid. This latter point warrants additional clarification.

The administrators' insights on CMC relevance can be grouped into three sub-themes: their experiences with Chimo open conferences, their role as principal, and the potential value of CMC for the principalship.

Regarding their experiences with Chimo open conferences, it should be first recalled that Chimo was designed specifically for and open only to educators and that subscribers increased from 300 to 500 over the course of the present study. One would think that such a system could offer a viable and useful forum for school administrators. For that reason, we encouraged the participants of our study to explore and exploit, if possible, the "open" Chimo conferences.

The administrators expressed several reservations with their open conference experience. First, they felt that many exchanges were of a social and personal nature, and therefore irrelevant to them. They felt strongly that, to be relevant to them, clear expectations should be laid out for all users, and that "it's for business and not for personal communication." What's more, the
system did not easily permit conference participants to bypass irrelevant exchanges prior to first reading.

Second, the open conferences lacked relevance in that there was no guarantee of feedback from a deposited question or comment. We dubbed this tendency the "black hole phenomenon", because of the real possibility that a comment or question would remain forever within the conference yet yield no visible response. Phil explained the problem this way:

> "...it's really a hit and miss situation....If you have a particular interest, you tack it up. . . hope somebody will take the time to read it, and then hopefully take the time [to respond]. [What's more] you hope they know more than you."

Third, a lack of relevance was also felt in terms of a lack of suitable reference group. True, Chimo was composed of educators. However, there were few principals online, and it appeared from conference comments that much of the information being exchanged was most appropriate for subject area teachers, department heads, or very knowledgeable computer users.

The issue of CMC relevance can also be directly linked to the nature of the principalship itself. Viewed strictly in terms of how the principal spends most of his or her work time, CMC use may well be an inappropriate or, at least, peripheral communication tool. Other research on the principalship suggests that the principal's role revolves around on-site and verbal interactions with a variety of people. Gronn (1982), for example, found that six of the eight studies on the principal's work that he analyzed reported that principals talk from two-thirds to three-fourths of the working day. Similarly, Morris and colleagues (1984) conclude that

> the principal is a talker and listener...a quintessential paradigm of what Marshall McLuhan called the "cool medium" (non-print) commu-nicator, relying almost exclusively on the spoken word in conducting the work of the school (pp. 55-56).

CMC comments from the participants of the present study reflected this bias toward verbal interactions; these included, for example, the medium's lack of "brainstorm effect" of a personal dialogue, the lack of spontaneity, of affective dimensions and of immediate feedback, and a sensitivity to the permanence of conference exchanges. What's more, they seemed to question the appropriateness of using CMC or the computer while on the job. Joe, for example, questioned the behaviour of a colleague in his board who spends, in his view, too much time at the computer terminal, and then he added, 'We're people people. That's the secret to our success as a principal.'" Bob agreed that there was little scope for CMC or computers in a principal's daily routine. He commented:
"It's the kind of activity, though, that I don't think that I'm going to do a lot of work as a principal. Like if I'm going to do it, I'm going to do it at home as a sort of hobby. To me, to spend my hours of my day on a computer is ridiculous. I don't think it's the right thing to be spending time in front of the monitor."

Taking a broader view of CMC and its potential relevance to their job, the participants were, however, cautiously optimistic. In addition to providing useful communication linkages within a board, they identified several potential advantages to a CMC network that reaches beyond the regional level. CMC could be used by a principal to keep in regular contact with a group of youngsters on an extended fieldtrip. Similarly, it could be used as a planning and follow-up tool to enhance the educational value of a student exchange program or as an ongoing exchange medium among students from different parts of the country or world. It could also play a role in the "electronic classrooms" currently being piloted in some boards.

CMC could also be used to help reduce a feeling of isolation that sometimes characterizes the principal's job. John explains:

"I started off as a vp, then the principalship without a vp. It's the loneliest position....I felt [with] the conferences...you have somebody to kick ideas off.. . somebody to give you feedback.. . to help you in the area of curriculum implementation....How did you handle this situation?...You could call it the Maytag conference."

It was also pointed out that CMC could also be used as a tool to enhance a principal's instructional leadership. Joe, for example, explains the networking potential for himself and his staff:

"So as a principal, I would try to say, could I use that for department heads . . . where they could come in and monitor conferences, where you could use that as away of getting your staff involved in communicating and opening doors between them and people elsewhere...you get an overview yourself. There may be some particular things that you might want to plug into."

Also, it was noted that many senior positions in school boards and at the Ministry level require incumbents to wear many hats, to be generalists. CMC, it was pointed out, could potentially provide principals access to experts, many of whom may be buried deep within the educational bureaucracy. Frank explained:

"Networking...I think that's its strengths. If we were all hooked up and I pick up on something like 'articulation'...Very quickly I can get online to X and say 'look can you tell me something about articulation' and bingo it's right there...I can see potential for principals who operate from research rather than intuition."
Principals are also often called upon to sit on committees to help solve board-wide concerns. CMC could potentially serve as a problem solving tool for such committees, as this comment suggests:

> "Say we get seconded to a committee for other kinds of topics. And the problem with that...advisory committee that you get put on, [is that] it's only as good as the sum total of the people present in that room. And so you gooffandyoumake some kindofdecision, andit'sreallyarotten decision, because the sum total is rotten...I think you view this so often within our own geographical area; we constantly reinvent the wheel, And we just basically line up; we have the same kind of successes and the same kind of failures as other people have already had. Wouldn't it be nice to just short circuit that loss of time."

It is also noteworthy that, despite the brevity of the Chimo project and their very limited CMC skills, several participants initiated several CMC activities within their schools. By the time the study had ended, one principal had already completed a Chimo hookup which had helped him keep in touch with students throughout their five-week field trip, and he was already involved with similar plans to have another group of students keep in touch with the school during a student exchange project. Another had begun downloading conference exchanges and casually passing them along to appropriate department heads for their reactions; and he reported that several staff members responded very favorably to the notion of computer conferencing among peers. He was also preparing a brief to his superintendent about the possibility of using CMC within his board. Another principal had demonstrated Chimo at a meeting of his peers and during a class period involving gifted youngsters. Another was petitioning his board for telecommunications costs on behalf of one of his teachers who hoped to use Chimo for classroom exchanges and guidance counselor conferencing.

In sum, then, the administrators identified both positive and negative aspects of CMC as a real and potential tool for the school administrator. Time and relevance emerged as the two key factors affecting its real and potential use. On the whole, it is fair to say that they found CMC to be in its infancy stages in terms of its current practical applications for school administrators. It is also fair to say that CMC did not, could not, and should not, play a central communication role within the principalship. At the same time, they were cautiously optimistic about its potentials as a planning tool, an instructional strategy, a peer networking system, a professional development tool, a human resource database, and a problem-solving tool for special committees.

## SUMMARY AND CONCLUSIONS

As CMC becomes more widely known as a potential communication tool in educational circles, new user socialization and job-related contextual factors
will become increasingly important issues. The present study has focused on both of these issues by examining the experiences of one "new user" group — experienced school administrators who were enrolled in a non-mandatory, university in-service leadership course.

Our findings that are related to new user socialization, that is, the process of moving from non-user to more experienced user of CMC!, suggest that the process is very complex for new or relatively inexperienced computer users. Three major factors were found to influence the participants' socialization. The first factor was related to "access," that is, access to equipment (including issues related to selection, set-up, location, and operation of local equipment) and easy access to a local computer resource person who is both knowledgeable about CMC and supportive.

The second factor influencing their socialization was the overall complexity of the CMC learning process, which required the new users to confront a myriad of questions, conceptual hurdles, and technical challenges. Acquiring CMC competence was more than learning a few basic skills. For the inexperienced computer user, it involved learning to ask the right questions; locating the general source of a problem; solving minor, yet essential, problems; and acquiring a relatively sophisticated set of CMC skills.

The third factor influencing socialization was the incentive/motivation structure operating during the CMC experience. The participants were intrinsically motivated to explore CMC by the desire to enhance their personal and professional lives. Except for the fact that their online time was cost-free, there were in fact few formalized extrinsic benefits to using CMC. Because of the non-mandatory, in-service nature of the course, and because participation in the CMC experience was voluntary, sanctions could not be applied to compel participants to be online. Extrinsic motivators took the form of encouragement, persuasion, and training-support offered by the investigators and, to some extent, by local computer resource persons and the participants themselves.

Access, system knowledge, and the incentive/motivation structure were interactive in that they limited and enhanced the participants' involvement throughout the CMC experience.

Our second set of findings are related to the contextual factors associated with the participants' role as school principal. Two salient, inter-related themes were identified: time implications in using CMC and the relevance of CMC for the principalship.

Time implications of learning and using CMC skills were viewed as both positive and negative. On the negative side, the time commitment required to learn and then productively use the system was a definite drawback for participants, who already faced heavy time demands on the job. Actively participating online could represent a high-cost, low-yield use of time by the contributor. What's more, finding time for CMC seemed complicated by the fact that the principal's job is often enacted outside the inner office and in a variety of school settings. On the positive side, the participants identified mail
messaging as a means of eliminating time-consuming telephone tag.

The second contextual finding concerns the issue of CMC relevance to the
principalship. In addition to time relevance, CMC's real and potential rele-
vance rested on its utility as an information-gathering and decision-making
aid. Although the CMC system that was used in the study permitted mail and
open conference exchanges with several hundred other educators, the partici-
pants of the study found that open conference exchanges were often more
personal than professional, that feedback from messages deposited into open
conferences could not be guaranteed, and that a relevant reference group
(other school administrators) did not exist. Comments made by the partici-
pants also suggest that CMC may not be particularly suited to the principal-
ship because principals rely heavily on verbal communication in the running
of their schools. At the same time, the participants identified several potential
uses of CMC that may one day enhance the principal's role. These included
using it as a planningtool, an instructional strategy, a peer networking system,
a professional development tool, a human resource database, and a problem-
solving tool for principals seconded to special committees.

Further research is warranted to identify to what extent these uses of CMC
are typical for other school administrators. Additional research is also needed
to examine what role CMC can play in school administration. Do principals see
it as an appropriate administrative and/or teaching tool for themselves, their
staffs, their students? What other job-related contextual factors influence
participation in CMC activities? What conditions at the board and school level
enhance and/or restrict the involvement of school administrators in CMC?
Whether or not CMC will one day become a useful tool for school administra-
tors rests in parton the perceived value placed on CMC by their superiors, their
staffs, and themselves. It would appear, from our research, that, given the right
conditions, CMC could serve well in a supportive but not central role for
principals. However, further research is needed in order to determine whether
or not CMC becomes an enduring and useful communication tool for school
administrators.

Additional research is also warranted to investigate how other groups of
educators participate in CMC activities. Do similar socialization patterns
emerge as new users gain more expertise? What other factors facilitate and/or
limit involvement? Does CMC have a role in professional development? How
does a text-based environment influence learning strategies? Whether or not
CMC will one day become a viable communication tool for a broad segment of
the educational community rests in part on sound professional development
strategies that consider and anticipate the needs and concerns of inexperi-
enced computer and CMC users. Further research is needed to explore how
CMC can be effectively incorporated into ongoing professional development
initiatives for educators.
REFERENCES

Adler, P.A. & Adler, P. (1987). *Membership roles in field research*. Newbury Park, CA: Sage.

Everhart, R.B. (1988). Fieldwork methodology in educational administration. In N. J. Boyan, (Ed.) *Handbook of research on educational administration*. New York, NY: Long-man.

Fullan, M.G. (1988). *What's worth fighting for in the principalship?* Toronto, ON: Ontario Public School Teachers' Federation.

Gronn, P.C. (1982). Neo-Taylorism in educational administration. *Educational Administration Quarterly, 18*(4), 17-35.

Harasim, L. (1987). Teaching and learning online: Issues in computer-mediated graduate courses. *Canadian Journal of Educational Communication, 16*(2), 117-135.

Harasim, L. (1989). Online education as a new domain. Forthcoming in R. Mason, & T. Kaye (Eds.) *Computers, communications and distance education*. Oxford: Pergamon Press.

Hiltz, S. (1986). The "virtual classroom": Using computer-mediated communication for university teaching. *Journal of Communication, 36*(2), 95-104.

Kaye, T. (1987). Introducing computer-mediated communication into a distance education system. *Canadian Journal of Educational Communication, 16*(2), 153-166.

Lincoln, Y., & Guba, E. (1985). *Naturalistic inquiry*. Newbury Park, CA: Sage.

Mason, R. (1988). Computer conferencing: A contribution to self-directed learning. *British Journal of Educational Technology, 19*(1), 28-41.

McCreary, E., & Van Duren, J. (1987). Educational applications of computer conferences. *Canadian Journal of Educational Communication, 16*(2), 107-115.

Morris, V.C., Crowson, R.L., Porter-Gehrie, C., & Hurwitz, Jr., E. (1984). *Principals in Action: The reality of managing schools*. Toronto, ON: Charles E. Merrill.

Siegel, J., Dubrovsky, V., Kiesler, S., & McGuire, T. (1986). Group processes in computer-mediated communication. *Organizational Behavior and Human Decision Processes, 37*(2), 157-187.

Wilton, J. (1988). CHIMO: The new kid on the block. *ECOO Output, 9*(4), 23-25.

AUTHORS

Janice Ahola-Sidaway is an Assistant Professor in the Faculty of Education at the University of Ottawa, 145 Jean Jacques Lussier, Priv., Ottawa, ON KIN 6N5

Margaret MacLean is an Association Professor in the Faculty of Education, University of Ottawa.
Jack Treuhaft is a PhD Candidate in Education at the University of Ottawa.

ACKNOWLEDGEMENT This research was jointly funded by the University of Ot-
tawa's Teaching Resource Service, the Teleducation/Continuing Education De-
partment, and the Faculty of Education. A preliminary draft of this paper was
presented at XVII Annual Conference of the Canadian Society for Studies in
Education (Canadian Association for the Study of Educational Administration),
Laval University, Ste-Foy, Quebec, June, 1989.
<figure><img src="image_13.png" /></figure>

Create the Wave: the Third Educational Technology

St. John's Newfoundland

June 9 - 12, 1990

Hotel Newfoundland

********

Preconference workshops, keynote presentations and concurrent sessions will focus on:

* Instructional Development

* Media Production

* Computer Applications

* Learning and Instructional Theory

* Resource-Based Learning

* Distance Learning

* School Librarianship

* Interactive Technologies

For further information on the conference, media festival, registration, contact:

Mary Kennedy or Jean Brown

Division of Learning Resources

Faculty of Education

Memorial University of Newfoundland

St. John's, Newfoundland A1B 3X8
PowerPoint

Len F. Proctor

PowerPoint is a product of Microsoft Corp., 16011 NE 36th Way, Box 97017, Redmond, Wa. 98073. (206) 882-8080. $395.00.

Minimum hardware requirements are: Macintosh Plus, SE or II with at least 1 mb of memory and two 800k drives or a hard disk.

Capturing the interest of viewers is a first priority item among teachers and other professionals who use media in their presentations. PowerPoint, one of the growing number of PC slide programs becoming available, is one more tool that can be used to achieve this goal. While this program was not intended to replace a graphic artist, it does give the ordinary presenter the ability to create professional looking slides. And, as a fringe benefit, control over the content, sequence and production time has been returned to the user.

Basic Features

PowerPoint can be used to produce 35mm slides, overhead transparencies, on-screen computer slide shows and paper-based flip charts. Sample presentations for slides and transparencies have been provided as starting point models for new users. A full set of graphics tools to create circles, squares and lines make up the heart of the drawing package. Once the images have been created, they can be moved, resized and outlined or filled with patterns, textures and color. A spelling checker has also been included with the word processor. The word processor, of course, can be used to create text slides, multi-level bullet charts and word charts or to label diagrams, graphs and tables. In each case, multiple fonts, type sizes and type styles are always available to the user. A find and replace feature has also been incorporated into the word processor. The searchfunctionfmds and replaces words orphrasesin all slides, notes, handout pages, and master templates.
## Import and Export Capabilities

The text and graphics used to create the slides do not have to originate with PowerPoint, they can be imported from other sources. For example, while PowerPoint does not have an outlining built into it, it can import text from other outlining programs such as MORE or ThinkTank. If the outline is properly organized, PowerPoint will automatically use the text to create text slides. In the graphics area, clip art and charts created by programs such as MacDraw, MacPaint and Excel can also be imported through the use of standard Macintosh cut and paste techniques. Similarly, graphics created with PowerPoint can also be exported to other compatible programs.

## Printer and Recorder Support

The Apple LaserWriter or ImageWriter can be used to print out paper copies of slides, note pages or handouts. Overhead transparencies can be made directly if transparency material is substituted for paper or from paper masters if they are run through a standard thermal copier. 35mm color slides, on the other hand, required a film recorder. If a film recorder is not available locally, the slide can be sent to Genigraphics via a modem. Once the data is received, Genigraphics will produce the slide and send it back to the user via return mail. For the presenter note pages, a miniature of the slide is automatically created on the note page along with space for presenters to add their notes. The same process is used for the creation of learner handouts. Miniatures are automatically created on the page with the choice of printing 2,3, or 6 slides per page being left to the discretion of the user.

## Documentation

As is characteristic of most Macintosh software, the documentation is very user friendly and extensive. Chapter 1, tells the user what hardware is required and how to install the program. Chapter 2, the quick tour, helps the user get acquainted with the basic features and operation of the program. Chapter 3, gives step-by-step tutorial instruction on the use of each option. Chapter 4, adds the dimension of working with color and Chapter 5, serves as the reference chapter.

## Recommendations

PowerPoint represents the distillation of many years experience in producing computer generated slides. It is a powerful, user-friendly, graphics tool that combines the collective wisdom of many typographers and graphic artists. For example, when working in color, after the user selects a background and foreground color for a slide, PowerPoint brings up a menu of six appropriate complimentary colors that could be used in other parts of the graphic. This helps the user avoid making inappropriate color choices which ultimately, may detract from the message being presented. After doing the introductory tour, the new user will be familiar enough with the basic features of the program to begin to produce high quality slides. However, like all powerful programs, it
will take the casual user some additional time and experience to become completely familiar with all of the program's capabilities. The creation of good graphics is always a time consuming process and most presenters have a limited amount of time available to produce slides, handouts and notes. Compared to the time required for their production by traditional methods, PowerPoint is certainly a step in the right direction.

## Similar Programs For Other Systems

Showoff, from Broderbund Software, is available for the Apple II computer. It is an excellent program for creating and displaying computer generated graphics. It does not yet however have the capability to output the graphics to a film recorder. For the MS-DOS world, there is the upgraded version of the aging Harvard Graphics program available from Software Publishing. It is easy to use and ideal for creating word charts. It is also the only PC slide program for the MS-DOS operating system that contains a built in spelling checker.

## REVIEWER

Len F. Proctor is an Associate Professor in the Department of Communications, Continuing and Vocational Education at the University of Saskatchewan, Saskatoon, SK S7N OWO.
<figure><img src="image_14.png" /></figure>
Book Reviews

*The Electronic Text: Learning to Write, Read and Reason with Computers* by William V. Costanzo, Englewood Cliffs, NJ: Educational Technology Publications, 1989. Hardcover ISBN 0-87778-208-3

Reviewed by Trevor J. Gambell

I usually find it difficult to warm to books or articles on computers in English language arts. Some authors are enamoured with the technology and lose sight of pedagogy, while others offer a narrow perspective of English language arts. Then there's the problem of contemporary relevance of the content; most books are outdated in terms of technological and pedagogical development before they even reach their intended readers.

I'll admit that I approached William Costanzo's book with some skepticism, but a quick reading of the table of contents promised a fresh and authoritative approach to the topic. That promise was upheld. This book deserves to be read by all English language arts educators who use, or would like to use, computers in the teaching of their subject area. As such books go it's quite lengthy at 300 pages, but it is also comprehensive and the content is well-organized into seven chapters with a foreword and afterword.

William Costanzo's credentials are quite impressive. He is a professor of English at a community college and director of the Commission on Media for the National Council of Teachers of English. His academic background is the study of English to the doctoral level. The author, then, comes to this book from the perspective of English teaching rather than that of technology, computer applications, or programming, although he clearly understands each of these fields.

Costanzo believes that word processing software embodies certain assumptions about the writing process. Therefore he has attempted in each chapter to seek "...to balance theory and research on the fulcrum of actual
practice." (p.x) However, I found research evidence very scant in many chapters, simply because the research just isn't there except for studies of student writing on computers and student attitudes toward writing. The theory base is well developed in the chapters. Most helpful to this reader was the inclusion at the end of each chapter of a bibliography separate from a listing of software mentioned or reviewed.

I couldn't help but agree in Chapter 7 with Costanzo's assertion that "no instructional software is likely to be truly effective unless it integrates our knowledge of three basic areas: subject matter, pedagogy, and technology." (p.3) In this chapter also he puts into perspective the claims of those who see the almost limitless possibilities of developments in Artificial Intelligence for teaching English with computers. According to Costanzo "...the smartest machines still know less about English than an average two-year-old." (p.18) Yet to be devised is a program that manages the complex ambiguities of English as competently as can users of the language.

The most disappointing chapter was the second, dealing with reading and electronic text. Costanzo sets up the dichotomy of the skills (decoding) versus strategies (comprehension) approach to reading, with Jeanne Chall representing the former and Frank Smith the latter. That is fair enough, but he makes some curious errors in his explanation of these two approaches. For one, Frank Smith hasn't been at the Ontario Institute for Studies in Education at Toronto for years (p.28). Smith neither adopts a psychoanalytic view (p.29), nor does he use a communication theory approach to reading (p.29). The problem, I think, is that Costanzo's source is Smith's 1971 book which has been superseded by his revised *Understanding Reading: A Psycholinguistic Analysis of Reading and Learning to Read* (1978). *Newer books such as Essays Into Literacy* (1983) and *Writing and the Writer* (1982) describe a shift in his thinking from a cognitive perspective toward a wholistic one, Costanzo really supports a psycholinguistic view of reading when he draws on Chomsky and Bever to support Smith's "psychoanalytic view."

At the end of this discussion of the two polarized theories of reading Costanzo tries to redeem himself in a summary statement.

> Perhaps the most productive way to reconcile these views for pedagogical purposes is to think about, the difference between learning to read and reading to learn. For children who are just learning how to read, it may be more important, to develop strong decoding skills. For mature reading, which is always reading for a purpose, the emphasis should shift to the strategies we use for understanding what we need. Neither emphasis should be exclusive. (p.40)

This middle-of-the-road stand may appease some members of both camps but it clearly ignores the evidence of wholistic language development, including reading-writing relationships, that is now becoming available thanks to naturalistic, case study, and phenomenological research initiatives.
Chapter 3 deals with interactive fiction, and here Costanzo provides some interesting insights into the possibilities for students' interactions with electronic fiction text. With interactive fiction the reader assumes an active role in shaping and exploring the world of fiction and imagination, and language becomes a tool whereby the reader investigates fictional environments. Costanzo notes that computer-assisted fiction typically is written in the second person singular, making the reader the main character. "The narrative evolves as a collaborative effort of the reader and the author." (p.71)

For me, Chapter 4 on writing with a word processor was the strongest part of the book, but that could be because it's my area of interest and expertise. Word processing helps writers to think of text as plastic, protean, manipulative. Writers become managers of prose; they are in a position of control. For young writers especially, word processors and computers free them from the tyranny of the text. Word processing has made it truly possible to incorporate a writing process approach in the teaching of composition.

In this chapter Costanzo has done his homework well in reviewing the research on writing and computers. We might well heed, in our enthusiasm for using computers in writing courses, research findings that point out that writing on computers does not necessarily lead to better writing. Also although students might make more changes to text on the computer such changes may be at a superficial (word or phrase) level, rather than at a deeper (ideational, organizational) level.

Costanzo is cautious of the benefits of grammar and style checker programs in Chapter 5. He calls such programs "writing aids," but warns that weak writers may be seduced by these programs, treating their advice as prescriptive medicine for ailing structures. The author admonishes that "for writers who lack confidence in their own judgment, such programs can be disastrous." (p.148)

There is much more in this book; there are chapters on natural languages and artificial intelligence, and programming for English. In the Afterword Costanzo discusses the most recent developments in computer technology with promise for English teaching such as interactive video, hypertext and hypermedia, CD-ROM, and computers sensitive to natural language. But lest we fear that the technology is getting ahead of us, he ends with these reassuring words:

> ..all these accountments of electronic texts remind you that reading means selecting from alternatives. As withreading, so with writing.

(p.234)

REVIEWER

Trevor J. Gambell is a Professor of Curriculum Studies (English Education), University of Saskatchewan, Saskatoon, Saskatchewan, S7N OWO.