Summer 2005: Volume 13, Issue 2

On CUEA Letter from the
EditorI hope this issue of On CUE ﬁnds you welland enjoying a much-deserved summer vacation. This issue is a bit thicker than usual,reﬂecting increases in both the quantity andquality of submissions we have been receiving lately.
We start off with two feature articles
dealing with student ratings of teachers
and teaching. First, Christine Winskowski
presents Part 1 of a two-part overview andcritique of conventional student ratings
instruments. In Part 2, which will appear in
the next issue, she will propose alternative
instruments for rating effective teachers
and teaching. Next, Peter Burden looks
at ratings from the studentsʼ perspective,
considering how their attitudes toward theprocess may inﬂuence results. Rounding out
the Features section is a contrastive study by
Kaoru Kobayashi which traces the use ofrhetorical patterns and structures in Japaneseand English research articles.
In a lively Opinion & Perspective section,
Gregory Strong responds to John Burrellʼs
recent article on the job outlook for university educators in Japan by comparing thesituation here with the reality for job seekers
in various other countries. Next, Michael
Guest takes issue with David Peatyʼs pointof view on global education.
The From the Chalkface sections offers
two projects for the university classroom.

First, Juergen Bulach presents his method
for using learning contracts with students.
Next, Marlen Elliot Harrison explains how
to use Impressionist artwork to encourage
creative self-expression.
Rounding out this issue are three shortarticles in the Cyber Pipeline, Conference
Reviews, and Book Reviews sections respectively. First, Brian Caspino provides internet sources of Beatles materials for classroom use. Next, Forrest Nelson reviews therecent JALTCALL conference held at the
Biwako campus of Ritsumeikan University.
And ﬁnally, Nevitt Reagan reviews Ideas &Issues, Pre-Intermediate and Intermediatefrom Macmillan LanguageHouse.
And On CUE continues to evolve. In aneffort to better serve our readers, this publication now includes Japanese summaries offeature articles. Please welcome translator
Yukihiro Kunisada to the editorial staff.
More changes are coming. My tenure as
editor is drawing to a close. The CUE executive board now seeks a new editor. I will stay
on to help the new editor get started, and I
will continue doing layout for On CUE andother CUE publications. If you are interested
in taking on this demanding and rewarding
duty, please contact me or Phil McCasland.

Mike Hood
On CUE Editor
1

From the Editor

Friends and Colleagues,

Summer 2005: Volume 13, Issue 2

On CUE

Documenting Instructor-Effectiveness, Part 1:
Vulnerabilities of Conventional Student Ratings
Instruments

Feature Article: Winskowski

Christine Winskowski
Morioka Junior College, Iwate Prefectural University
from 334 in 1998 to 547 in 2003 (RuthvenStuart, 2004). This number will undoubtedly
continue to rise in this era of shrinking student populations and tight budgets.
In examining the growth of SRI use andrelated literature, a few things become clear.
First, the majority of mainstream researchsuggests that SRIs are reliable, at least moderately valid, and useful; see, for example,
the representative arguments and literaturesummarized in Cashin (1995), Centra (1993),
Kulik (2001) and Ory (2001). Second, despite widespread acceptance, controversy
over SRI use—sometimes passionately
expressed—persists. Criticism generally
revolves around the troublesome notion ofvalidity and how it is demonstrated; biases
related to grades, workload, gender, race,and other variables; and the misuse of SRIs
by administrators. Arguments expressed by
Birnbaum (1999), Hake (2000b), Haskell(1997), Johnson (2002), Huemer (n.d.),
Lewis (1998), and Trout (2000) are representative (summarized in Winskowski,2005).
My argument is that by themselves, conventional SRIs are not sufﬁciently informative and may be misleading. A number of
weaknesses of conventional SRIs hingeon the fact that they follow conventional
survey methodology. Conventional SRIs
often use high-inference items, in which
students choose some degree of agreement or

本論は、慣習的に行なわれている学生による授業評
価法（SRI）の基づく方法論が、推論への高い依存性
や、学生の論理的思考・動機付けを紛らわす抽象的な質問に特徴付けられる典型的な例であり、その本質に於いて問題を抱えていることを議論するものである。更に、声に出して考察させる面接調査の結果
は、学生の認識プロセスが質問の内容と無関係な要素によって影響を受けていることを示唆している。
評価結果はまた、質問の順序、使われる評価尺度その他の要素によっても左右される可能性がある。更
に、広く行われている非公認SRIの使用が問題を複雑にしている。かくして、慣習的SRIの手法は評価の手段としては不正確であり、教員や教育機関にはこれに替わる他の評価手法が必要とされている.

There are some 2,000 publications in English on student ratings of university courses
and instructors (Felder, 1992; Wilson, 1998).
Student ratings instruments (SRIs) wereintroduced in a few U. S. institutions in the
mid-1920s, according to Seldin (1993), who
also notes that the percentage of colleges anduniversities in the U. S. using student evaluations rose from 29% in 1973 to 86% in 1993.
The ﬁgure is almost certainly higher today.
Use of SRIs has spread to Canada, Australia,
many countries of Europe, and Asia (Haskell,
1997; Wilson, 1998).
Japanese universities began using SRIs
relatively recently, following a mandate by
the Ministry of Education, Culture, Sports,
Science, and Technology in 1991 (A Vision,
1998; Ruthven-Stuart, 2004). The number of
institutions using SRIs has grown steadily,2Summer 2005: Volume 13, Issue 2

On CUEIn Part 1 of this study, Vulnerabilities ofConventional SRIs, we will examine theweaknesses of conventional survey methodology in SRIs. First, we will address the waythat variables are deﬁned and how high-inference items may skew results. Next we will
explore the troublesome issues of validityand bias in SRIs. In Part 2: Moving from
High-Inference to Low-Inference SRIs wewill discuss instructor-designed, low-inference alternatives to conventional SRIs.1

Conceptual deﬁnitions andoperational deﬁnitions of variables
The distinction between variables that areconceptually-deﬁned and operationally-deﬁned has long been known in social science
methodology. Bernard (2000) sets out this
distinction:
Conceptual deﬁnitions are abstractions,articulated in words, that facilitate understanding. They are the sort of deﬁnitions
we see in dictionaries, and we use themin everyday conversation to tell people
what we mean by some term or phrase.
Operational deﬁnitions consist of a set of
instructions on how to measure a variable
that has been conceptually deﬁned.
Suppose I tell you that Alice and Fredjust moved to a spacious house. Nice
concept. You ask: What do you mean byspacious? and I say, You know, it has lots
and lots of space; the rooms are big and theceilings are high. If that isn’t enough foryou, we’ll have to move from a conceptual
deﬁnition of spacious to an operational
one. We’ll have to agree on what to measure: Do we count the screened-in porch
and the garage or just the interior living
space? Do we count the square footage or
the cubic footage? (p. 36)

Note that terms 'lots of space,' 'big'and 'high,' require judgment, and people
conceive matters differently. The inclusionof speciﬁcally named and measured spaces is3

Feature Article: Winskowski

disagreement, evaluation, or frequency withstatements that (presumably) describe effective teaching. Responding requires students
to give opinions. They are not asked to document classroom events or instructor behavior
and evaluate them according to any objective
criteria. Often they are not asked to explain
how they were engaged with the coursework,
nor how they arrived at their opinions. We
must infer the meaning of student responses,
hence the high-inference label.
Abrami, d’Apollonia, and Cohen (1990)
offer two deﬁnitions of the validity of student
ratings: “the [accurate reﬂection of] students’opinions about the quality of instruction,
regardless of whether ratings reﬂect what
students learn” and “the [accurate reﬂectionof] instructional effectiveness” (p. 219).
That these two deﬁnitions may not reﬂect
or inform one another goes unnoted by theauthors. Use of students’ opinions for evaluative purposes has not, however, escaped thenotice of critics (see, for example, Huemer,n.d.; Trout, 1997, 2000; Hake, 2002b). Despite the fact that research on SRIs employs
a sophisticated arsenal of methodologicaltools, in practice opinions about classroom
events are often treated as evidence-based
observations and evaluation.
We can certainly ask students to tell us
what they thought of a course and its instructor, what happened in the course, how it was
set up and organized, what the instructor did,and how well the students learned. But wewill learn something more substantive anduseful about their responses if conventional
SRIs are supplemented with an instructordesigned student ratings instrument that a)
is based on the course objectives as Rando(2001) advises, b) refers to speciﬁc eventsand processes of the course, and the students’
engagement with it, and c) uses low-inference items to solicit information directly.

Summer 2005: Volume 13, Issue 2

Feature Article: Winskowski

On CUEto those found by other researchers, though
the authors conservatively conclude that anequally viable interpretation of factor studies
is that student rating forms measure a global
component, “General Instructional Skill”(d’Apollonia & Abrami, 1997, p. 1201), i.e.
that the sub-skills may be an artifact of the
SRIs. Correspondences in all three dimension sets are shown in Table 1.

less a matter of judgment than observation.
Bernard points out that conceptual
deﬁnitions and operational deﬁnitions are
not inimical; rather they complementary(2000). Complex variables are difficult
to express conceptually, and certainly formeasurement, must often be reduced tosimpler terms. However, conventional SRI
items more closely resemble Bernard's
example of a conceptual deﬁnition of teacher
effectiveness than an operational deﬁnition.
This resemblance, I believe, reflects thehigh-inference style in which SRI items areoften written. In the next section, a detailed
description of conventional SRI content will
illustrate this point.

Table 1. Dimensions of teaching effectivenessTeacher effectiveness as
characterized in conventional SRIs
Teacher effectiveness is a complex variable. In a series of studies dating back to
the 1970s, Feldman identiﬁed dimensionsof teaching effectiveness based on students’ideas of effectiveness and dimensions ofeffectiveness that had been examined inresearch. He illustrates 28 such dimensions(1989).
The measurement of 28 dimensions
might be unwieldy for a single SRI. A moremanageable 6 dimensions of teaching effectiveness that are frequently found in ratings
forms have been identiﬁed by Cashin (1995,
citing Braskamp & Ory, 1994).
A similar set of dimensions was derived
by d’Apollonia and Abrami (1997), in ananalysis of 458 items in 17 SRIs. Factor
analysis indicated four sub-skills represented in these items: 1) The instructor’sdelivery of information, 2) instructor’sfacilitation of interaction with students, 3)
instructor’s evaluation of student learning,and 4) a collection of miscellaneous teacher
behaviors. The ﬁrst three skills were similar

Cashin’s dimensions

Feldman’s
numberd’Apollonia& Abrami’s
number

1. Course organization &planning

5 & 50–2. Clarity, communication
skills & planning6, 9 & 50

1

3. Teacher-student
interaction & rapport

16, 18, &19

2

4. Course difﬁculty &workload

24 & 25–

5. Grading & examinations

13 & 15

3

6. Student rating of theirown learning––

These two lists of teaching effectivenessdimensions or skills, plus Feldman’s 28 dimensions, give a reasonable overview of key
behaviors and events that are associated withteacher effectiveness seen in the literature.
Indeed, commercially available SRIs allseem to contain items that reﬂect many ofthese dimensions.

Teacher effectiveness as
operationalized in conventional SRIs.
The design of conventional student ratings
are modeled on a standard questionnaire
format. Positively- or negatively-worded
items meant to measure teaching effectivenesscomprising dimensions like those listed above
are rated on a 5- or 6-point Likert-type scale
indicating a continuum, from agreement todisagreement, high (excellent) to low (poor)
4Summer 2005: Volume 13, Issue 2

On CUE

learning, course difﬁculty and workload,
anticipated grade, and student effort may
appear (Cashin, 1995; Centra, 1993). Some
SRIs can be tailored to the instructor’steaching objectives, such as Kansas State
University’s IDEA instrument. Others can be
tailored to course type, such as University ofTennessee’s Student Assessment of Instruction System (SAIS, 2003), which includes 11
versions (e.g. for small lecture/discussions,large lectures, seminar discussions, lab science classes, etc.), or University of Washington’s Instructional Assessment System (IAS)
which has 9 versions (Centra, 1993).
In Table 2, a sample of items from commercially available SRIs illustrates theoverlap of content and provides an informal
sampling of dimensions covered. To thedegree possible from similarity of wording
and common-sense interpretation, the corresponding instructional dimensions are shown
from Feldman (1989), identiﬁed as F1-28
with an identifying phrase; Cashin (1995),
identiﬁed as C1-C6; and d’Apollonia andAbrami (1997), identiﬁed as AA1-AA4.

Table 2. Comparison of items found in commercially available SRIs
Example 1. Using a rating of Not Applicable/Strongly Agree/Agree/Disagree/Strongly Disagree• The instructor's objectives for the course have been made clear. (F9 Clarity of course objectives andrequirements; C2; AA1)
• The instructor was well-prepared for class. (F5 Teacher preparation; Course organization; C1)• In this class I felt free to ask questions or express my opinions. (F16 Teacher's encouragement of questions anddiscussion, and openness to opinions of others; C3; AA2)
• The instructor was readily available for consultation with students. (F19 Teacher's availability and helpfulness;
C3; AA2)(From SIR ETS, 1971, 1981, 1989, cited in Centra, 1993. In addition to 20 items on the instructor and the course,
additional items ask about course variables [e.g. level of difﬁculty, pace, workload—F23 and 24], student variables[e.g. expected grade, reason for enrollment, GPA, class level], and overall ratings for the text, readings, exams,
lectures, and value of the course.)
Example 2. Using a rating of Hardly Ever/Occasionally/Sometimes/Frequently/Almost Always• Encouraged students to express themselves freely and openly. (F16—see above; C3)• Clearly stated the objectives of the course. (F9—see above; C2; AA1)
• Made presentations which were dry and dull. (F1 Teacher's stimulation of interest in the course and its subject
matter; C2; AA1)
• Made it clear how each topic ﬁt into the course. (possibly F5—see above; C2; AA1)
(From IDEA, Center for Faculty Evaluation and Development, 1988, cited in Centra, 1993. Additional items cover
student progress and skills course variables, student variables, and overall ratings. C6)

5

Feature Article: Winskowski

value, high to low frequency, or somethingsimilar. Centra (1993) provides helpful
illustrations of well-known evaluation forms,
including the Student Instructional Report II(SIR II) from Educational Testing Service,
Instructional Development and Effectiveness
Assessment (IDEA) from Kansas State
University, the Student Evaluation of
Educational Quality (SEEQ, developed by
H. Marsh), and the Student Instructional
Rating System (SIRS) from Michigan State
University. Illustrations may also be foundonline (SAIS, 2003; Marsh, 1976).
These and similar instruments contain bothglobal and speciﬁc items. Global items ask
for overall impressions of the course (and/or
the instructor, the lectures, the textbook, etc.).
These items lend themselves to norming andto the comparison of a particular course
against similar courses.
Speciﬁc items comprise the dimensions ofteaching effectiveness described in the previous section. For example, items measuring
instructor-student interaction, instructor’scommunication, course management, andpresentation skills, the value of assignments
and exams, fairness of grading, student’sSummer 2005: Volume 13, Issue 2

On CUE

Feature Article: Winskowski

Example 3. Using a rating of Superior/Above Average/Average/Below Average/Inferior• The instructor's enthusiasm when presenting course material. (F2 Teacher's enthusiasm [for subject or for teaching];
AA1)• The instructor's ability to relate the course concepts in a systematic manner. (F6 Clarity and understandableness[sic]; C2; AA1)
• The course organization. (F5— see above; C1)• The instructor's stimulation of class discussion. (F16—see above; C2, C3; AA2)
(From SIRS, MSU Board of Trustees, 1982, cited in Centra, 1993. 21 items include some about learning [e.g.
improvement in competence], and an additional 4 items cover course variables and student variables. C6)
Example 4. Using a rating of Not Applicable/Strongly Agree/Agree/Neutral/Disagree/Strongly Disagree• Lecturer gave presentations that facilitated taking notes. (possibly F5 and 6—see above; C2; AA1)
• Lecturer presented background or origin of ideas/concepts developed in class. (AA1)• Feedback on examinations/graded materials was valuable. (F15 Nature, quality, and frequency of feedback fromthe teachers to students; C5; AA3)
• You found the class intellectually challenging and stimulating. (F17 Intellectual challenge and encouragement of
independent thought [by the teacher and the course])(From SEEQ, Marsh 1976. Items also examined background subject/class characteristics and overall ratings.)
Example 5. Using a rating of Excellent/Very Good/Good/Fair/Poor/Very Poor
• Instructor's ability to present alternative explanations when needed was:
• Contribution of assignments to understanding course content was:
• Use of class time was: (possibly F25 Classroom management; C1)• Amount you learned in the course was: (C6)(SAIS, University of Tennessee, 2003. An additional six items ask the student to compare the course to other courses withrespect to the expected grade, intellectual challenge—F17 Intellectual challenge; compare the amount of effort given,
number of hours spent on the course—F23 & 24 Difﬁculty of the course workload; and other student variables.)fact, the ubiquitousness of SRIs could conﬁrm and reinforce stereotypes of teachingeffectiveness, and good instructors who fail
to conform to those stereotypes would not
be recognized.
The second point is that the items have an
abstract quality: They might ﬁt virtually any
class, and most solicit the students’ judgment
about the instructor’s behavior or the course;
Excellent use of class time (from Example
5 above); agreement that feedback on examinations/graded materials was valuable(Example 4); judgments that the instructor
occasionally encourages students to express
themselves freely and openly (Example 2).
Understanding responses to these items
requires a high degree of inference, since
we cannot know the events and observations motivating a student's choice. Yet, if
a student claims that examinations and assignments had no value, for example, surely
an instructor or administrator would want toknow why the student felt so. If one student
considers himself occasionally encouraged
to express himself, and another does not,There are two points to this lengthy illustration. First, while it is not a formal analysis, it shows that these items contain highly
standardized wording, so much so that mostof them echo examples illustrated in Feldman (1989), Cashin (1995), and d’Apollonia
and Abrami (1997). While items and theirdimensions describe actions and events thatare desirable, it is important to note that they
have not been derived from systematic, naturalistic observation of independently-judged
effective teachers. Nor can we know thatteachers who are judged effective by peers,
students, etc., manifest their effectivenessinevitably in the same way, as described inthese dimensions or conventional SRI items.
McKeachie considers that “effective teachers come in many shapes and sizes” (1997,
p. 1218). Using the standard dimensions
offered on conventional SRIs may penalize
unorthodox instructors, or those who are
effective despite low SRI scores. Indeed,
McKeachie suggests that negative scores on
SRIs may weigh heavily even when there isother evidence of teaching effectiveness. In
6

Summer 2005: Volume 13, Issue 2

On CUE

ensure that it discriminates between students
who positively evaluate a course or instructor and those who do not. Items which failto discriminate must be rewritten and theprocess repeated.
Validating a ratings form is a long, laborintensive, and expensive process. Validation
procedures are likely to be undertaken byresearch institutions or commercial testing
companies; individual instructors or departments are unlikely to have the resources. Overthe past 25-30 years, dozens of such studies
with SRIs have been undertaken. Greenwald(1997) notes in his overview and introduction
to a special issue of American Psychologist
on student ratings that the majority of publications in the preceding 25 years support
the argument that student ratings are valid
measures of teaching effectiveness (p. 1182).
Indeed, the general conclusion that SRIs aremoderately valid is supported not only bymany individual studies, but also by large
meta-analytic studies, such as d’Apollonia
and Abrami (1997) Cohen (1981, cited in
Kulik, 2001), and others.
Despite this general approval, there remainsome issues of validity that are particularly
relevant: non-validated SRIs, survey design and its inﬂuence on response, and thereasoning processes of students completing
conventional SRIs.

Issues of validity and bias
It is desirable that an SRI be validated,that is, shown that it indeed measures what
it claims to measure. The validation process
involves administering a new ratings instrument to at least hundreds—preferably thousands—of students to see if the results forma normal curve. The results must be shownto correlate positively with some other objective measure of teaching effectiveness.
Students’ grades are commonly considered the most direct measure of teachingeffectiveness and are often correlated with
SRI results. However, Cashin (1995) lists a
number of other, less direct measures, suchas ratings of instructors by themselves, byadministrators, colleagues, alumni, trained
observers, and written student comments.
Furthermore, in the SRI validation process,each item of the new SRI must be analyzed to

Non-validated SRIs
Given the attention devoted in SRI literature to validity, it is surprising that many
university student ratings forms have notbeen validated. Commercially available instruments like those whose items were illustrated above have certainly been extensively
validated. Still, in a recent review of the SRI
literature (Winskowski, 2005), no indicationof how widely they are used could be found.
Many institutions have developed their own
7

Feature Article: Winskowski

surely an instructor or administrator would
want to know what lies behind the difference.
Unfortunately, there is no way to know unless students choose to elaborate. Such items
are probably considered operational deﬁnitions of teacher effectiveness by advocates of
conventional SRI use, but they really appear
to more closely resemble Bernard's conceptual deﬁnitions in actual practice.
What hangs thereby? If conventional SRIs
conform to standard survey methodology,
and standard survey items are high-inference, requiring judgment on the part of the
respondent, perhaps we should just makedo. After all, taking surveys and reading theresults is a common part of the academic
experience, and most accept that surveys
represent what people think. However, theimplications of such acceptance become
clearer when we consider whether SRIs arevalid and look at the possibility of bias instudents’ responses.

Summer 2005: Volume 13, Issue 2

Feature Article: Winskowski

On CUE

student ratings forms, adapting the wordingof items and rating scales from other instruments. Ory and Ryan (2001) note that:...it is fair to say that many forms used
today have been developed from previous forms without much thought totheory or to construct domains. We havesome evidence that traces the origin of
several campus forms (including ourown) to an item pool originally developed at the University of Michigan.
(p. 32)In the meta-analysis by Abrami et al.
(1990), 23 of the 43 validity studies analyzed
used student ratings instruments that wereof local or unknown origin. Trout (2000)
also states that his institution does not use avalidated form. In my own career, I have seen
on at least three occasions the hallmarks of
the inhouse-made SRI, i.e. photocopies of aword-processed or typed document with noidentiﬁcation of the source.
Given the similarity of items in various
commercial SRIs, this may not seem likeanything to get excited about. However, if
concept validation is the goal, it cannot bedismissed. The following sections illustrate
why this is so.

when respondents are confronted with survey
items that ask for self-reports, they rely on
the design of the ratings scales to make senseof the item (Schwarz , 1999).
The order of items is an important design
feature. Schwarz (1999, citing Schwarz,
Strack, & Mai, 1991) discovered that when
respondents are asked how satisﬁed they
are with their life as a whole and then asked
how satisﬁed they are with their marriages,
the correlation between responses was r= .32. However, when the question aboutsatisfaction with marriage was asked ﬁrst,
forming the standard against which generallife satisfaction was measured, the correlation between responses was r = .67. Furthermore, if a person is asked How is yourwife? and then asked How is your family?
the second question would almost certainly
be interpreted to mean everyone else besides
your wife. However, if one is simply asked
How is your family? it is taken to include
the wife (Graesser, Bommareddy, Swamer,& Golding, 1996, citing Schwarz, 1994
and others). Thus one survey item is taken
to form the context for the following item.
It is reasonable to conclude that SRI items
could have similar effects on each other. So
we might ask, should global items asking
for overall ratings of the course be placed at
the beginning or end of an instrument, andwhat is the effect?
Madsen (2005) also points out that smallchanges in wording of a survey instrument
can signiﬁcantly effect self-report data. Careful wording of survey items involves four issues, according to Madsen. Will participants
agree on the meaning of a term? Does theitem make implicit assumptions about theparticipants? Does each item measure theconcept of interest? And, can the data be accurately interpreted? She demonstrates thatalternate wording of an item, and alternate

Inﬂuence of survey design
There is a substantial body of work demonstrating that the design of a survey and its
items subtly shapes a survey-taker’s response.
Schwarz (1999) contends that self-reported
attitudes and behaviors “are strongly inﬂuenced by features of the research instrument,
including question wording, format, andcontext” (p. 93). A leading researcher in theﬁeld, Schwarz argues that when respondents
interpret survey items, they tacitly rely on theordinary conventions of conversation, suchas those described by Grice (1975), i.e. to be
informative, relevant, truthful, and clear. So
8

Summer 2005: Volume 13, Issue 2

On CUE

ratings scales yield quite different results.
For example, she asked her students to respond to both of the following items (among
others) on 7-point scale (1 = strongly agree;
7 = strongly disagree).
I regularly perform routine maintenanceon my car.

Mean=2.90

Sometimes I don’t change the oil in mycar on time.

Mean=3.27(Madsen, 2005, p. 41)

The mean ratings are higher for the second
item. Possibly the speciﬁcity (routine maintenance vs. change the oil) and whether an
item is cast in the positive or negative (I dovs. I don’t) may make a difference in how
people respond. Here is another illustrationfrom Madsen in which the phrasing of thequestion is altered:
How often do you exercise?
1. infrequently

17%

2. occasionally48%

3. often35%

In the last six months, how often have you
engaged in at least 20 min of aerobic activity?
1. almost never17%

2. less than once a week13%

3. once a week12%

4. twice a week15%

5. three times a week15%

6. four times a week15%

7. more than four times a week13%

(p. 41)
In both items, the ﬁrst choice receives 17%of the responses. Given all the choices in thesecond item, we must wonder how peopleinterpret the choices in the ﬁrst item: Whatis the distinction between occasionally andoften?
Schwarz (1999) has also looked exten9

Feature Article: Winskowski

sively at frequency scales and has found, forexample, that if people rate their daily TV
consumption on a low-frequency scale (fromup to half an hour to more than two and ahalf hours), they take the middle choice (oneand a half hours) as the norm, and it receives
the largest proportion of responses (26.5%).
When a high-frequency scale is presented(from up to two and a half hours to more thanfour and a half hours), the largest proportion
goes to the ﬁrst choice (62.5%), as we might
expect. However, in the low-frequency scale,
only 16.2% of German respondents reported
watching TV for more than two and a half
hours a day whereas 37.5% reported doing
so when the scales presented high-frequency
response alternatives (Schwarz, 1999, p.
98, citing Schwarz, Hippler, Deutsch, &Strack, 1985). This suggests that the rangeof response alternatives deﬁnes a frame ofreference; it is this frame of reference that
respondents use to estimate the frequency ofthe event, i.e. a high-frequency set of alternatives suggests to respondents that the average
TV watching time is higher.
It must be noted that Schwarz and hiscolleagues were working with individuals’
reports of their own behavior; estimatesof the frequency of instructor behavior orother classroom events may not follow theseﬁndings. Still, while only one of the conventional SRIs described in the earlier part ofthis article used frequency scales (IDEA),
a concern must be raised that any of thesurvey response scales contain assumptions
and implications that affect the respondent’s
reasoning. Ostrom and Gannon (1996) showthat there are two patterns of cognitive activation with common types of rating scale
formats that are prominent in social science
literature. In the ﬁrst pattern, the authors
found that a concept and its opposite (e.g.
honesty and dishonesty) were generated inSummer 2005: Volume 13, Issue 2

Feature Article: Winskowski

On CUE

people’s cognition by rating formats likethese (p. 305):a) very dishonest/moderately dishonest/slightly dishonest/neither dishonest nor honest/slightly honest/moderately honest/very honestb) strongly disagree the person is honest/moderately disagree/slightly disagree/neither disagree nor agree/slightly agree/moderately agree/strongly
agree that the person is honest
However, the activation in people’s thinking of a single concept (honest only) was
found when they responded to the following
types of rating formats (p. 305):c) not at all honest/slightly honest/
somewhat honest/moderately honest/
fairly honest/very honest/completely
honestd) 0.0 completely improbable theperson is honest/.17 (probability)/.34/.50/.66/.83/1.0 completely
probable the person is honest
The authors indicate that the unipolar scale
formats (c and d) provide better discrimination of the category than do the bipolar scale
formats (a and b). Yet SRIs frequently use abipolar rather than a unipolar scale.
Finally, in describing a model of humanquestion answering, Graesser et al. (1996)
point to some difﬁculties: The syntax of an
item could be difﬁcult to process; respondents may have trouble identifying the referents of technical or ambiguous nouns (seealso the following section); and respondents
may not accurately interpret the meanings ofverbs (for example, cause, consume, hurt),
adjectives (happy, wealthy, large), and adverbs (never, rarely, frequently).
These studies indicate ways in which thedesign of an SRI may inﬂuence respondents’cognitive processes. It is certainly possiblethat commercial SRIs may not have taken

all these recent ﬁndings into account. And it
is certainly conceivable that non-validated
SRIs, adapted from other SRIs by instructors or administrators, may contain the same
shortcomings. In view of these ﬁndings, it isreasonable to ask whether the design of SRIs
are theoretically and empirically sound. We
must also ask whether faculty and administrators would interpret SRI results with theseﬁndings in mind. And certainly, we must
develop a more fundamental understanding of students’ cognitive processes as they
respond to SRIs.

Students’ cognitive processes
Beyond knowing how survey design
shapes response, the way students arrive
at a choice may be surprising. BillingsGagliardi, Barrett, and Mazor (2004) conducted thinkaloud interviews to ﬁnd out what
students were thinking as they completed
a conventional course evaluation form fora science course. The SRI addressed three
dimensions of the classroom: overall course
design, educational materials and methods,
and [how various instructors] helped thestudent learn. It used a four-point scale, fromstrongly agree to strongly disagree. Twentyfour students explained how they interpreted
the items, formed judgments, and selected
the responses as they completed a typicalstudent rating form. The authors state thatwith this technique “the resulting data canreveal previously covert issues such as how
students do actually interpret items or what
may prevent them from responding honestly”(2004, p.1062).
As Graesser et al. anticipated, students often
expressed uncertainty about the meaning of
educational terms, such as 'independent
learning,' 'feedback,' and 'integration.'
They deﬁned these terms differently fromone another, sometimes in uncommon
10Summer 2005: Volume 13, Issue 2

On CUE

speculation. We simply do not know enoughabout what is obscured by SRI methodology and in what way it might be signiﬁcant.
Given the massive instructional investment
in SRIs, we should know. After all, in applied linguistics, we know that descriptionsof conversational conventions, however
intuitively recognizable, remained speculative until ethnomethodology’s ﬁrst hand-observation and corpus linguistics’ processingof language data documented actual speech
patterns and frequencies. Until some corresponding observational approach is applied
to SRIs, we can only speculate on what theresults mean.
Our weak grasp of students’ intentions,reasoning, and judgment are compoundedby other issues that have been raised in theliterature, such as biases. Biases are variables
which are unrelated to teacher effectiveness but affect student ratings nonetheless(such as liking the instructor as a person).
Potential biases include consideration ofgender, politics, or race. Among studies
reported by Cashin (1995) and Ory (2001)
examining the gender of instructors, a veryslight differences (r = .02) in favor of women
was found in one review of 28 studies. In
other studies, students rated same-gender
instructors more highly then opposite-gender
instructors. There is little research on theinﬂuence of race on student ratings, but what
there is indicates that race is not a factor,
according to Cashin (1995).
However, Sinclair and Kunda (2000)
suggest the matter may be more complex.
Looking both at university classrooms andan experimental laboratory, they found thatmale and female instructors were rated
equally highly when students had received
positive evaluation. However, students rated
female instructors lower than male instructors when receiving negative evaluation.
11

Feature Article: Winskowski

ways. When rating teaching effectiveness,
students sometimes made judgments on
criteria unrelated to classroom teaching
performance. For example, judgments weresometimes tempered by perceptions of thefaculty member's caring or effort: "I didn'tlike the lecture she gave...I just couldn't
stay awake. But I liked Dr. DD as a person,
so I said 'agree'..." (p. 1066). Sometimes
students compared instructors to each other,and sometimes they judged on an absolute
criterion. Students sometimes selected an
option for idiosyncratic reasons:
I am not sufﬁciently sure that I am
interpreting the question correctly
to say I strongly agree, so I just put'agree.' ...I never rate Integration below
agree because I believe the onus is on
me to integrate material.... A lot of
independent learning means the course
isn't providing needed material. So if I
say a lot, it's a negative.... (p. 1066)
Additionally, students avoided the lower
end of the scale and used the highest choiceselectively (Billings-Gagliardi et al., 2004).
This study strongly suggests that the student
responses involve cognitive processes that
do not correspond with what the instrument
assumes, particularly regarding how students
understand the forms and whether student
ratings in fact measure teaching effectiveness(Billings-Gagliardi et al., 2004).
While thinkaloud methodology has limitations (it is difﬁcult to work with large numbers of subjects, and responses are difﬁcult
to quantify, as the authors point out), theBillings-Gagliardi study presents powerful
criticism of conventional SRI methodology.
Proponents of conventional survey methods
may claim that idiosyncratic reasoning anddecision-making balances out across large
groups of people, rendered insignificant
by the totality of choices made. But this isSummer 2005: Volume 13, Issue 2

Feature Article: Winskowski

On CUE

Additionally, a separate study by Sinclair andKunda (1999) showed that people praised bya black professional did not express negative
stereotypes about them; those criticized by
the black professional were more likely to
express stereotypes. Together, these studies
suggest that when people receive negativeevaluation, they may employ stereotypes(e.g. that women are not as competent asmen, or that blacks are less competent thanwhites) to counter negative evaluation andpreserve their own self-image.
Noting the apparent conﬂict with previous
studies, Fich (2003) states that...the problem is that there are many
variables unrelated to the quality ofteaching that may affect evaluations
and that interact in complex ways.
Furthermore, most of this work consistsof statistical analyses, where factorsthat are signiﬁcant for a small segment of the population, for example,
women computer science professors,
can be insigniﬁcant in the aggregate
data. (p. 2)
Small but signiﬁcant interactional effects
have been found by Basrow (1995) for
teacher gender, student gender, and division.
Four years of SRI data from a private liberalarts institution suggested that male faculty
were rated similarly by male and female students in the humanities, social sciences, andnatural sciences. Female faculty, especiallyin humanities and social sciences, were rated
highest with respect to appropriate speechby female students. Female faculty werealso usually rated higher by female students
on sensitivity, student comfort, and respect.
Male students were more likely to rate female
faculty lower than male faculty on fairness,thought stimulation, and overall effectiveness. The author agrees that these ﬁndings
are small and that, while statistically signiﬁ-

cant, they may not be practically signiﬁcantacross large amounts of data. However, itappears that students may evaluate malefaculty for teaching, and female faculty forteaching and for gender. That is, “whereas
men need to be strong in such instrumental
areas as organization, explanations, anddynamism to receive good student ratings,
women need to be strong in those areas aswell as the interpersonal ones” (p. 663). One
cannot help recall the Billings-Gagliardi etal. illustration of a student altering his or her
original rating because, “I liked Dr. DD as aperson” (p.1066).
These ﬁndings reveal that vulnerabilities
may emerge when SRIs hinge on students’assumption-informed judgments rather thanon more objective reﬂection on classroom
experience. In other words, if we accept the
representations of teacher effectiveness as
documented by conventional SRIs, we can
expect bias.

Other biases
Other biases, well-known in social
psychological research, must also be
considered. First is the error of central
tendency, which refers to people's inclination
to avoid the extreme choices in a survey item.
It may well be that this is what BillingsGagliardi et al. observed when they notedthat students tended to avoid using the lower
end of the scale, and used the highest choiceselectively, while using the second-highest
choice indiscriminately (for a variety ofreasons). It would be difﬁcult to know theextent to which this distribution is a resultof the error of central tendency or a measureof teaching effectiveness.
Second is the halo effect, in which one’s
judgment of a person on a given aspect inﬂuences judgment on a number of other aspects.
Coren reports that at his institution, three12

Summer 2005: Volume 13, Issue 2

On CUE

Conclusion
Conventional SRIs offer standardized
products which readily lend themselves to thekind of nomothetic research that permits cor13

Feature Article: Winskowski

relation between student ratings and course
variables (e.g. workload, required/elective
status, and size), student variables (e.g.
grades, motivation, and major) and instructor
variables (e.g. gender and race). This kind ofwork is valuable to disambiguate complex
classroom dynamics that may impact student
ratings (see, for example, Greenwald, 1997;
March & Roche, 2000.) Indeed, it is in thistechnical sense that SRIs are claimed to bevalid, i.e. that SRIs are consistently correlated with grades, and that grades measure
the effectiveness of teaching.
However, we have seen that conventional
SRIs contain inherent vulnerabilities: Standardized forms may not discern effective
teaching in individual cases. It is known that
non-validated SRIs are widely used (though
it is not know how widely). How respondents
construe item meaning is impacted by the order of items. The rating scale may inﬂuenceinterpretation and response. Reasoning processes during choice selection can be affectedby a number of irrelevant factors, including
well-known, systematically observed biases.
When examined from this perspective, thefundamental meaning of SRI validity (i.e.
that they measure teaching effectivenessin a particular instance) is called into question. SRIs, as Fich (2003) has pointed out,
have many “sources of error” and are “lowprecision instruments” (p. 3). At the very
least, we must concede that interpretationof teaching effectiveness from SRIs should
be qualiﬁed.
Nonetheless, there are alternatives. Instructors who are seriously interested ininvestigating and documenting the effectiveness of their courses need not settle for SRIs
that reﬂect the realities of their classes andthe experience of their students so imprecisely and problematically. Instructors candesign their own complementary alternatives

items were added to an SRI asking students
whether the instructor had displayed any
apparent cultural bias, racism, or sexism.
Examining results of 248 SRIs, he foundcorrelations (p < 0.001, with most r’s ranging from 3.0 to 4.9) between ratings on 14
teacher effectiveness items and ratings on thethree bias items. He concludes that students’judgments of the political and social attitudesof their instructors may inﬂuence their own
attitudes toward the instructor (1995).
Third, the fundamental attribution error
refers to the inclination to attribute behavior to internal causes (attitudes, personality,
mental states) rather than external causes(situational constraints). This phenomenon
is well-documented in the social psychology
literature. Suppose an instructor presents
an unpopular political view (e.g. of Fidel
Castro, as in Jones & Harris, 1967, citedin Myers, 1993), or presents evidence thatgenes, as well as culture, could contributeto differences in intelligence across racial orother groups (Coren, 1995) or to differences
between men and women (de Courten-Myers, 1999). It has been demonstrated thatpeople will often attribute to the presenter
attitudes and beliefs that are politically, racially, or sexually biased (i.e. attribution tointernal causes) even when that person was
simply presenting information. People may
have difﬁculty separating the message andthe messenger.
Of course, we assume students intend wellwhen they complete SRIs. The difﬁcultyis that, since we do not have the luxury of
thinkaloud interviews for all our classes, wecannot know when such biases are at work.

Summer 2005: Volume 13, Issue 2

On CUEto the conventional SRI which ﬁt speciﬁc
instructional goals and course conﬁgurations.
This will be addressed in Part 2.

www.ncsu.edu/felder-public/Columns/Eval.html
Feldman, K.A. (1989). The association between
student ratings of speciﬁc instructional
dimensions and student achievement: Reﬁning
and extending the synthesis of data frommultisection validity studies. Research inHigher Education, 30(6), 583-645.
Fich, F.E. (2003). Are student evaluations ofteaching fair? Computing Research News, 15(3). Retrieved March 15, 2005, from http://
www.cra.org/CRN/articles/may03/ﬁch.html
Graesser, A.C., Bommareddy, S., Swamer, S., &Golding, J.M. (1996). Integrating questionnaire
design with a cognitive computational modelof human question answering. In N. Schwarz
and A. Sudman (Eds.), Answering questions:
Methodology for determining cognitive andcommunicative processes in survey research(pp. 343-375). San Francisco: Jossey-Bass.
Greenwald, A.G. (1997). Validity concerns andusefulness of student ratings of instruction.
American Psychologist, 52(11), 1182-1186.
Grice, H.P. (1975). Logic and conversation. In
P. Cole & J.L. Morgan (Eds.), Syntax andsemantics: Vol. 3. Speech acts (pp. 41-58). New
York: Guilford Press.
Hake, R.R. (2002a). Lessons from the physics
education reform effort. Conservation Ecology,
5(2): 28. Retrieved March 15, 2005, from http://
www.ecologyandsociety.org/vol5/iss2/art28/
Hake, R.R. (2002b). Problems with student
evaluations: Is assessment the remedy? Message
posted to POD listserv, archived at www.stu.
ca/~hunt/hake.htm
Haskell, R.E. (1997). Academic freedom, tenure,and student evaluations of faculty: Galloping
polls in the 21st century. Educational Policy
Analysis Archives, 5(6). Retrieved May 31,
2005, from http://www.olam.ed.asu.edu/epaa/
v5n6.html
Huemer, M. (n.d.). Student evaluations: A criticalreview. Retrieved August 29, 2004, from http://
home.sprynet.com/~owl11/sef.htm
Johnson, V.E. (2002). Teacher course evaluations
and student grades: An academic tango.
Chance, 15(3), 9-16.
Kulik, J.A. (2001). Student ratings. Validity, utility,
and controversy. In M. Theall, P.C. Abrami, &L.A. Mets (Eds.), The student ratings debate:
Are they valid? How can we best use them? San
Francisco: Jossey-Bass.

Feature Article: Winskowski

References

Abrami, P.C., d’Apollonia, S., & Cohen, P.A.
(1990). Validity of student ratings of instruction:
What we know and what we do not. Journal of
Educational Psychology, 82(2), 219-231.
Basrow, S.A. (1995). Student evaluations of college
professors: When gender matters. Journal of
Educational Psychology, 87(4), 656-665.
Bernard, H.R. (2000). Social research methods.
Thousand Oaks: Sage.
Billings-Gagliardi, S., Barrett, S.V., & Mazor,
K.M. (2004). Interpreting course evaluation
results: Insight from think aloud interviews withmedical students. Medical Education, 38(10),1061-1070.
Birnbaum, M.H. (1999, Fall). A survey of faculty
opinions concerning student evaluations ofteaching. California State University, Fullerton
Senate Newsletter, XIV(1). Retrieved March
15, 2005, from http://www.faculty/fullerton.
edu/senatenews/page2.html
Cashin, W.E. (1995). Student ratings of teaching:
The research revisited (IDEA Paper 32). Kansas
State University Center for Faculty Evaluation
and Development. Retrieved March 15, 2005,
from http://www.idea.ksu.edu/resources/index.
html
Centra, J.A. (1993). Reﬂective faculty evaluation.
San Francisco: Jossey-Bass.
Coren, S. (1995, Winter). When teaching isevaluated on political grounds. The Montana
Professor, at Montana State UniversityNorthern. Retrieved March 15, 2005, fromhttp://www.mtprof.msun.edu/Win1995/Coren2.
html
d’Apollonia, S., & Abrami, P.C. (1997). Navigating
student ratings of instruction. American
Psychologist, 52(11), 1198-1208.
de Courten-Myers, G.M. (1999). The humancerebral cortex: gender differences in structureand function. J Neuropathol Exp Neurol, 58(3),217-226.
Felder, R.M. (1992). What do they know anyway?
Chemical Engineering Education, 26(3), 134135. Retrieved March 15, 2005, from http://14

Summer 2005: Volume 13, Issue 2

On CUE

Schwarz, N. (1999). Self-reports: How thequestions shape the answers. American
Psychologist, 54(2), 93-105.
Selden, P. (1993, July 21). When students rate
professors. Chronicle of Higher Education, p.
A40
Sinclair, L., &. Kunda, Z (1999). Reactions to a
black professional: Motivated inhibition andactivation of conﬂicting stereotypes. Journalof Personality and Social Psychology, 77(5),885-904.
Sinclair, L., & Kunda, Z. (2000). Motivatedstereotyping of women: She’s ﬁne if she
praised me but incompetent if she criticized me.
Personality and Social Psychology Bulletin,
26(11), 1329-1342.
Student Assessment of Instruction System (SAIS).
(2003). Ofﬁce of Institutional Research andAssessment, University of Tennessee. Retrieved
May 31, 2005, from http://www.oira.tennessee.
edu/sais/forms.html
Trout, P.A. (1997). How to improve your teachingevaluation scores without improving yourteaching! The Montana Professor, 7(3).
Retrieved May 31, 2005, from http://mtprof.
msun.edu/FALL1997/HOWTORAI.html
Trout, P.A. (2000). Flunking the test: The dismal
record of student evaluations. The Touchstone,
Vol. X, No. 4. Retrieved May 31, 2005, fromhttp://www.rtis.com/touchstone/sept00/20ﬂunk.
htm
A Vision of Universities in the 21st Century
and Reform Measures To Be Distinctive
Universities in a Competitive Environment.
(1998). University Council Report, MEXT.
Retrieved March 15, 2005, from http://www.
mext.go.jp/english/koutou/index.htm
Wilson, R. (1998, January 16). New researchcasts doubt on value of student evaluations ofprofessors. The Chronicle of Higher Education,
p. A12.
Winskowski, C. (2005). Student ratings offaculty and courses: Investigating instructor
effectiveness. Bulletin of Morioka JuniorCollege, 7, 39-62.

Notes
Part 2 will appear in the next issue of On CUE.

1

15

Feature Article: Winskowski

Lewis, R. (1998). Student evaluations: Widespreadand controversial. The Scientist 12(9). Retrieved
May 31, 2005, from http://www.the-scientist.
com/1998/04/27/12/1
McKeachie W.J. (1997). Student ratings: The
validity of use. American Psychologist, 52(11),1218-1225.
Madsen, L. (2005). Demonstrating the importanceof question wording on surveys. Teaching of
Psychology, 32(1), 40-43.
Marsh, H.W. (1976). Student evaluation of
educational quality (SEEQ). Center for
Excellence in Learning and Teaching,
Pennsylvania State University. Retrieved May
31, 2005, from http://www.backup.uts.psu.
edu/seeq/seeqdemo/seeq.tp3
Marsh, H.W. & Roche, L.A. (2000). Effects ofgrading leniency and low workload on students
evaluations of teaching: Popular myth, bias,
validity or innocent bystanders? Journal of
Educational Psychology, 92(1), 202-228.
Myers, D. (1993). Social psychology (4th ed.). New
York: McGraw-Hill.
Ory, J. C. (2001). Faculty thoughts and concerns
about student ratings. In K.G. Lewis (Ed.),
Techniques and strategies for interpreting
student evaluations. San Francisco: JosseyBass.
Ory, J.C., & Ryan, K. (2001). How do student
ratings measure up to a new validity
framework? In M. Theall, P.C. Abrami, & L.A.
Mets (Eds.), The student ratings debate: Arethey valid? How can we best use them? San
Francisco: Jossey-Bass.
Ostrom, T., & Gannon, K.M. (1996). Exemplar
generation: Assessing how respondents givemeaning to rating scales. In N. Schwarz and
A. Sudman (Eds.), Answering questions:
Methodology for determining cognitive andcommunicative processes in survey research.
San Francisco: Jossey-Bass.
Rando, W.L. (2001). Writing teaching assessment
questions for precision and reﬂection. In K.G.
Lewis (Ed.), Techniques and strategies forinterpreting student evaluations. San Francisco:
Jossey-Bass.
Ruthven-Stuart, R. (2004, November). Class
Evaluations; how can they improve education?
Paper presented at the annual meeting of
the Japan Association of Language Teachers(JALT), Nara.

Summer 2005: Volume 13, Issue 2

On CUE

Student ratings: Meaningful
voices or meaningless ritual?

Feature Article: BurdenPeter Burden
Okayama Shoka University
arousal of interest, ease of understanding,teacher’s speaking style, and appropriate useof blackboards or audio-visual equipment(“Grading,” 1997). Recently in Japanesehigher education, there has been a trend
toward market-oriented principles of performance indicators and quality assurance(MEXT, 2001). A government shift toward
decentralization, marked by economic-centered policies and sensitivity toward markets(Tsurata, 2003; Yamada, 2001), coupledwith a belief in teacher accountability to thestakeholders in education has led to an era ofretrenchment in Japan: The business model
is being applied more readily to institutionsof higher education.
Arguably, teacher assessment aimed at
fostering learning has given way to greater
emphasis on summative student evaluation. Implicit in such evaluation is the assumption that students complete the forms
anonymously, honestly, and (importantly)
willingly. However, there have been few attempts to ask students how conscientiously
they respond to the questions, whether theytake the whole process seriously, and what
purposes they think are served by evaluations. As Spencer and Schmelkin (2002, p.
398) note, "since students are unsure whether
their opinions matter, or to what purpose theratings are put, they may not pay attention tothem in contrast to stated desire to provide
feedback." Are students mindful to complete

学生による授業評価の手法は、その信頼性、有効性、有用性が教育関係者の間で疑問視されているにも拘わらず、日本の大学において定着してきた。本論は、大学で一学期を終えたばかりの学生に対して行われる制限応答形式のアンケート調査結果が、教師の将来をも左右しかねない決定を下すための適切なクライテリアとみなせるか否かを問うものである。203名の学生に対して聞き取り調査を実施
し、評価アンケートへの回答が意味のある行為だと考えるか、評価の低かった教師はどう扱われるべきか、授業評価の目的は何だと考えるか、そして何を評価の対象とするべきかについて質問を行なった。
その結果、一部の学生からは、アンケート調査が行われるタイミングのために授業期間中に評価結果のフィードバックが出来ないことへの不満が表明された。学生は授業の評価を意味のあるものと考えているが、その目的については十分に説明を受けておらず、そのことが授業評価の実効性に影響を与える可能性がある

Introduction
End of semester evaluation of English conversation classes is ubiquitous in Japanesetertiary education, and many teachers seemto accept the use of ratings unquestioningly.
Evaluation or ratings forms usually ask students to rate speciﬁc teacher characteristics
such as teacher enthusiasm, knowledge ofthe subject, classes starting and ﬁnishing ontime, along with a ﬁnal global characteristicof instructor’s overall effectiveness. Ratings
are typically recorded on a 1 (poor) to 5 (verygood) scale.
Prestigious Keio University was one of
the first institutions to introduce student
evaluation in 1990. Kansai University established a similar system in 1994, asking
students to respond to questions regarding
16

Summer 2005: Volume 13, Issue 2

On CUE

compounded by questionnaire fatigue—theinsistence on the use of standardized ratings
which requires that students ﬁll out the same
form on every instructor (Abbot et al., 1990).
This can lead to a "coat-tails effect" (Block,1998a, p. 418) where a respondent assigns
the same number for all questionnaire items
out of inertia or a lack of interest, which has
potentially disastrous consequences if results
are norm referenced.
Little is known about the actual processes students follow when responding torating forms. Do students respond to items
by comparing the instructor's performanceto that of other instructors or to some ideal?
What motivates learners to respond to thequestions?

Research questions
How do learners themselves rate the end ofsemester rating process? If students question
the purpose and see it as a meaningless ritual,
resulting scores may be adversely effected.
To gain insight on these questions in order toconsider what teachers and institutions mightwant to do differently, the following issues
demand closer scrutiny:
1) Do learners think evaluations are useful?
Why or why not?
2) How should evaluations be subsequently
used?
3) What should happen to teachers who
receive a low evaluation?
4) Do learners think teachers care about
evaluations?
5) What are the important characteristics
of effective teachers?

MethodI administered an ofﬁcial student evaluation form in the 12th week of a 15-weeksemester in six classes of ﬁrst-year students
at a National University in western Japan.
17

Feature Article: Burden

evaluations in a meaningful way, or are they
"simply discharging a boring chore?" (McKeachie, 1997, p.1223). Dunegan and Hrivnak(2003) put it succinctly when they note:“When mindless...information processing
is limited, attention to detail is reduced, andour cognitive representation of the context is
simpliﬁed. When mindless, we respond like
automatons, drawing on scripts and schemain the past to deﬁne the present” (p. 282).
This has potential implications in the lightof MEXT requirements that there should bea "focus on abilities of teaching staff" (2001)
with comments from students being a corecomponent of evaluation. To ensure consequential validity in summative evaluation,
students need to realize that their opinions
do matter. In order for students to feel connected with their comments and feedback tofaculty, there must be "tangible immediacy
to the results" (Spencer & Schmelkin, 2002,
p. 398). Yet many students are not taught in
subsequent classes by the same teacher, cannot see any "improvement" and thus may feelthat expressing their opinions is meaningless.
Validity is further compromised through suchlearner indifference. Many students have acynical attitude, which teachers themselves
may "inadvertently promote through haphazard or scornful administration" (Smith &Carney 1990, p. 1), and as the use of results isoften unclear to students, evaluation becomesa "perfunctory exercise of little impact" (p.
6), which further jeopardizes reliability. For
many teachers and students, it is a consumer
rating done after the fact (Braskamp & Ory,1994); it does not offer opportunities forteachers to make changes while the students
are still involved in the class. Teacher concern over fairness and usefulness of feedback
has led to claims that only 23% of faculty in
American universities made changes basedon student evaluations (Senior, 1999). This isSummer 2005: Volume 13, Issue 2

Feature Article: Burden

On CUE

Results

There were 40 students in each class, majoring in Medicine, Engineering, and Law.
A week after the ofﬁcial administration, I
gave a survey of 24 closed items (see Appendix A), and students were informed that
I hoped to gain some knowledge about theirviews on "ofﬁcial" student evaluation forms.
Students were asked to circle the most appropriate answer on a 7-point scale, with one
representing the lowest level of agreement
and seven representing the highest.
A total of 203 students completed the questionnaire; the quantitative data are shown aspercentages in Tables 1 through 5. While theresults reﬂect the views of students at only
one university in Western Japan, it is hopedthat the ﬁndings may be generalizable tosimilar contexts.

General impressions

Table 1 reﬂects students' general impressions of the evaluation process. The data
reveals that 78% of students feel that evaluation is a good thing to do, while 74% believethat the teachers do indeed use ratings andcomments to improve the class. However,
only 21% agree that students are clearly told
how evaluations will be used, while 42% feelthat the evaluation process could be better.
Smith and Carney (1990) similarly foundthat while students were uncertain of how
evaluation was used, they stated that theyevaluated their teachers seriously. Perhaps
this may help to dispel many teachers' fears
that "students too often use the power of their
pencils to get even with professors and rating
The proceduresystems may turn the evaluation of effective
Data from the closed item survey were teaching into a personality contest" (Kulik,
analyzed using SPSS v.11.0 for descriptive 2001, p. 10).
statistics and an acceptable Alpha reliability
Students know that one purpose is coursescore of .78 was obtained. For the purpose improvement, but they are uncertain whetherof analysis, the responses to 'Strongly agree' ratings are used to determine salary, promo(7), 'Agree' (6), and 'Slightly agree' (5) tion, or teacher retention. Such information
are combined to create an overall score of needs to be shared so they are aware of theagreement with the question. Similarly, the impact of their ratings. This lack of clarity
sum of responses to 'Strongly disagree' (1), results from the haphazard way evaluation'Disagree' (2) and 'Slightly disagree' (3) were is often administered. Teachers may adcalculated to gain a measure of disagreement. minister evaluations themselves unaware
These aggregated data are reported in the of, or perhaps not in a position to explain,subsequent analysis of ﬁndings.
the purpose or how
they will be used.
Table 1. Students’ General Impressions (n = 203, responses as a
Evaluation becomes
percentage)a "ritual that prinQuestions 1-4
1
2
3
4
5
6
7 MeanSD
cipals and teachers
1. Evaluation of teachers is a good1
1
6
15 30 31 15
5.31
1.246
thing to do.
engage in because
2. Teachers use students’ evaluations
it is expected—not
1
3
7
16 38 24 12
5.06
1.237
to improve their classes
because they value
3. Students are clearly told how
15 18 20 26 12
5
4
3.36
1.618
evaluations will be used.
it" (Good & Mul4. The evaluation process could beryan, 1990, p. 201),35
14 36 22 10 10
4.41
1.44
better.
leading students to18

Summer 2005: Volume 13, Issue 2

On CUE

teaching" (p. 23).
Instructional support is crucial in the faceof negative ratings Without it, faculty who
have prepared long and hard have to reconcile
certain knowledge of effort expended againstlack of success. If the situation persists overtime, these faculty may develop negative
attitudes towards students and student evaluations, leading to "professorial melancholia"(Theall & Franklin, 2001, p. 47), including
hostility towards students and administrators,
arrogance, alienation and grade abuse
Consequences of evaluation
What should happen to teachers who con- against students. So that evaluation is notsistently receive poor evaluations? Table 2 punitive, resources for improvement andreveals that 36% of students feel that they support of teaching and teachers must bepart of a complete
system and cannot
Table 2. The Consequence of Poor Evaluations
be omitted. Support
Question 5
1
2
3
4
5
6
7 MeanSD
for teaching froma. The teacher should lose his job.
12 10 18 23 13 14
9
3.95
1.8
administrators andb. The teacher should get a cut in17
8
13 18 12 17 14
4.06
2.027
salary.
senior teachers is rec. The teacher should get “effective
quiredif the evalua323
12 20 20 39
5.56
1.549
teacher” training.
tion process is to bed. The teacher should not be allowed
7
3
15
15
19
23
17
4.72
1.746
perceived as useful
to do the class.
and non-threatening. Anything lessshould lose their job, while 43% feel that results in polarized views about the purposessuch teachers should get a reduction in salary. of evaluation, which leads to anxiety, resistSeventy-nine percent agree that effective ance, and hostility.
teacher training should be introduced. This
Fifty-nine percent of participants believe
has implications for definitions of what that a teacher receiving poor evaluations
an effective teacher does and who sets the should no longer teach the class, presumcriteria. Brown and McIntyre (1993) note ably if he or she cannot change the teachingthat different groups have different criteria approach to suit the students. Therefore, toby which they make their judgments about increase validity, student awareness of theteachers, including personal achievements or importance of their input needs to be raised.
characteristics, such as warmth, politeness, Studies suggest that students wanted theor sense of humor. Brown and McIntyre note results made public, believing that not onlythat there is a "...plethora of criteria used to teachers but also students have the right tojudge teaching and teachers. All are salient to view them for future class selection (Spencer
some group's concerns for what, in their eyes & Schmelkin, 2002).
constitutes 'good teaching'; but they are ofvirtually no help in our efforts to understand19

Feature Article: Burden

question why they use the same form in every
subject and leaving them with a feeling ofdecreasing impact each time the evaluation"ritual" comes round. Thus, the use by some
unknown administration for unknown purposes may bias the results as the literaturesuggests that students tend to be lenient if
they think data will be used by someone
other than the instructor (Centra, 1979) for
non-diagnostic purposes.

Summer 2005: Volume 13, Issue 2

On CUE

Table 3. Student Opinions

8% agree that evaluations are used in
6. Teachers don’t care about students’teacher retention.
9
14 18 24 24 10
2
3.74
1.51
evaluations of teaching.
As noted earlier, if
7. I think that teachers get friendlier as
41 20 19 11
5
3
2
2.33
1.491
students know that
evaluation day comes near.
evaluations are used
8. When I’m evaluating, I’m worried
9
7
3
2
2.14
1.567
the teacher will somehow recognize 55 13 11
in personnel decimy handwriting.
sions, they are likely
to be more generous
in their appraisals, which has obvious imConﬁdence in the evaluation
plications for consequential validity of theratings. Students are not aware of the hidden
process
Thirty six percent of students feel that agenda in that they lack information about ateachers do not care about the results of the crucial purpose of evaluation. i.e., retention
evaluations. There is a need for tangible or non-retention of teachers. One impression
immediacy in evaluation as in many cases gained from the learners’ answers is thatstudents will not have that teacher again. they lack knowledge about the administraTherefore, expressing opinions does not in tive procedures; students seemingly believe
the long run beneﬁt that particular student forms are just collected from boxes aroundanyway. Only 10% of students feel teach- campus and simply handed to the teacher.
ers get friendlier in the lessons approaching Teachers are then presumably free to use or
evaluation. This may mean that students disregard as they see ﬁt.
It is important to remember the cyclical nafeel teachers are indifferent to the evaluation process and that, again, the ritualistic ture of evaluation, and in order to "close themeaningless belief is emphasized. Similarly, loop" (Harvey, 2003, p. 4) there is a need to(only) 12 % feel that teachers will be able to provide feedback of outcomes to students via
recognize handwriting which may bias rat- postings on noticeboards, e-mail or Intranet.
ings that are collected often on the same day As well as providing data on the purpose,
how data will be used and disseminated,
as summative, end of semester tests.
publication of feedback provides consultation to initiate action in response to students'
What are the evaluations used for?

Feature Article: Burden

Questions 6-8

1

2

3

4

5

6

7

Mean

SD

Table 4 reveals
Table 4. Student Beliefs about Evaluation Usethat 71% of students
Question 9
1
2
3
4
5
6
feel that teachers
a. I think they are used to make39 16 16 17
92
use evaluations todecisions about teachers’ salary.
improve teaching b. I think they are used to make35 14 17 15 12
5decisions about promotion.
while only 12% feelthey are used to make coursethat evaluations are c. I think
3
3
5
15 28 22
improvements.
used in determining d. I think they are used to improve
34
6
17 25 24
salary; 20% believe
teaching.
they are used to make e. I think they are used to ﬁre 48 14 14 16 4 1
teachers.
decisions about promotion. Similarly,
20

7

Mean

SD

1

2.5

1.527

3

2.8

1.721

23

5.18

1.532

22

5.16

1.514

3

2.27

1.545

Summer 2005: Volume 13, Issue 2

On CUE

concerns, and it shows to students that theircomments are valued.

21

Feature Article: Burdenof human motivation, as well as pedagogicalskills necessary to help students understandthe curriculum.
What should be evaluated?
However, 77% of students thought that
While Simmons’ (1996) claim that students teachers should be evaluated on subject
may judge a teacher as not "aesthetically ac- knowledge, or as Richards (2001) calls it
ceptable" and are "rendered less capable" "content knowledge" (p. 209). This is justof teaching based on the teacher's physical one knowledge base, teachers also needing
appearance have been rejected in this study, practical knowledge, a repertoire of class39% of students do feel that evaluating the room techniques, and pedagogical knowlteacher according to how amusing they are edge including the "ability to restructure
is valid. In addition, 66% feel they should content knowledge for teaching purposes andevaluate teachers by their friendliness. Fifty to plan, adapt and improvise" (p.209). Arguseven perably, therec e n t o f Table 5. How Students Think Teachers Should be Evaluated should be
students
discussion
Question 10
1
2
3
4
5
6
7 Mean SD
rated in- a. I think teachers should bebeyond simevaluated by their personal 61 18 11
5
3
1
2
1.77
1.243
terest inplistic manappearance.
the textifestations
b.I think teachers should bebook asof knowlevaluated by how much they 43 21 17
9
6
2
1
2.25
1.438
use
AV
materials.
important.
edge, as tothink teachers should beThis may c. Ievaluated
how differ3.95
1.797
by how amusing 12 12 14 22 19 10 10have hugeent kinds ofthey are.
i m p l i c a - d. I think teachers should beknowledge,
evaluated by how much 18 18 16 23
9
3
3
2.9
1.639
tions forthe corehomework they give.
part-time e. I think teachers should becompoevaluated by how interesting 879
19 23 21 13
4.59
1.737
teachers
nents, canthe textbook is.
who ofbenefit thef. I think teachers should beten do notlearning
941.753
evaluated by how punctually 11 10 17 22 18 12
they start and ﬁnish class.
chooseexperience.
g.
I
think
teachers
shouldbetheir own
It needs to3
5
17 27 27 13
4.85
1.634
evaluated by how friendly 3
texts andbe rememthey are.
m a y b e h. I think teachers should bebered that
3
1
3
7
21
31
35
5.73
1.392
evaluated
by
how
much
they
stuck withstudent
know about the subject.
materials
evaluation
not suited to their teaching approach. Using is more a measure of students' transient satsimilar high inference items in end of semes- isfaction than of the teaching and learning.
ter evaluation will lead to an interpretation Student evaluation, while important, shouldof competent teachers as only needing good never be overemphasized at the expense ofcommunication skills. Teaching involves many other aspects of the communicative
more than effective communication as it English language program which must alsoentails the application of principles of hu- be evaluated.
man cognitive development, understandingSummer 2005: Volume 13, Issue 2

On CUE

Feature Article: Burden

Implications for teachingvalued, and while it is difﬁcult for teachers
to be the sole agents of chance they can draw
attention to viable assessment alternatives.
Facilitating dialogue between instructor andstudents can explain products and processes
including how learning will be achieved aswell as outcomes and gives learners opportunities to treat their own teaching as topic
of inquiry. As questionnaires actually prompt
the kind of answers students can give as thequestions are from a teacher's or administrator's point of view, Brown (1995) suggests
using student representative meetings. Thesemeetings enable administrators to ﬁnd out
what is on students’ minds and to informallyassess students’ expectations of the teaching
and learning processes.
Teachers should encourage students totake ownership of their own answers andcomments through dialogue as, without thisquestion, many students make unrealistic
comments based on unrealistic perceptions.
In general, students may complain about
the quality of the lecture but rarely consider
whether it was an appropriate method to use.
Braskamp, Brandenburg and Ory (1984) suggest collecting student opinions through amock letter of recommendation. In the formof a one-page letter, students comment about
speciﬁc components of the course as wellas the course as a whole. The letter should
conclude with a recommendation for others
to enroll or avoid the course.
Many universities write their own evaluations which are then administered across the
subject range regardless of subject content,
teaching approach, methods, and (often
overlooked) class size. As each subject areateacher has his or her own conceptions ofteaching which vary when fronting large
lectures of three hundred students or duringpersonalized seminars of just a few students,
this means that comparatively evaluating stu-

Teachers and learners both need to feelvalued. Teaching is important and so moretime should be spent analyzing and reﬂecting on it (Braskamp & Ory, 1994). One-shot
end of semester ratings devalue the process
and demean the student input. While 76% of
students think that evaluating is useful, only
21% say they are clearly told how evaluations are used and 42% of students think theprocess could be better. Students appear tobe going through a ritual while 36% thinkthat teachers do not care about evaluation
results. It is important in the teacher/student relationship that teachers are seen asa "real person" (p. 157) who makes errors,
but who takes comments on board. Teacher
scepticism needs to be alleviated through anevaluation which does not oversimplify theconditions required for language learning
by reducing them to a set of discrete points
because the "whole is more than the sum of
the parts." Evaluation should be classroom
based, bottom up and emic— based on theframework used by language learners to
describe interpret and evaluate their language learning experiences (Block, 1998b).
Teachers need to tell students of changes
made due to constructive feedback, or carry
feedback over from one semester to thenext, announcing at the beginning of a new
course that they are trying a new approachbased on comments of previous students.
Acknowledgment reinforces the importanceof ratings and comments. Amongst English
teachers, administrators and students, there
needs to be a consensus about what is important in ELT, what will be evaluated, whowill contribute, and what criteria will be are
all important steps in good practice. The
evaluation process often becomes a ritualthat administrators and teachers engage inbecause it is expected—not because it is22Summer 2005: Volume 13, Issue 2

On CUE

Brown, J. (1995). The elements of language
curriculum. Boston: Heinle and Heinle.
Brown, S., & McIntyre, D. (1993). Making
sense of teaching. Buckingham: Open
University Press.
Centra, J. (1979). Determining faculty
effectiveness. San Francisco: Jossey-Bass.
Dunegan, K., & Hrivnak, M. (2003).
Characteristics of mindless teachingevaluations and the moderating effects
of image compatibility. Journal ofManagement Education, 27(3), 280-303.
Good, T., & Mulryan, C. (1990). Teacher
ratings: A call for teacher control and selfevaluation. In J. Millman & L. DarlingHammond (Eds.), The new handbook ofteacher evaluation (pp.191-215). Newbury
Park, Ca.: Corwin Press.
Grading the teachers: More schools allowing
student evaluation of classes. (1997,
November 21). Trends in Japan, pp.1-4.
Harvey, L. (2003). Student feedback. Quality
in Higher Education, 9(1), 3-20.
Kulik, J. (2001). Student ratings: Validity,
utility, and controversy. In M. Theall, P.
Abrami, & L. Mets (Eds.), The student
ratings debate: Are they valid? How can webest use them? (pp. 9-25). San Francisco:
Jossey-Bass.
McKeachie, W. (1997). Student ratings: The
validity of use. American Psychologist,
52(11), 1218-1225.
MEXT (2001). Educational reform plan forthe 21st Century, archived at http://www.
mext.go.jp/english/topics/21plan/010301.
htm
Richards, J. (2001). Curriculum development
in language teaching. Cambridge:
Cambridge University Press.
Senior, B. (1999). Student teachingevaluations: Options and concerns. ASC
proceedings of the 35th Annual Conference.
California Polytechnic State University-San Luis, Obispo,California. April 7-10,
1999 pp. 251-260.
Simmons, T. (1996). Student evaluation ofReferences

Abbot, R., Wulff, D., Nyquist, J., Ropp, V.,& Hess, C. (1990). Satisfaction with theprocess of collecting student opinions
about instruction: The student perspective.
Journal of Educational Psychology, 82(2),201-206.
Block, D. (1998a). Exploring interpretations ofquestionnaire items. System, 26, 403-425.
Block, D. (1998b). Tales of a language learner.
Language Teaching Research, 2(2), 148172.
Braskamp, L., & Ory, J. (1994). Assessing
faculty work. San Francisco: Jossey-Bass.
Braskamp, L., Brandenburg, D., & Ory, J.
(1984). Evaluating teaching effectiveness:
A practical guide. Thousand Oaks: Sage.
23

Feature Article: Burden

dents according to some externally imposed
norm is both complex and of dubious value
if one considers all the different learning
tasks that different subject-specialists use.
If evaluations are to be rigorous and credible they must acknowledge the essential
and substantive aspects of EFL teaching,rather than just the most common attributesof teaching and learning. Arguably, relianceon student ratings reinforces a conservative
pedagogy whereby there is notion that an
ideal teacher exists and that teachers canimprove by changing their behavior to moreclosely match the students' ideal, which isoften socio-historically determined based on
previous learning experiences. If the use of
ratings is seen to reinforce speciﬁc facultyteaching behaviors, their use may constrict
teaching styles rather than encouraging adiversity of classroom strategies. Students
would be more willing to rate carefully items
on a long diagnostic form if they knew thatlong forms were used only when the instructor is working to improve the course. Thismay help alleviate incidental fears that theiropinion counts for little.

Summer 2005: Volume 13, Issue 2

Feature Article: Burden

On CUE

teachers: Professional practice or punitive
policy? JALT Testing and Evaluation SIG
Newsletter, 1(1), 12-16.
Smith, M., & Carney, R. (1990, April).
Students’ perceptions of the teachingevaluation process. Paper presented
at the annual meeting of the American
Educational Research Association, Boston.
Spencer, K., & Schmelkin, L. (2002). Student
perspectives on teaching and its evaluation.
Assessment and Evaluation in Higher
Education, 27(5), 397-409.
Theall, M., & Franklin, J. (2001). Looking
for bias in all the wrong places: A search
for truth or a witch hunt in student ratingsof instruction. In M. Theall,

P. Abrami, & L. Mets (Eds.), The student
ratings debate: Are they valid? How can webest use them? (pp. 45-56). San Francisco:
Jossey-Bass.
Tsurata, Y. (2003). Globalisation and Japanesehigher education. In R. Goodman, & D.
Phillips (Eds.), Can the Japanese change
their education system? (pp. 119-151).
Oxford: Symposium Books.
Yamada, R. (2001). University reform in the
post-massiﬁcation era in Japan: Analysisof government education policy for the 21st
century. Higher Education Policy, 14, 277291.

Appendix
This is a survey about the end of semester evaluations you complete. I would like to know your opinions
on student evaluation. Please do not write your name. Please read the questions and tell me what you think
about the teacher evaluation process at University. This information will be used as part of my Doctoral
studies at Exeter University in Britain. There are no right or wrong answers. Your answers will remaincompletely conﬁdential. Thank you!
Peter Burden.
Please read the questions and put a circle around one number:
Strongly agree 7 6 5 4 3 2 1 Strongly disagree
A: Generally speaking:1) The evaluation of teachers is a useful thing to do
Strongly agree 7

6

5

4

3

2

1 Strongly disagree2) Teachers use students' evaluations to improve their classes
Strongly agree 7 6 5 4 3 2 1 Strongly disagree
3) Students are clearly told how the evaluations will be used
Strongly agree 7 6 5 4 3 2 1 Strongly disagree
4) The evaluation process could be better
Strongly agree 7 6 5

4

3

2

1 Strongly disagree5) If the teacher consistently gets poor evaluations from students, the teacher shoulda) lose his or her job
Strongly agree 7

6

5

4

3

2

1 Strongly disagreeb) get a cut in salary
Strongly agree 7

6

5

4

3

2

1 Strongly disagreec) get effective teacher training
Strongly agree 7

6

5

4

3

2

1 Strongly disagreed) not be allowed to do the class
Strongly agree 7

6

5

4

3

2

1 Strongly disagree24Summer 2005: Volume 13, Issue 2

On CUE6) teachers don't care about students' evaluations of teaching
Strongly agree 7 6 5 4 3 2 1 Strongly disagree
B: About you:7) I think that teachers get friendlier as evaluation day comes near
Strongly agree 7 6 5 4 3 2 1 Strongly disagree
8) When I'm evaluating, I'm worried the teacher will somehow recognize my handwriting
Strongly agree 7 6 5 4 3 2 1 Strongly disagreea) Make decisions about teachers salary
Strongly agree 7 6 5 4

32

1 Strongly disagreeb) Make decisions about promotion
Strongly agree 7 6

5

4

3

2

1 Strongly disagreec) Make course improvements
Strongly agree 7

6

5

4

3

2

1 Strongly disagreed) Improve teaching style
Strongly agree 7

6

5

4

3

2

1 Strongly disagreee) Fire teachers
Strongly agree 7

6

5

4

3

2

1 Strongly disagree10) I think teachers should be evaluated bya) Their personal appearance
Strongly agree 7

6

5

4

3

2

1 Strongly disagreeb) How much they use AV materials
Strongly agree 7 6

5

4

3

2

1 Strongly disagreec) How funny they are
Strongly agree 7

6

5

4

3

2

1 Strongly disagreed) How much homework they give
Strongly agree 7 6

5

4

3

2

1 Strongly disagreee) How interesting the textbook isStrongly agree 7 6

5

4

3

2

1 Strongly disagreef) How punctual they are starting and ﬁnishing class
Strongly agree 7 6 5 4 3 2 1 Strongly disagreeg) How friendly they are
Strongly agree 7

6

5

4

3

2

1 Strongly disagreeh) How much they know about the subject
Strongly agree 7 6 5 4

32

1 Strongly disagree25

Feature Article: Burden9) I think that the student evaluations are used toSummer 2005: Volume 13, Issue 2

On CUEThe Structure of English and Japanese
Research Articles: A Contrastive Study

Feature Article: Kobayashi

Kaoru Kobayashi
Tokyo University of Agriculture
Background

これまでの日英比較言語学では英語と日本語の解説文（expository writing）および論説文（
argumentative writing）において論旨の展開の比較対照研究が行われている（H. Kobayashi,
1984, Oi and Kamimura 1997, Kubota 1996,
1997, 1998）。しかしその対象を研究論文としたものは少ない。本稿では応用言語学分野
における実験論文（日英各３０本）の総体的構成及び各部の構成をSwales（1990）のGenre
Analysisをもとに比較した。その結果、全体的な構成は概ね同じと考えられるものの、各部のmoveを単位とした構成に違いがみられた。またそれぞれのmoveを示唆するlexical signalについてもその使用に違いがみられた。

The structure of Japanese research articles
in the social science ﬁeld was imported from
Western culture in the late 19th century. In
Japan, social science studies, including applied linguistics, began by assimilating what
had already been established in Westerncountries, for there were no such studies inJapan (Kamishima, 1975). When there is anunequal power relationship, there tends tobe a signiﬁcant inﬂuence of the stronger language on the weaker language, in this case,English on Japanese. It has been pointed
out that the spread of English and AngloAmerican culture has imposed Aristotelian
logical development upon the languages ofthe subordinate culture (Kachru, B., 1983;
Kachru, Y., 1999). The present study was
carried out in order to see the generic inﬂuence of English research articles on Japanese research articles in the ﬁeld of applied
linguistics. The objective was to answer thefollowing questions:Introduction
The role of contrastive rhetoric is signiﬁcant in teaching second language writing
because it identiﬁes systematic differencesin discourse between the two languages.
The present study attempts to clarify the
structural difference as well as the differencein writing styles between English and Japanese research articles in the ﬁeld of applied
linguistics. The study was carried out basedon Swales’ framework of moves analysis(1990, pp. 93-201). ‘Move,’ according to
Nwogu (1991) is “…a text segment madeup of a bundle of linguistic features (lexical
meanings, prepositional meanings, illocutionary forces, etc.) which give the segmenta uniform orientation and signal the content
of discourse in it” (p. 114). The studies ofgenre analysis have had a great inﬂuence on
the teaching of academic writing.

1. How does the structure differ between
English and Japanese applied linguistics research articles as a whole?
2. How does the move structure of eachsection differ between the two sets ofarticles?
3. Are there any differences in the use of
lexical signals?

26

Summer 2005: Volume 13, Issue 2

On CUE

Methodlearning. Nihongo Kyoiku is considered anequivalent of TESOL Quarterly in the ﬁeldof Japanese language teaching. Although
Nihongo Kyoiku Ronshu is a university bulletin, it is widely read, for the University ofTsukuba is considered a leading school in
the ﬁeld of education in Japan.

Data
A corpus of 60 applied linguistics experimental research articles (30 in English and
30 in Japanese) that closely conform to theconventional Introduction-Methods-ResultsDiscussion (I-M-R-D) pattern was selectedfrom journals in the ﬁeld of language teaching and learning published from 1995 to
2000. The difﬁculty in selecting the data
was that while some articles started with theintroduction section, followed by the section
on background information (hereafter background section following Holmes, 1997),
such as previous studies or basic concepts,
others started directly with the backgroundsection. For the data of the present study,
both types were included as long as the article was written based on an experiment andcontained the methods, results, and discussion sections. The same criteria were applied
when selecting Japanese articles. Among the
30 English articles, 15 were obtained fromTESOL Quarterly and the other 15 were obtained from Studies in Second Language Acquisition. Among the 30 Japanese articles, 26
were obtained from Nihongo Kyoiku (Journalof Japanese Language Teaching) and theremaining 4 were obtained from Nihongo Kyoiku Ronshu (Journal of Japanese Language
Teaching). The English titles of the Japanesejournals are incidentally the same, but they
are published by different organizations. Nihongo Kyoiku is published by the Society for
Teaching Japanese as a Foreign Language.
Nihongo Kyoiku Ronshu is published by the
University of Tsukuba. These four journals
were selected for their popularity among
researchers in the respective ﬁelds. TESOL
Quarterly and Studies in Second Language
Acquisition are considered major in the ﬁeldof second or foreign language teaching andProcedure

Results
Structure of English & Japanese
articles as a whole
As a result of studying the heading andcontents of each section, the following sections were found to occur in both English andJapanese articles in applied linguistics: theabstract, introduction, background, method,results and discussion, and conclusion sec27

Feature Article: Kobayashi

For the ﬁrst research question, the present
research focused on the structure of the research article as a whole. The heading andcontents of each section of the articles werestudied to clarify the structure. For the second research question, moves analysis was
performed on the various sections. In thepresent study, a move structure model that included all the moves found was then createdfor each section for the contrastive study on
the predictable move patterns that followed.
The predictable move patterns were detectedusing Microsoft Excel spreadsheet software.
The number of predictable move patterns
will show the structural ﬁxedness in each
set of articles. For the third research question, the number of articles that contained aparticular lexical signal indicating a movewas counted. In the present study, the signalsthat occurred in more than 50% of the articles
that contained the move were singled out fora contrastive study.

Summer 2005: Volume 13, Issue 2

Feature Article: Kobayashi

On CUE

fore or after another move. Since the models
contain all the moves that occurred in both
English and Japanese articles, not all the
moves in the following ﬁgures were foundin every article.

tions. The abstract was typographicallyconspicuous in the two sets of articles. The
introduction was usually not preceded by any
heading in either language. The contents ofthe section usually included one or more of
the following: introduction of the context,
the literature review, and the introductionof the research. The background section wasusually preceded by a heading that indicatedthat the section provides the backgroundinformation of the study. The heading was
simply “Background” in English or “Haikei”(Background) in Japanese, or any other content speciﬁc heading, for example, “Previous
Research,” or “Basic Assumptions.” Theother sections were recognized based on
their headings. Among the sections found,the abstract, background, method, results
and discussion sections were present in all
the articles, but the introduction section occurred in 23 (77%) English articles and 13(43%) Japanese articles, and the conclusion
section occurred in 26 (87%) English and 27(90%) Japanese articles. It can be concluded
that the introduction and conclusion sections
are not obligatory in this genre. Another conclusion is that English writers assign more
importance to the introduction section thando Japanese writers.
The background and conclusion sections,
neither of which is speciﬁed in the term ‘IM-R-D pattern,’ appear to be particular tothis genre. Holmes (1997) also found thebackground section in political science andsociology articles and explained that this isbecause there is not ‘an agreed theoreticalframework’ in social science studies.

Figure 1. Abstract Section
Move 1Introducing the study

Move 2

Describing the methodology

Move 3Presenting the ﬁndings

Move 4

Concluding the abstract

Figure 2. Introduction Section
Move 1

Specifying the topic

Move 2

Commenting on previous researchMove 3Introducing the present research

Figure 3. Background Section
Move 1Introducing the ﬁeld

Move 2Introducing the general topic

Move 3Introducing the key notion

Move 4Introducing the particular topic

Move 5

Reviewing previous research on theparticular topic

Move 6

Commenting on previous researchMove 7

Describing the present researchMove 8Limitation of the present researchMove 9Signiﬁcance of the present researchMove 10

Research design

Move 11Specifying methodology

Figure 4. Method Section

Move structure of each section
Figures 1 to 6 show the moves found ineach section. Moves labelled with a number
occurred numerically according to its label
and those labelled with a letter occurred be28

Move 1Deﬁning the Subjects

Move 2Clarifying the focus of the study

Move 3

Describing materials/instruments

Move 4

Describing data collection procedure,
including task and/or treatment

Move 5

Describing the data analysis

Move 6

Presenting the results

Move A

Introducing research question/hypothesis/
purposeMove B

Describing the research design

Move C

Presenting the key notion

Move DDeﬁning the data collection period

Summer 2005: Volume 13, Issue 2

On CUE

Figure 5. Results and Discussion Section
Move 1
Move 2

Reviewing the objectives

Move 3

Announcing the topic of a result

Move 4

Restating hypothesis or research question

Move 5

Announcing the result with a pointerMove 6

Describing the procedure for a particularanalysisMove 7

Restating the results

Move 8Highlighting the ﬁndings related to one issue

Move 9Presenting an unexpected outcome

Move 10Presenting an incidental ﬁnding

Move 11

Evaluating the hypothesis

Move 12Presenting positive support

Move 13

Presenting negative support

Move 14

Presenting a deduction

Move 15

Presenting an explanation

Move 16

Extending the ﬁndings

Move 17

Evaluating the research positively

Move 18

Presenting a possible counterclaim

Move 19

Evaluating the methodology

Move 20

Presenting a counter argument

Move 21

Presenting a pedagogical implication

Move 22

Presenting a theoretical implication

Move A

Providing background information

Move B

Describing the structure of the section

Move C

Reviewing previous researchMove D

Summarizing the earlier text

Move E

Pointing to limitations or caveats

Move F

Summarizing the results to be discussed

Move G

Presenting an implication for further research

Figure 6. Conclusion Section
Move 1

Summarizing the research activity/ﬁndings

Move 2

Evaluating the researchMove 3

Evaluating the method

Move 4Presenting a pedagogical implication

Move 5Presenting a theoretical implication

Move 6

Presenting an implication for further
researchMove 7

Presenting the writer’s subjective reﬂection
on the researchMove 8Providing background information

The model of the abstract section closely
29

Feature Article: Kobayashi

conforms to the one presented by Bhatia(1993, p. 78) as typical which is comprised
by Move 1: Introducing Purpose, Move 2:
Describing Methodology, Move 3: Summarizing Results, and Move 4: Presenting
Conclusion.
The moves found in the introduction section are in accordance with those in Swales’
CARS model comprised by Move 1: Establishing a Territory, Move 2: Establishing a
Niche, and Move 3: Occupying the Niche(1990, p. 141).
Some of the moves in the background section (Moves 2, 6, and 7) also occurred in the
introduction section. The fact that the skipping of the introduction section is acceptablesuggests that both the writers and readers
acknowledge that the contents of these two
sections are similar or that they overlap.
The model of the method section is quite
different from Nwogu’s (1997) model of thissection of medical research papers. Nwogu’s
model comprises three moves: Describing
Data-Collection Procedures, Describing
Experimental Procedures and Describing
Data-Analysis Procedures. The difference
could be due to the difference of the two
academic ﬁelds, medicine and applied linguistics.
The number of moves found in the results
and discussion section in the present study
was signiﬁcantly larger than those found inprevious studies on other ﬁelds (Hopkins &Dudley-Evans, 1988; Brett, 1994; DudleyEvans, 1994; Holmes, 1997; Posteguillo,
1999; Peacock, 2003). This could be because
while the previous studies analyzed only
the discussion section, the present study
analyzed the results and discussion section.
Another possible reason is that the information contained in the results and discussionsection in the applied linguistics research
articles is more varied.

Describing the research designSummer 2005: Volume 13, Issue 2

On CUE

Feature Article: Kobayashi

The moves found in the conclusion section are similar to those found in the results
and discussion section except for Move 7. It
has been shown that this section was foundin 26 (87%) of the English and 27 (90%) of
the Japanese articles. This suggests that the
majority of both English and Japanese writers feel the necessity of placing the moves
that could occur in the results and discussionsection in the conclusion section.

Move patterns found in each section
Table 1 shows the patterns of moves thatwere detected in more than 20% of each setof articles.
The study on each section reveals that the
English article sections are more establishedthan the Japanese article sections, with a larger number of predictable move patterns.
There are two deductions that can be
drawn from the above results on move patterns: (i) the move structures of the article
sections are similar with common patterns

Table 1. Move Patterns
Move Patterns
Abstract

Number of Articles
English Japanese
25 (83%)28 (93%)16 (53%)25 (83%)15 (50%)13 (43%)21 (70%)20 (67%)7 (23%)
20 (67%)——

Introduction Move 1→ Move 2
Move 1→ Move 3
Move 1→ Move 2→ Move 3

16 (70%)—15 (65%)7 (54%)
6 (46%)
7 (54%)

BackgroundMove 4→ Move 7
Move 5→ Move 6
Move 4→ Move 5→ Move 6
Move 5→ Move 6→ Move 7
Move 4→ Move 5→ Move 6→ Move 7—
17 (57%)
10 (33%)15 (50%)10 (33%)

11 (37%)
11 (37%)—7 (23%)—

MethodMove 1→ Move 4→ Move 5
Move 1→ Move 3→ Move 4→ Move 5

17 (57%)6 (20%)

14 (47%)6 (20%)

Results andDiscussion

Move 5→ Move 7
Move 5→ Move 8
Move 6→ Move 7
Move 7(8)→ Move 14
Move 7(8)→ Move 15
Move 9→ Move 15
Move 5→ Move 8→ Move 7
Move 6→ Move 7→ Move 8

14 (47%)20 (67%)15 (50%)11 (37%)6 (20%)
14 (47%)—14 (47%)16 (53%)16 (53%)14 (47%)7 (23%)
7 (23%)—
7 (23%)—

Conclusion

Move 1→ Move 4
Move 1→ Move 6
Move 4→ Move 6

6 (23%)
10 (38%)6 (23%)

7 (26%)
6 (22%)—

Move 1→ Move 2
Move 2→ Move 3
Move 3→ Move 4
Move 1→ Move 2→ Move 3
Move 2→ Move 3→ Move 4
Move 1→ Move 2→ Move 3→ Move 4

Note 1: The denominator of percentage is the number of articles that contained thesection. The introduction was found in 23 English and 13 Japanese articles.
The conclusion section was found in 26 English and 27 Japanese articles.
Note 2: — indicates that the percentage was less than 20.

30

Summer 2005: Volume 13, Issue 2

On CUE

in the occurrence of moves in the two setsof data; (ii) the move structures of English
article sections are more established, withmore patterns of move occurrence than theJapanese articles.

found that the move structure of the English
research article sections is more established,with more predictable move patterns. The
extensive use of lexical signals found in theJapanese articles may suggest that Japanese

Table 2. English and Japanese Lexical Signals in Research Article Sections
NumberMove
Lexical Signalof Articles
E Move 1 (27)
E Move 3 (29)
J Move 3 (28)this study/the present study/this empirical study 16 (59%)
results
17 (63%)kekka/chousakekka (result/research result)
25 (89%)Introduction

J Move 3 (28)

honkou (the present article)/honkenkyuu (the present research)

Background

J Move 6

Shikashi/Tokoroga (However/but)

5 (50%)E Move 5 (19)E Move 11 (15)
E Move G (18)

J Move 5 (28)
J Move 15 (20)shows
support (Noun, Verb)further research
futurefurther
shimesu (Verb) (to show)
kangaerareru (can be assumed)

12 (63%)9 (50%99 (50%)
9 (50%)
12 (67%)15 (54%)13 (65%)J Move 6 (17)kongo (future)

5 (64%)

Results and Discussion

Conclusion

7 (53%)

E: English
J: Japanese( ): the number of articles that contained the move(%): percentage of articles that contained the signal in the articles that contained the movewriters prefer to resort to the use of lexicalsignals to orient the readers through logicaldevelopment. Two reasons can be put forward to explain the less established structure of the Japanese articles. One is that the
English structure based on Aristotelian logic
is exogenous to the Japanese language; theother is that in Japanese education, writing isless emphasized than in the Western world.
Although Aristotelian logic seems to havebeen accepted in today’s Japanese culture
and language, this may not mean that it has
penetrated into the Japanese language so
deeply as to exclude the Japanese writing
tradition in which systematic categorization of logical development does not exist.
Secondly, the teaching of writing in Japan

Lexical signals
Table 2 shows the result of the study on
lexical signals observed in more than 50% of
the articles that contained the move.
The results show that the Japanese articles
made use of lexical signals in all the sections
except for the method section while the useof lexical signals in the English articles wasobserved only in the abstract and the results
and discussion sections. However, more extensive use of lexical signals was observed in
the English articles in these two sections.

Discussion
Despite the structural similarities foundin the English and Japanese articles, it was
31

Feature Article: Kobayashi

AbstractSummer 2005: Volume 13, Issue 2

Feature Article: Kobayashi

On CUE

tends to receive less emphasis than it does
in English-speaking countries. Actually, thetextbooks used in Japanese language education in Japanese elementary schools, juniorhigh schools and high schools focused onreading until 1999. A pedagogical implication is that when teaching English academic
writing to Japanese speaking students, successful outcomes can be expected if lexicalsignals are presented together with explicit
instruction on appropriate usage.
One limitation of this study might be thefact that the audience of the Japanese articles is limited to Japan, whereas the English
articles are written for an international audience.

social sciences: An investigation of thestructure of research article discussionsections in three disciplines. English forSpeciﬁc Purposes, 16, 321-377.
Hopkins, A., & Dudley-Evans, A. (1988). A
genre-based investigation of discussionsections in articles and dissertations.
English for Speciﬁc Purposes, 7, 113-22.
Kachru, B. (1983). Introduction: The other sideof English. In B. Kachru (Ed.), The othertongue (pp. 1-12). Oxford: Pergamon Press.
Kachru, Y. (1999). Culture, context andwriting. In E. Hinkel (Ed.), Culture insecond language teaching and learning (pp.
75-89). Cambridge: Cambridge University
Press.
Kamishima, J. (1975). Nihonjin no hassou.
[Japanese Ideas]. Tokyo: Kodansha.
Nwogu, K. N. (1991). Structure of science
popularizations: A genre-analysis approachto the schema of popularized medical texts.
English for Speciﬁc Purposes, 10, 111-123.
Nwogu, K. N. (1997). The medical research
paper: Structure and functions. English forSpeciﬁc Purposes, 16, 119-138.
Peacock, M. (2003). Communicative moves inthe discussion section of research articles.
System, 30, 479-497.
Posteguillo, S. (1999). The schematic structureof computer science research articles.
English for Speciﬁc Purposes, 18, 139-160.
Swales, J. (1990). Genre analysis: English
in academic and research settings.
Cambridge: Cambridge University Press.

Corpus
A total of 60 research articles were analyzed, 30 from each language. These articles
were taken from the following journals published between 1995 and 2000:
English articles
TESOL Quarterly
Studies in Second Language Acquisition
Japanese articles
Nihongo Kyoiku (Journal of Japanese
Language Teaching)
Nihongo Kyoiku Ronshu (Journal ofJapanese Language Teaching)

References

Bhatia, V. (1993). Analysing genre: Language
use in professional settings. London:
Longman.
Brett, P. (1994). A Genre analysis of the results
section of sociology articles. English forSpeciﬁc Purposes, 13, 47-59.
Dudley-Evans, T. (1994). Genre analysis: An
approach to text analysis for ESP. In M.
Coulthard (Ed.), Advances in written text
analysis (pp. 219-228). London: Routledge.
Holmes, R. (1997). Genre analysis, and the32Summer 2005: Volume 13, Issue 2

On CUE

Opinion & Perspective
Workaday Worlds: Teaching EFL
and ESOL

Gregory Strong
Aoyama Gakuin University

unfairly dismissed from their positions. For
example, on April 21st of this year, the Kyodo
News service reported a teachers’ union in the
Hyogo prefecture ﬁling an appeal against the
Amagasaki city government for cutting thewages of 11 foreign teaching assistants by
15 per cent while later raising those of othercivic employees by 1 per cent.
However, the inequitable two-tier employment system in Japan, in which full-timeteachers have bonuses, medical and pension
beneﬁts, and the part-time teachers havenone, is not unusual. A TESOL position
paper (2003) cited the National Center for
Education Statistics in tracking the shift fromfull-time positions to part-time, adjunct, andcontingent faculty in American universities
and colleges. The proportion of part-timeworkers rose from 33 per cent in 1987 toalmost 47 per cent in 1997. The increase
was particularly evident in community colleges (p.1).
Not only are part-time positions on therise elsewhere, but also there are few postsecondary institutions in which an EFL/ESOL specialist like myself can teach. The
prospects are most often limited to teaching
remedial classes and adjunct or shelteredcourses for foreign students. These are
growing in number but are mostly delivered
through language institutes afﬁliated with
universities, often on the same campus, butwith much poorer employment conditions.
33

Opinion & Perspective

In an opinion piece in On CUE earlier
this year John Burrell (2005) stated the caseagainst working in Japan, citing limited-term
contracts and declining opportunities due
to changing demographics. In this article
I argue that the large numbers of part-timecollege and university EFL positions in Japan and the higher rates of pay continue tocompare favourably with English language
teaching in English-speaking countries. To
support this view, a number of teachers who
had spent some time in Japan were interviewed by e-mail about their experiences ofreturning to work in their countries of origin.
Some of their comments are included here. I
also offer some ideas for ways of improving
the situation for teachers in Japan.
First, I would like to say that the relatively
positive conditions in Japan by no means
excuse the discrimination regarding the different contracts given to Japanese nationals
and foreign teachers at the same institutions.
Arudou (2003) has documented many universities on his website that openly practise thisabuse. Both full-time and part-time teachers
need to become well-informed about this issue, about their rights, and need to be aware
about the possibility of seeking union protection. Teachers also need to raise the issue
with their foreign and Japanese colleagues.
The teachers’ union, Tokyo Komu Kokyo
Ippan Roko Kumiai has been very active inaiding teachers who have been harassed orSummer 2005: Volume 13, Issue 2

Opinion & Perspective

On CUE

Teachers are hired one semester at a time,often with cancellation clauses in theircontracts if too few students enroll in a particular course. Eskey (1997) describes these
institutes as the “cash cows” of the university
system, drawing in new operating capital,
but discriminating against their personnel
who receive less pay, prestige and power
than those in the traditionally recognized
academic disciplines (p.24).
Six teachers contacted by e-mail about
their experiences of returning to work intheir countries of origin or moving to another
English-speaking country confirmed theabove situation. One teacher who taught inJapan for some 13 years, four of them fulltime, left Japan two years ago and moved to
Brisbane, Australia, where she teaches parttime in a university education department.
She reported that:
Most TESOL jobs in Australia are atinstitutes, some of which belong to universities, and their focus is on making
money. In most cases, an MA is needed
for a full-time job at a university institution. Other qualiﬁcations like a CELTA
instructors’ certiﬁcate or an IELTS examiner qualiﬁcation also help.
The same teacher also commented on theoverall employment picture:
There are more people than jobs in
TESOL in Australia. It is therefore
easier to get work in Japan if you are anative speaker of English.
This teacher, like the other teachers contacted, found it harder to ﬁnd part-time work
once she left Japan. A qualiﬁed teacher canmove to Japan and within a relatively shortperiod of time, obtain part-time university
work. For example, in the case of the English
Department at Aoyama Gakuin University,
we have an annual turn-over of 20 to 25 percent of our 48 part-time faculty members

as people leave Japan, take other work, ormove from Tokyo. And every February andMarch, no matter how carefully university
departments like ours prepare their teachingschedules, there are numerous last-minutejob vacancies.
The high rate of job turn-over and the total
number of English teaching positions available here continue to attract foreign teachers.
Jannuzi and Mulvey (2005) contrast the 89
universities and 69 colleges of education in
the UK with the 1,200 accredited colleges
and universities in Japan. Furthermore, inJapan part-time teachers are often paid overa twelve-month period though the teachingyear may consist of only 24 weeks.
Teachers who have spent much of theircareer here in Japan are often surprised by
the relative lack of opportunities in post-secondary education in their countries of origin.
Many teachers rely on getting teachingpositions on the basis of a second graduate
degree or they move into the public school
system.
For example, one of the teachers interviewed benefited from having two MA
degrees. He left Japan two years ago, planning to relocate to the US. He had 22 yearsof teaching experience in Japan, was a frequent contributor to textbooks, and had an
MA in English Linguistics as well as one in
Spanish:I sent out hundreds of resumes, from theEast coast to Hawaii, from Boston all
the way to Puerto Rico – I’m qualiﬁed
to teach Spanish and I would have liked
to have lived there – and I got very few
responses. People just didn’t bother toanswer. One response I got back fromGreensboro, the University of North
Carolina, was from an autonomous
language school on campus offering a
position at $28,000 per year, with very,34

Summer 2005: Volume 13, Issue 2

On CUEWork in universities is not badly
paid, in London at any rate, $60 to$70 per hour. Beneﬁts are few inpart-time work; usual with full-time.
Of course, we do have the National
Health Service which removes a lotof pressure compared with the U.S.
Another teacher who returned to England,in this case, after nearly three years on afull-time contract in Japan, also found an
entry to university teaching through summer
session courses. At the time, she had an MA,
more than 20 years of teaching as well as
experience in teacher-training, and materials
development and testing in Sweden, Canada,
China, and Japan. She noted that:
When I returned, I upgraded by doing a
distance Diploma TESOL with Trinity
College, London as I needed a more
recent UK qualiﬁcation to prepare forthe search. I got the job via summer
teaching as there are often one or two
positions that follow if you stay nearby
and keep in touch and watch for theadverts. Full-time jobs are few and far
between and they certainly don’t pay
as well as in Japan!...Some universities have overseas campuses which
can offer job opportunities such as the
University of Nottingham’s campus
in Ningbo, China. There are many
two-and-two-years-in-the-UK degree
programs in China.
A sixth and ﬁnal example: after teachingin Japan for 11 years, six years of that undera full-time term contract, a teacher returning
to California earlier this year unsuccessfully
applied for a number of college positions.
She is now considering the public school
system:You need a California teaching credential. It takes about a year of full-time
study, or two, part-time. I think it opens
35

Opinion & Perspective

very little offered in the way of healthbeneﬁts.
He found that another difference between
Japan and America is the much higher cost ofhealth care in the US. He learned that basic
coverage for himself and his wife would costthem about $900 per month. Finally, he wascontacted by one of his former professors at
his hometown university. Because he had
a second degree in Spanish, he obtained acontract position planning, implementing,and managing the Spanish language lab at
Fayetteville University for $45,000 per annum.
The situation encountered by one educator
and writer who spent 15 years in Japan and
who also has two MAs proved even worse.
Five years after moving to Ottawa in 2000,
this author of a well-received book aboutteaching English in Japan is still trying to
break into regular part-time college teaching:
Basically, in Japan, there are lots ofuniversities and so lots of jobs. Even
though the jobs in Japan are part-time,
you can build up enough hours to make aliving. In Ottawa, for example, there are
only two universities and one college,so that yields a total of 30-40 jobs. Jobs
are hard to ﬁnd and once someone gets
in, they stay there forever.
According to a fourth teacher, the part-timeteaching situation improves in bigger cities.
In addition to similar qualiﬁcations to thethird teacher, he was an IELTS examiner
and a past program director with 11 years
experience in Japan as a full-time teacher
on several term contracts. He returned to
London two years ago, and found plenty ofpart-time work, especially with summer andpre-sessional courses though again througha second graduate degree, in history. He
observed that:

Summer 2005: Volume 13, Issue 2

Opinion & Perspective

On CUEup a lot of opportunities because there
are many new schools opening.
Qualiﬁed public school teachers in the
U.S. and elsewhere enjoy job security, good
health and pension beneﬁts. According to the
Bureau of Labor Statistics (2004), mediansalaries in the U.S. ranged from $39,810
to $44,340. The variations reﬂect different
school districts, pay increments given for
graduate degrees and for teaching experience, and the local cost of living. However,when a teacher accepts a position in one district, prior teaching experience from another
district or any teaching experience abroad
is usually not counted. An experienced EFL
teacher would get credit for a graduate degree, but might have to start near the bottomof the salary scale. A further trade-off is thatwith a 40-hour week and a ten-month school
year, a public school teacher works many
more hours than do teachers at a Japanesecollege or university.
These teachers’ experiences illustratesome of the obstacles that educators facewhen resuming a career in the country theyleft. They also suggest the need for each of
us to engage in strategic career planning.
That means becoming active in teacher associations such as JALT or JACET, making
presentations and publishing in the ﬁeld. In
terms of advanced degrees, distance education makes it possible for teachers to undertake a second degree or to upgrade their
qualiﬁcations to a PhD while continuing towork in Japan. Each of us needs to constantly
re-assess our working life and our career andpersonal goals and whether or not living here
is a suitable place to pursue them.
Japan can be an excellent place to gain tertiary teaching experience in a cross-cultural
setting but for most EFL/ESOL teachers, as
elsewhere, it will not provide the securityof a permanent position. On the other hand,that elusive full-time job, which may not bethat secure in any case, is not essential toa career in teaching that can be rewarding
ﬁnancially and on other levels. If a teacher
wishes to pursue part-time textbook writing,editing work, journalism, acting, or to follow
other interests, then the ample holiday andvacation time are attractive.
The same TESOL position paper describedearlier called for increased institutional support for part-time teachers through access tomaterials, and opportunities for professional
development. In Japan, full-time teachers
should take more initiative in this area.
First of all, there should be more openness
and responsibility in dealing with teachers
seeking part-time work. One of the teachersinterviewed said, “In Japan, it’s often a matter of ‘Who you know’ in the school that getsyou hired.” Others commented on the openness of job competitions back home.
An additional area for improvement is thatthe full-time teachers, both Japanese andforeign, who are responsible for schedulingthe part-time teachers in their departments
should provide some employment security by recognizing seniority when offering
additional classes or cutting them from ateacher’s schedule. As well, before part-timeteachers are hired, they should be informedof the criteria for teacher evaluation andrenewal of a contract.
In terms of professional assistance anddevelopment for part-time teachers, fulltime teachers should provide new part-time
faculty with a campus and program orientations. They should try to establish study areas
and a teacher resource area with educational
texts, journals, and audio-visual materials
for classroom use, and ensure that their department provides locker space, mailboxes,
e-mail accounts, library access, and theopportunity to publish in the department or36

Summer 2005: Volume 13, Issue 2

On CUE

school journal. These measures would not
require much expense. In addition, full-time
faculty should work with part-time teachers
in developing curriculum and teacher guidelines for courses. Taken together, these actions would go a long way to supporting theefforts of part-time teachers and to improving
their working environment and the quality ofEnglish teaching in Japanese colleges anduniversities.

1&DID=3777&DOC=FILE.PDF
U.S. Dept. Of Labor, Bureau of Labor
Statistics, Occupation Handbook.
(2004, May 18). Teachers—Preschool,kindergarten, middle and secondary.”
Retrieved April 25, 2004, from http://
www.stats.bls.gove/uco/coos069.htm

Referencesa. The author wishes to thank the teachers
quoted in this article.

Notes:

b. To contact the Tokyo Komu Kokyo Ippan
Roko Kumiai, write or call the chairman,
Noboru Shida, 605 Takasago Samariyamantion, Oiwakecho 8-10, Hachiojishi,Tokyo, 192-0056/Ph: 03-5395-5255 Fax:
03-5395-5139.
c. The teachers interviewed for this article
mentioned several useful internet sites forjob seekers:
1) Academic and research jobs in England:
http://www.jobs.ac.uk
2) The Education sector of the Jobs Siteat the Guardian Newspaper: http://jobs.
guardian.co.uk/browse/education/index.
jsp3) A site that offers information aboutteaching jobs in the U.S. and links to related ones: http://www.cccregistry.com***

Gregory Strong teaches in the English
Department at Aoyama Gakuin University
where he is the coordinator for the Integrated
English Program. He has taught in colleges
and universities in China and Canada as wellas in Canadian public schools.

37

Opinion & Perspective

Arudou, D. (Dec. 2003). Blacklist ofJapanese universities. Retrieved April
26, 2005, from http://www.debito.org/
blacklist.html
Burrell, J. (2005). The future for university
English language instructors in Japan.
On CUE, 13(1), 22-25.
Eskey, D. (1997). The IEP as a nontraditional entity. In M. A. Christisonand F. Stoller (Eds.), A handbook for
language program administrators (pp.
21-30). Burlingame, CA: ALTA Books.
Jannuzi, C. & Mulvey, B. (2005). Japan’s
post-secondary educational system- A vast job market for EFL teachers?
ELT News. Retrieved April 25, 2005,
from http://www.eltnews.com/guides/
universities/universities2_1.shtml
Japan Today. (2005, April 21). Appeal
ﬁled over wage discrimination against
foreign teachers. Kyodo News Service.
Retrieved April 21, 2005, from www.
japantoday.com/e/?content=news&cat=
1&id=334888
Teachers of English to Speakers of Other
Languages. (2003). TESOL position
paper on equitable treatment for parttime, adjunct, and contingent faculty.
Retrieved October 4, 2004, from http://
www.tesol.org/s_tesol/bin.asp?CID=24

Summer 2005: Volume 13, Issue 2

On CUEThe Teacher as Missionary; Is This
Really Global Education?

Opinion & Perspective

Michael Guest
Miyazaki University Medical College
vocated” (p. 16) which he glosses from the
Universal Declaration of Human Rights. But
this declaration focuses upon generally-heldvalues such as respect and dignity of human
life, not on the narrower conﬁnes of partisan political positions. For Peaty though, it
seems to be a given that his values automatically equal “teaching for a better world” (p.
16). The argument thus runs along the following lines:
The Declaration of Human Rights seeks
to defend human dignity; my politicalviews also aim to defend human liberty;
therefore if I teach my political views,
I am doing precisely what the U.N.
declaration on human rights, with its
mandate from society, is doing; so how
can that be indoctrination?
This is seriously ﬂawed logic. Advocatesof all stripes believe that what they do is
aimed at maintaining and preserving humanrights and dignity in some form. Nevertheless, Peaty goes on to characterize his views
as non-mainstream, alternative, even radical.
But if this is true, how is it possible that hecan then claim that society accepts theseprinciples and goals? He seems to be talking
out of both sides of his mouth here. Peaty
also tries to hijack the Crick report (1998)
as support when in fact the report notes thatteaching only one side of an issue is actually
illegal in Britain (p. 59)!
Peaty further justiﬁes his use of the class-In a recent issue of The Language Teacher, Michael
Guest (2005) responded to an earlier article byDavid Peaty on global education (2004). In thisopinion piece for On CUE he expands upon someof his arguments.

As a supporter of both content-basedlearning and global education, I read David
Peaty’s (2004) article in The Language
Teacher—Global issues in EFL: Education or indoctrination? with interest. In thisarticle Peaty argued that advocacy in theclassroom was not only permissible, but
morally desirable. Unfortunately though,the article served to confuse me as it did
not address an advocacy of Global Education as an academic issue per se but rather amuch narrower focus, promoting so-called‘progressive’ positions on certain global
issues which Peaty appears to erroneously
conﬂate with global education. It seems thatfor some, advocating global education means
subscribing to a narrow set of ‘progressive’socio-political values. In a recent response tothat article (Guest, 2005), I addressed someof the contradictions in his article as well as
critiquing the ‘teacher as missionary’ rolethat he advocates. In this particular response
for On CUE I would like to focus upon somefurther implications of Peaty’s arguments infavour of classroom advocacy.
Peaty argues that teaching with a bias is
acceptable if “society accepts and respects
the principles, values and goals being ad38

Summer 2005: Volume 13, Issue 2

On CUE

Peaty (Guest, 2005), many beliefs and issues—such as the effects of consumerism
on the environment and the selﬁsh antics
of big business—are hardly “marginal” or
unknown themes in the mass media. In fact,
so media-saturated are many of these issues
and positions that they have become fodder
for parody in many hip comedy TV shows.
These allegedly progressive, marginalized,
radical views have in many ways becomerather antiquated, standard, default positions
readily and widely dispersed throughout oursociety. Ironically, it seems that Peaty andmany of his supporting sources are simply
regurgitating these already media-drenchedperspectives. It is therefore quite arguablethat it is Peaty and like-minded advocates
who are dispersing ‘myths’ regarding thenature of the media. In fact, if Peaty reallywants the voices of radical or marginalizedgroups to be heard, he should realize that thiswould not include so-called “progressive”
views alone but also those of Aryan supremacists, religious fanatics, conspiracy theorists,
unrepentant Stalinists and the like.
In my previous response to Peaty’s article
I also outlined how Peaty’s approach verymuch resembled the missionary attitude
towards education—that of enlightened
outsiders saving the deceived masses byoffering them the wisdom of great ‘truths.’
Interestingly, in many of the world’s mostrepressive regimes this approach is known
as re-education, and using education to persuade learners to healthy beliefs has long
been a euphemism for attempts at indoctrination. Making this scenario even more malodorous is the reality that for most of Peaty’s
readers, such a classroom setting will involve
Western teachers ‘illuminating’ large groupsof Japanese. Since many Western language
teachers cannot or do not read or listen to theJapanese media, why is there the assumption
39

Opinion & Perspective

room as a podium for espousing this type of
personal dogma, primarily because he seeks‘balance’ by bringing in his ‘marginal’ or‘radical’ perspectives. All of these, interestingly, correspond to a single ‘progressive’agenda, which do not reﬂect ‘mainstream’
views. Such mainstream views are ‘unfair’because they are the products of a mass
media which, in quoting Anderson (1996),
he conﬂates with the ‘positions’ of government and commerce. However, this notionof a singular, united, monolithic media andits liaisons with the realms of governmentand commerce is altogether too simple andhighly reductionistic.
Moreover, this notion simply ignores theactual plurality of the world around us infavour of using a pre-constructed image thatallows Peaty to engage in this dubious dialectic. For example, one can easily ﬁnd a varietyof viewpoints, including all of those advocated by Peaty, within ‘mainstream’ media.
For example, I have in front of me as I write,
today’s Daily Yomiuri newspaper, whichtends to follow a rather right-wing position
and is by any standard, ‘mainstream.’ And
yet, in this issue, there is an article (originallyprinted in another ‘mainstream’ newspaper,
the Chicago Tribune) which states that U.S.
agricultural subsidies to Africa have had theeffect of keeping African farmers poor byundercutting the competition from the developing world. Moreover, the following day’s
issue had an article reprinted from the L.A.
Times about multinational oil corporations
threatening a whale habitat in the Russian
Far East. And if one doesn’t like the alleged
right-wing slant of the Yomiuri, one can easily ﬁnd and read the more left-oriented Asahi
Shimbun. Yet Peaty would have us believethat such ‘progressive’ views are not at all
typical of such newspapers.
As I mentioned in my earlier response toSummer 2005: Volume 13, Issue 2

Opinion & Perspective

On CUE

that their Japanese students haven’t heard
these views before and therefore need to be‘balanced’ by them now?
The implicit assumption seems to be thatthe learners are basically dupes who havebeen brainwashed by the government, commerce and media, and that the classroom isthe place where they can be re-programmed.
The treatment of learners as if they haven’t
already been thinking independently on
issues or hold well-founded opinions (although the enlightened and knowledgeable
teacher apparently has) I ﬁnd to be a rather
frightening abuse of the teacher’s role. Unfortunately, Peaty’s endorsement of Strain’s(1991) questionable assumption that “Alleducation is constructed for the realization ofsocial values” (p. 16), indicates a predisposition to politicizing one’s classroom. Afterall, if the classroom is inherently a politicalfootball it behooves the teacher to be on thepolitical offensive. And furthermore, if one
views students as ideological tabula rasas
heretofore stained by the imprint of the ‘unfair myths’ of the enemy, the temptation to
overwrite such brainwashing with one’s own
advocacy must certainly be tempting.
Yet this advocacy is not about getting
people to think for themselves but rather
to accept a certain fundamental worldview.
For example, Peaty’s advocacy is based
upon his belief that our “present way of life
is unsustainable” and that “the future of theplanet is at stake” (p.16). These fundamental
claims are simply stated, not supported. A
similar resort to ‘given truths’ can be foundin Peaty’s advocacy of using the language
classroom for “challenging myths” (p.
17– my italics) where, as an example, readers are informed that they can show students
that the ‘real’ cause of hunger is not because
not enough food is grown. This alleged myth
(although it seems like a straw man myth

to me) regarding the cause of hunger is notchallenged—it is merely contradicted. The
proposition is simply not so; the truth has
been pre-determined.
Pedagogically, all this has the force of
putting the teacher at the center of the classroom as the conveyor of truths to be imparted
to impressionable learners, a pedagogy thatmost ‘progressive’ teachers eschewed longago. Although Peaty makes reference to both
Stradling (1989) and Higgins (1990) in orderto pay lip service to nurturing the skills ofcritical thinking, and quotes McLeod (1991)
and McIntyre (1996) on avoiding bias, it isevident that he wishes to use his classroom
in order to lead the learners to the ‘correct’answers. For example, “Critical thinking”
is advanced solely as a means to “subvert”
the “dominant or mainstream perspective”(p. 16) because this perspective is “clearly
unsustainable in the long run” (p. 16–myitalics). So much for any notion of using
critical thinking for impartial inquiry. Here
we are moving from the area of legitimate
educational inquiry into the realms of kneejerk anti-authoritarianism. On top of this, theassumption that such a critical approach tothe ‘mainstream’ would lead inevitably to
accepting certain ‘progressive’ perspectives
is rhetorical hubris of the highest order. Peaty
though does not want to characterize all of
this approach as indoctrination, apparently
because, again, for him ‘truths’ are apparently self-evident.
One of Peaty critiques in this regard is
aimed at Sargent (2004). Sargent’s call forbalance against turning academic global
education into global advocacy is misrepresented by Peaty saying that “teachers are
required to present all sides of an issue” (p.
17) and, because this is unwieldy, claimingthat the logical corollary of Sargent’s concerns is that “[teachers] should avoid issues
40Summer 2005: Volume 13, Issue 2

On CUE

entirely” (p. 17). But this argument is in facta red herring. ‘Balance’ does not necessarily
imply ‘every and all.’ A balanced meal does
not require all foods to be on the table. A balanced survey does not imply that all people
on the globe be polled.
But I would like to ask a further question.
Why is it that ‘issues’ have to be ‘presented’in the language-learning classroom at all? Isa presentation-practice-production module
the only classroom paradigm for dealing withissues? Moreover, while we may be educators, we are language educators ﬁrst andforemost. Our primary responsibility should
be to help learners understand and express
themselves in English. Therefore, our focus
should be upon aiding learners in using thelanguage of rhetoric and argument, and on
the forms and structure of a cohesive argument, rather than on the nature of its content.
Surely, the content of an argument is an areathat should belong to the language learner.
One should allow the learner to researchwhat is being said (and how) on certain issues
and then develop the language skills that help
him or her to formulate in English a cohesive
opinion based on this research.
As an example of this, a teacher can easily set up a research project or debate on
an issue by simply announcing a topic (asopposed to presenting it—and particularly
as opposed to presenting a ‘marginalized’view that the teacher personally believes
will teach learners truth about the ‘myths’
they have supposedly been spoon-fed thusfar). For example, the teacher might simply
introduce the topic without any expression
regarding their own beliefs or attitudes, thus
maintaining a genuine sense of neutrality.
Here is a topic:

to research and express yourself on this
topic, but I myself will not and need not
express any opinion on this topic.

References
Anderson, G.G. (1996). Global issues
in the university EFL classroom. The
Language Teacher, 20(11), 20-25.
Crick, B. (1998). Education for citizenshipand the teaching of democracy inschools: Final Report of the Advisory
Group on Citizenship. London:
Qualiﬁcations and Curriculum
Authority.
Guest, M. (2005). A response to David
Peaty. The Language Teacher, 29(5),11-12.
Higgins, M. (1990). Overcoming teacher
bias in the global issues language
classroom. The Language Teacher,14(5), 31.
McIntyre, D. (1996). Global issues in EFL:
Why and how. JALT Journal, 18(1),117-131.
McLeod, K. (1991). Human rights andmulticulturalism in Canadian schools.
In H. Starkey (Ed.), The Challenge of
Human Rights Education (pp. 176-189).

Japan resuming commercial whaling for or against? Research and justify yourarguments. I, as a teacher, will help you
understand or use the language you need
41

Opinion & Perspective

In fact, the list Peaty provides of Higgins’ (1990) critical thinking skills would
serve well here, that is if they were divestedof their pre-determined political agenda ofsubverting the ‘mainstream.’ This focus uponempowering the receptive and productive
language skills of learners should be therole of the teacher who is truly interested in
global education and open-minded enquiry.
Unfortunately, just about everything that
Peaty advocates in his article contradicts
this spirit and instead unwittingly endorses
unfairness, imbalance (or ‘a lack of debate’),and even those ‘mainstream’ values he seeks
to redress.

Summer 2005: Volume 13, Issue 2

On CUE

Opinion & Perspective

Council of Europe/Cassell.
Peaty, D. (2004). Global issues in EFL:
Education or indoctrination? The
Language Teacher, 28(8), 15-18.
Sargent, T. (2004). Advocacy-oriented
global education in TLT special issues.
The Language Teacher, 28(2), 9-13.

Stradling, B. (1989). How’s your crap
detector? In D. Randle (Ed.), Teachinggreen: A guide to education for life onearth. London: Green Print.
Strain, J. (1991). Interview: Anita L.
Wenden. The Language Teacher, 15(9),13-16.

ANNOUNCING:

The CUE Forum: Professional
Development: Writing in Academia
at JALT/2005
Saturday October 8th, 16:20-17:20.
Room 910

Philip McCasland (Tokai University)&
Martha Robertson (Aichi University)“I don’t like writing, but I like having written,” writes one modern author. Many of us
can identify with this sentiment. Whether writing is experienced as drudgery or as anopportunity for professional growth depends upon one’s attitude and approach, as wellas one’s familiarity with the process. This forum welcomes both the novice and theveteran to share their perspectives as authors and collaborators, while we navigate theroad from a worthwhile idea to a polished published work.

The CUE AGM immediately follows the forum.
All CUE members are encouraged to attend.

42

Summer 2005: Volume 13, Issue 2

On CUE

From the Chalkface
Sign Your Name Right Here
Juergen J. Bulach
Jissen Women’s University
in their respective groups to actually do mostof the work. Not surprisingly, throughout
the course of the project I heard many complaints from students about group members
not doing their share of the work. Thus, myaim was to use learning contracts as a tool tomotivate them to cooperate more.

Learning contracts provide instructors
with a ﬂexible device that can complement
any class. They are written commitments inwhich students state their speciﬁc objectives
over a limited period (McGarrell, 1996). A
typical learning contract can contain up to
four basic elements: learning objectives,
resources and strategies, the evidence thatthe objectives have been met, and the criteria
used to assess the evidence (Knowles, 1986).
Proponents of learning contracts argue thatthe clear goals and directions contained incontracts help to prevent disagreements or
misinterpretations that can occur in projectwork (Anderson et al., 1996). In this paper, I
outline the development of learning contracts
in a project work setting that took place inan elective listening and writing course at a
private women’s university.

The project
I informed the students that the project
would require them to work together ingroups of four or ﬁve members over the duration of six weeks. They then formed theirgroups. I explained that the project would
engage each group in the production of aTokyo survival handbook for international
exchange students at their university. The
completed handbooks were to be placedin the International Center where the international students could read them at theirconvenience. The project involved the following steps:

The students
The students were 36 second, third andfourth-year English literature majors who
ranged from low-intermediate to advanced
in English ability. In an earlier group project,many of the same students exhibited somereluctance in cooperating with one another.
Although the project demanded that theywork together, they usually did this in pairs
or individually. As a result, there was little
communication. In addition, I observed thatmany of them relied on one or two members

1. Research culture shock
2. Interview foreigners living in Tokyo
3. Visit and collect information on thingsand places useful for international
students to know
4. Write articles on their ﬁndings (thelength of each article was from 500 to
700 words)
5. Compile their articles in a handbook

The groups were responsible for editing
their members’ articles for accuracy, length
43

From the Chalkface

Introduction

Summer 2005: Volume 13, Issue 2

On CUE

From the Chalkface

and content. They received two separate
grades for their work on the project. One
grade was for the quality of their work
which I assessed. The students themselves
were responsible for the second grade whichcorresponded to the quantity of their work
(number of articles). It was the second gradethat was addressed in their contracts. My explanation of the project lasted 10 minutes.

learning contract and brieﬂy discussed its
parts in detail.

Deciding on the contract option
After ﬁnishing my explanation on learning contracts, I distributed a contract form
to every student which included the samelearning objective. This learning objective
was for students to learn more about how foreigners adjust to living in Tokyo. The form
also contained three different options eachcorresponding to a speciﬁc amount of workand a resulting grade. The C grade option
required the least amount of work, and the A
grade option the most. I advised the groups
to carefully decide the option they would undertake. Students consulted each other about
their schedules. Out of the eight groups, ﬁveof the groups selected the A grade option,
while three chose the B grade option. No
group selected the C grade option.

Procedures
The development of the contracts entailed
a total of six procedures: explaining learning
contracts, deciding on the contract option,
discussing the contract, negotiating the contract, writing the contract and renegotiating
the contract. Following my explanation of
the project, the students completed the ﬁrst
ﬁve procedures on the ﬁrst day of the project.
I allowed them to renegotiate their contracts up to two weeks before the end of the
project. They also referred to their contracts
throughout the entire six-week duration of
the project. The stages outlined in Learning
Contracts (Anderson et al., 1996) formed thebasis for my procedures.

Discussing the contract
I included a question in the contract form
under the heading of Learning Objective.
The purpose of this question was to get students to list things or places that would be
helpful or interesting for international students. It also served as a discussion prompt
for the groups. In their subsequent discussions, I observed no shortage of answers
to the question. The groups discussed suchdiverse topics as transportation, restaurants,
stores, hospitals, entertainment areas, postofﬁces, libraries, foreign-language support
services, etc.

Explaining learning contracts
As this was the ﬁrst time for my students
to write and use learning contracts, it wasnecessary to acquaint them with the concept
and process of contract learning. I informed
them that, unlike the previous project, thenew project required that they ﬁrst writea learning contract. I then distributed ahandout that included a short deﬁnition oflearning contracts, descriptions of the rolesof students and teachers in developing learning contracts, and an explanation on theirusefulness. I went over the handout with thestudents and made sure that they understoodits contents. In addition to the handout, I
also provided my students with a sample

Negotiating the contract
The students then negotiated which of
their discussion answers they wanted toinclude in their contracts. The negotiations were divided into two stages. In theﬁrst stage, group members proposed and44Summer 2005: Volume 13, Issue 2

On CUE

grades, while two other groups renegotiatedfor lower grades. One of these four groups
decided to renegotiate their grade in the third
week, while the three other groups chose torenegotiate in the last week of renegotiations. In the four other renegotiation cases,
the groups chose to change their topics andrelated resources and strategies. These latterchanges occurred at different times throughout the ﬁrst four weeks of the project.

Variations
A great advantage of learning contracts isthat instructors can have students tailor theircontracts by choosing to enhance or restrict
any of the four elements mentioned earlier.
For example, students may be allowed tomake their own learning objectives with very
little input from their instructor. Conversely,
the instructor may exercise more control overthe direction of the contract by pre-specifying the objectives. In my class, I speciﬁed
the general learning objective beforehand.
However, I allowed them to select their own
topics connected to the learning objective.
The basis for my decision was to avoid introducing too many changes in a class that was
already experiencing problems in adjusting
to project work learning. Every class has
special needs and learning contracts can beeasily modiﬁed to meet those needs.

Writing the contract
The groups were ﬁnally ready to write
their contracts. Each group used one contract
form and ﬁlled in the sections under the headings of Learning Objective and Resources
and/or Strategies. After the contracts werecompleted they were given to me. Before
the next class, I made copies for myself andfor each student, and signed and dated all
the copies. In the next class, I returned themto the groups for the students to sign anddate. I retained one copy of each student’s
contract.

Renegotiating the Contract
I permitted the groups to renegotiate speciﬁc parts of their contracts up to two weeks
before the end of the project. The areas open
to renegotiation were the type of contract
option, the topics related to the learning
objective, and the resources and strategies.
At the beginning of every class meeting, I
reminded the groups that they could renegotiate their contracts if there was mutual
agreement among the members to do so. The
deadline was not negotiable. Renegotiations
occurred a total of eight times. In four cases,
two different groups renegotiated for higher

Conclusion
The impact of learning contracts on mystudents was substantial. Prior to the introduction of the contracts, many of my students
lacked focus, were not communicative anddid not cooperate well. This changed immediately after they started working on
their contracts and these changes remained
constant throughout the course of the project.
They no longer complained about their groupmembers. In the end, everyone finished45

From the Chalkface

negotiated the topics related to the learning objective, and the related resources andstrategies among themselves. For this stage,
I stipulated that all members should be incomplete agreement on the ﬁnal proposals.
In the second stage, I had to approve their
proposals. In determining the suitability of a
proposal, I judged whether or not it involveda task that was unreasonable in anyway, or
if it was simply not helpful to international
students. After approving a proposal, I wrote
my initials on it. A group could proceed to
the next stage only after I had initialled alltheir proposals.

Summer 2005: Volume 13, Issue 2

On CUE

References

From the Chalkface

their handbooks on time and fulﬁlled theircontractual obligations. Learning contracts
motivated my students simply because they
provided them with some needed structurethat had been previously absent. Most importantly, the learning contracts did this without
impacting the integrity of project work and its
emphasis on independence and responsibility.

Anderson, G., Boud, D. & Sampson, J.
(1996). Learning contracts. London:
RoutledgeFalmer.
Knowles, M. (1986). Using learning contracts.
San Francisco, CA: Jossey-Bass.
McGarrell, H.M. (1996). Self-directed learning
contracts to individualize language learningin the classroom. Foreign Language
Annals, 29(3), 495-508.

Appendix
Learning Objective
To learn how international students must adjust tolife in Tokyo

Resources and/or Strategies

Option #1 for a grade of C (must complete 6 articles)
What do you think would be helpful for international students toknow about Tokyo?

1. ____________________________________
2. ____________________________________
3. ____________________________________
4. ____________________________________
5. ____________________________________
6. ____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Option #2 for a grade of B (must complete 12 articles)
What do you think would be helpful for international students toknow about Tokyo?

7. ____________________________________
8. ____________________________________
9. ____________________________________
10. ___________________________________
11. ___________________________________
12. ___________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Option #3 for a grade of A (must complete 18 articles)
What do you think would be helpful for international students toknow about Tokyo?

________________________________________________________________________________________________________________________________________________________________________________________________________________________

13. ___________________________________
14. ___________________________________
15. ___________________________________
16. ___________________________________
17. ___________________________________
18. ___________________________________

If at any time you wish to renegotiate this contract for a higher or lower grade, you have the option to do so up to two weeks before
the end of the project (Nov. 4).
Signed _________________________
Signed ___________________________(Student)(Teacher)
Date__________

Date

Group Name _________________________

46___________

Summer 2005: Volume 13, Issue 2

On CUE

Monet, Renoir and Tomoko too:
Using Impressionist artwork to aid
self-expression

Marlen Elliot Harrison
Momoyama Gakuin University

This lesson introduces target structures
for creative self-expression via discussionof famous images (picture speculation andreﬂection) by impressionist artists. The goalof the lesson is for students to learn basic
sentence structures and vocabulary thatwill allow them to report their thoughts andfeelings as well as elicit them from others.
Additionally, it is a starting point for creative
expression and experimentation with autonomous verbal expression.
Students
Students should have experience withbasic self-expression and sentence structure;
High beginners and up.

Procedure

Materials
Images of impressionist paintings suchas Seurat’s famous “A Sunday on La Grand
Jatte” or Van Gogh’s “The Starry Night”(many images may be downloaded fromthe internet; one of my favorite sites is theSmithsonian Institute: http://www.si.edu).

Introduction
Begin with an explanation of the lesson, “Today we will practice talking about
our ideas and feelings and look at famous
paintings,” and a timed warm-up (approx.
5 minutes) asking students to ﬁnd out who
each other’s favorite artists are and why. A
sample conversation that may be pre-taught
could include:

Time required
30-45 minutes.
Rationale“Impression” may be deﬁned as a cogni-

A: Who is your favorite artist?
B: I like (painter’s name) , how about
47

From the Chalkface

tive or emotional response to exposure to a
stimulus. Introducing the imaginative andfamous images of impressionist painters
can help students understand how their own
creativity may be applied to the English
language to express their own impressions.
How many times have we asked students
questions only to be met with responses that
could be more complex and personalized?
Perhaps part of the reason for this is students
lack the language tools required for suchself-expression. If the great impressionists
used color, brushstroke, and paint as tools
to express their thoughts and feelings about
the world around them, then it seems thatstudents of foreign languages are performing
quite a similar task with words, sounds, andbody language as their tools.

SummarySummer 2005: Volume 13, Issue 2

On CUE

Picture speculation exercise
Show images (one at a time) of impressionist paintings such as Seurat are famous“A Sunday on La Grand Jatte” or Van Gogh’s“The Starry Night.” Model the conversation
procedure by having a student ask the above
sample questions and providing appropriate
responses. I ask students to “use their imagination” to think of creative responses that reﬂect their own impressions using the sample
sentences from above. The remainder of theactivity may be spent encouraging student
interaction though a brief restatement of lesson goals and a summary of lesson activities
by the instructor are recommended.
It is important for the instructor to remember to provide concrete examples oflanguage targeted, apply these kinds of tasks
only when appropriate to the students’ needs
and skill levels, and choose images that areappropriate to the professional classroom
environment. Additionally, the instructor’s
primary role is to help the students determine
the direction of the lesson, depending upon
their particular language abilities and needs,
and provide concrete examples. Regardlessof the outcome, students have a chance toconverse, build vocabulary, practice sentence
structure, and most importantly, increase
their abilities as impressionist speakers.

From the Chalkfaceyou?
A: Ummm, I like (painter’s name) .
Or
A: Ummm, I can’t think of the painter’s
name.

It is helpful to also spend about 5 minutes
eliciting ideas about the words “impression”
and “impressionists.” At least one student isusually familiar with the genre of impressionist painting and can name one or two artists.
Students can also build their vocabulary byreviewing such phrases as “ﬁrst impression,”“bad impression,” etc. Appendix 1 contains
a list of words that students can use for adictionary/translation exercise (one of my favorite dictionary exercises is a contest wherestudents see who can ﬁnd the Japanese word
the fastest) to help build their vocabulary forlater stages of this activity.
Next, ask students, “Why would a lesson
about impressionist painters be good for an
English conversation lesson?” as a timed(5-minute) brainstorming task, students
should work together in a small group andthink of at least three possibilities. After
students share their ideas, which typically
include responses like “vocabulary,” “manycolors,” and “fun thing for talking,” explainthat foreign language students are impressionist speakers who, like the impressionist
painters, must use a set of tools to create animage that expresses how they feel. Finally,
provide a number of sample sentences thatcan be used when discussing not only art but
most any topic. For example:

Assessment
The lesson is most successful when the
instructor limits correction of vocabulary
or grammar and encourages creative experimentation with language. There are many
possibilities for assessment at the instructor’s
discretion such as a follow-up conversation
task or quiz.

1. Q: What does this make you think of?
A: It makes me think of….
2. Q: How does this make you feel?
A: This makes me feel….
3. Q: What does this remind you of?
A: This reminds me of….
4. Q: Do you like it?
A: I like/ dislike it because….

Reﬂection
This lesson has a variety of outcomes: Students may create sentences or stories similar
48

Summer 2005: Volume 13, Issue 2

On CUEto poetry; students may issue opinions about
the artists’ intentions or abilities; and often
students will express their dislike or approvalof the images. Using the model question
and answer formats, students frequently
have entered into brief conversations thathave resulted in surprising and imaginative
speculations. In response to Van Gogh’s“Starry Night” one ﬁrst-year non-English
major responded:
This reminds me of winter-time in
Hokkaido…my grandmother’s place.
Sometimes star is bright and shiny.
Cold feelings can be happy.

has many possibilities and is a great way to
encourage students (with only a little help
from the instructor) to practice newly learned
vocabulary or grammatical structures.

• metaphors or similes: Q: “How does
she look?” A: “She looks as happy as abride.”• vocabulary: “Name as many adjectives/
adverbs/verbs for this image as you canthink of.”• action speculation: “What happened
before this point in time?” or “What will
happen next?”• title speculation: “What would be a good
title for this image?”

Another student offered the following uponviewing Cassat’s “The Boating Party”:
On a beautiful afternoon a family
sails far from their trouble. This
painting makes me feel love and I
wish to have this experience too.
Additionally, students should be encouraged to keep a list of all new words learned
during the exercise and to practice using
them during their conversations. This format

Appendix
Vocabulary for impressionist exercise
an impression
a painteran idea
Impressionist
to remind one ofa brushstroke

an imageto feel (an emotion)
to createtools
feelings
creative49

favoritea speakerart
a painting
an artist
imagination

From the Chalkface

Variations
There are many possible approaches tousing artwork in a language lesson. Photographs, drawings, or paintings of any genre
may offer students a chance to enter into
conversation and express themselves. Many
forms of language may be emphasized, forexample:Summer 2005: Volume 13, Issue 2

On CUE

Cyber Pipeline

I am the Internet: Using Beatles
Materials on the Web

Cyber Pipeline

Brian Caspino
Himeji Dokkyo University

Beatles Lyrics

Recently I brought a karaoke machine into
a conversation class to use during a classparty. Students were shy at ﬁrst. I did a horrible rendition of a Patsy Cline song and a
Michael Jackson song to lighten the moodand offer encouragement. A few minutes
later, a student stepped up to the mic and sangsome Beatles. One by one students crowded
around, bobbing their heads, clapping andsinging along with her. She paved the way
for other students to try. Beatles songs arevery popular with students. During the party,
students sang a total of eight songs—one
each by John Lenin and John Denver, and six
Beatles songs. The singers seemed to know
nearly all of the words to the songs, and others sang along while attentively watching thesinger or the words on the screen.
My experience in karaoke boxes and bars
in Japan had already led me to believe that
all Japanese people old enough to frequent
these places were familiar with the Beatles,
could sing some Beatles songs, were familiar
with their music, or were interested in the
Beatles. I realized that the Beatles might bea hot topic for my younger students down
the road.
After the party, I went online to make themost of this situation. I searched Beatles,
karaoke, trivia and lyrics and retrieved some
useful and entertaining sites. You may ﬁnd
the following sites helpful.

http://welcome.to/beatleslyrics allows you
to view all the Beatles’ albums. Click one forlyrics to the songs on that album. There arelinks to John Lennon and Ringo Starr sites.

Beatles Karaoke
http://www.geocities.com/SunsetStrip/
Club/8815/music2.html provides free online
karaoke versions of the Beatles’ songs.

Beatles Trivia
http://www.geocities.com/Sunset Strip/8703/trivia.html and http://members.
aol.com/Razmatazzz/beatriv1.html offer advanced-level Beatles trivia that might be used
for an a homework assignment or project.

Beatles Info
http://www.beatlesagain.com is full oftrivia, rumors, history, facts, and humor.
http://members.tripod.com/~taz4158/
beatles.html offers video, audio, interviews
and articles, including an introduction to the‘Paul is Dead’ controversy.
These sites supply material to structure
a lesson, a unit, or a semester. If these sites
don’t suit you, a Google search for Beatles
retrieves 7,860,000 sites. If only to give students a taste of the Beatles, a 15-20 minutetrip to the computer lab can provide a wealthof useful and fun resources.
50

Summer 2005: Volume 13, Issue 2

On CUE

Conference Review
Glocalization through CALL:
Bringing People Together

Forrest Nelson
Tokai University

for presentations that would answer my question, “How can I make Moodle track whether
students have seen and heard my video andaudio ﬁles?”I started the morning off with a coffee andthe opening presentation by Uschi Felix. She
related that the use of “asynchronous” and“synchronous” chat communities can provide anonymous communication which inturn has been shown to reduce the speaker’s
fear of making mistakes. In turn, the student
opens up to more authentic communicative
activity (Uschi, 2005). Ms. Felix provided
the following sites as examples that support
anonymous communicative activities:•••

Wimba-voiced bulletin boards at
http://horizonwimba.com
Traveler-voiced chat communities at
http://www.digitalspace.com/traveler/
Lyceum-audiographics at http://kmi.
open.ac.uk/projects/lyceum/

My next stop was Kevin Ryan’s “Digitally Editing Sound.” This presentation wasboth informative and helpful, providing mewith new ideas on how to edit audio ﬁlesand how to serve them over the Internet.
Ryan also introduced a powerful open source
audio tool called Audacity, which can be
downloaded at http://audacity.sourceforge.
net/. This software allows users to cut andpaste and add sound effects. Finally, after
editing, Audacity can export audio ﬁles tomany formats, including MP3. One of its
51

Conference Review

Glocalization through CALL was the chosen title for the recent JALTCALL conference at the Biwako campus of Ritsumeikan
University held June 3rd to June 5th. Be careful not to misread the title. If you read it as“Globalization,” you are forgiven. Wikipedia
Encyclopedia online states that Glocalization
is “the creation of products or services intended for the global market but customized
to suit the local culture” (Wikipedia, 2005).
A general example of glocalization is the
web site, www.meetup.com, which allows
users to create local meeting groups basedon personal interests and then bridges themwith other groups of similar interest aroundthe world. In other words, a local meetup.
com group of chess players in “lil ole” Soso,
Mississippi would now have connections
with other chess player groups around theworld. As for JALTCALL, “Glocalization”
refers to ESL/EFL learning websites, according to the plenary speaker Uschi Felix in her
opening presentation.
Saturday morning I woke up to a beautiful view of Lake Biwako, knowing it would
be a great day to be outdoors ﬁshing on oneof the many boats dotting the lake. Instead,I headed for the Ritsumeikan Biwako campus which should not be confused with the
Kyoto campus as several people had done.
The previous night, I had prepared by looking over all the presentations and writing
down my personal want list. I was looking

Summer 2005: Volume 13, Issue 2

Conference Review

On CUE

best features, however, is its multi-trackrecorder which lets the user add backgroundsounds such as music, or people talking in a
crowd. For example, if you wanted to takea recording of a conversation at a park, you
could add another audio track of birds chirping, children laughing, and wind blowing
through the trees. For my purposes, I want
to upload MP3s to my Moodle online classmanagement site for students to practicelistening. Mr. Ryan’s presentation will help
me to complete this task. However, tracking
student activity online still eluded me. My
quest was still incomplete.
With a new tool for handling audio ﬁles,I wanted to see what others were doing withvideo, so I went to see Robert Chartrand’spresentation on editing videos. He instructedparticipants on the use of Movie Maker starting from importing video clips to importing
narration and ﬁnally creating the ﬁnishedproduct. One month later, I am sitting at mydesk writing this paper and while taking a
break from typing, I decided to try and makea quick movie using my PC, my chat camera and Robert’s presentation instructions.
Within 20 minutes, I had recorded 2 video
clips, added a title frame and credits, audio
and background music. It is just that simple!
Movie Maker is a powerful tool that teachers
can use to create videos for their classes.
Another video presentation I saw wasgiven by Simon Hunter and Richard Hawking, both of Obirin University. Although they
discussed different videos, both presented on
their method of delivering video via the Internet. Their focus was on the use of Flash asa video delivery system adding that the Flashcyber-community contains hundreds ofpre-designed templates. This is particularly
helpful because Flash is still not the easiest
software to use. It would be easier just tolink a video ﬁle in an HTML page, but Flashcan add a lot more functionality than HTMLcan. Having hundreds of templates to choose
from makes Flash so much easier to use andmany of the templates look professionallydesigned. Simon Hunter presented an online interview of various professors about
the books they had read and why reading
is so important. To do such a homepage inHTML, the average instructor would only
be able to link text to various audio or video
ﬁles. The design would be very simple andnot very pleasing to the eye. On the other
hand, using Flash allowed Simon to link
video interviews to animated text, as well as
add feedback messages to the user showing
how much of the video had been played andhow much was left. Next, Richard Hawking
went on to show how Flash can be used to
set up a basic speech feedback site. His idea
was to provide a place where students could
upload their own presentation video ﬁles that
could be checked and graded by an instructor on the Internet. Again, to do this would
require either Flash or PHP/MYSQL. Generally, PHP/MYSQL requires a server that
can handle a database and the scripting forstoring video and data in MYSQL requires
a high technical skill in programming. However, Richard showed how Flash could do
the same thing without using web databaseserver technology. It is still difﬁcult to dowhat was presented, but at least there aremany Flash templates in cyberspace.
After their presentation, I began to think that
Flash was the technology that might meet mygoal of tracking audio/video viewing activity
in Moodle. How to connect Flash to Moodle
was still a difﬁculty. However, just down
the hall was a presentation by Paul Daniels
on adding data ﬁelds to Moodle’s MYSQL
database and enabling Flash to be tracked
within Moodle. What Paul did in this presentation was show how teachers with some52

Summer 2005: Volume 13, Issue 2

On CUE

References

Uschi, F. (2005, June 3). Multiplying
modalities: Opening up the Fourth
Dimension to the Online Learner. Paper
presented at the JALTCALL Conference
2005.
Wikipedia online encyclopedia. (2005).
Deﬁnition of Glocalization. Retrieved July
3, 2005, from http://en.wikipedia.org/wiki/
Glocalization

Been to any good
conferences lately?
Share your experience withyour colleagues!
On CUE invites reviews of ESOL conferences orindividual presentations.
Inquiries to Phil McCasland: mccaslandpl@rocketmail.com
53

Conference Review

cal skills. In previous years when attending
JALTCALL, I went there with no clear goalin mind—this may be indicative of ﬁrsttime attendees. However, my advice is to
go to this type of conference with obtainable
goals such as technical problems to solve.
A personal want list will deﬁnitely focus
your attention, increase your understanding,encourage your participation, and improveyour overall experience.

programming skills can tell Flash to pass avalue such as a ﬂash quiz score to a variable
that can be sent and stored in a MYSQL
database table. When the teacher logs on to
his/her Moodle web page, the teacher will beable to see that the student viewed the Flashﬁle containing the video or audio. If the ﬂash
ﬁle contained any questions that were scored,
then they would also be able to see student
scores in the grades section of Moodle. Finally, let me mention that at the time of theconference, Paul’s solution to connect Flashto Moodle was only temporary. He went on
to say that a Flash module for Moodle wasunder development and should come out atthe beginning of 2006.
At last, all my questions had been answered. Starting this fall semester, I will be
tracking video and audio viewing activityof my students—thanks to the cumulativetechnical information provided at this conference. JALTCALL proved this year to be agreat help in my desire to increase my techni-

Summer 2005: Volume 13, Issue 2

On CUEBook Review

Ideas & Issues: Pre-Intermediate
Geraldine Sweeney. Tokyo: Macmillan LanguageHouse, 2000. pp. 96. Textbook. ¥2,000.
ISBN: 4-89585-318-7. Teacher’s Guide. 318. Audio Cassettes.

Ideas & Issues: Intermediate
Olivia Johnston & Mark Farrell. Tokyo: Macmillan LanguageHouse, 2000. pp. 102.
Textbook. ¥2,000. ISBN: 4-89585-319-5. Teacher’s Guide. 319. Audio Cassettes.

Book Review

Reviewed by Nevitt Reagan
Kansai Gaidai University
class cassettes (or CDs) for each book:one contains the listening passages andpronunciation exercises; the other has thereading passages.
Both books feature high-interest,
contemporary issues, such as girl power,
fashion design, mobile phones, green
issues, and sexism. The listening passagespresent natural delivery and include a
range of (mostly) British accents. The shortreading passages in Pre- Intermediate are
graded; the slightly longer, more complex
readings in Intermediate are excerpted from
popular British and American newspapers
and magazines. Overall, the two books
portray “Cool Britannia” – a young, hip,
multicultural Britain.
The books are visually stimulating,each page loaded with a great variety
of eye-catching color photographs andartwork. Intermediate, however, includes
rather unusual photos (for a textbook). For
example, the ﬁrst three units have images of
naked women, a dead sheep, and a bloody
gunshot wound on a human head.
There are a couple of minor organizational
problems. First, the recordings of the listening
and the reading passages are on two different
tapes, causing the inconvenience of having

These two textbooks (part of the ﬁve-volume
Ideas & Issues series) are great looking andclearly designed. Originally published by
Chancerel International Publishers Ltd., theMacmillan LanguageHouse editions contain
English-Japanese glossaries in the student
books as well as Japanese translations in
the teacher’s guides of listening passages.
Although they purport to teach four skills,
they are perhaps most useful for listening/speaking courses.
Each unit is organized as a tightly packed
four-page spread. After an initial reading or
listening passage, the grammar, vocabulary
and pronunciation points are presented
and practiced brieﬂy. These are followedby open-ended communicative pair- andgroup-work activities. “Language Practice”
exercises at the back of the Pre-Intermediatebook supply additional work on vocabulary
and grammar items. Each book claims toprovide about 32 hours of material (22 90minute periods).
T h e t e a c h e r ’s guides include an
introduction to the teaching approach,lesson plans for Lessons 1 & 2, notes forthe remaining lessons, tapescripts, answer
keys, and Japanese translations of thelistening passages. There are also two54Summer 2005: Volume 13, Issue 2

On CUE

supplies only a bit of written consolidationof the material rather than actually teachingor practicing any new writing skills.
The Ideas & Issues series certainly has
several strengths. The artwork is truly
engaging and the topics will appeal toteens and young adult learners. The simple
exercise formats make classwork go quickly
and smoothly. Bilingual glossaries prevent
constant in-class dictionary searches.
Grammar and pronunciation points aresequenced well. Finally, the extensive
Japanese language support could make thesetextbooks good choices for teachers whowant to deal with young adult-level issues
without spending too much class time on
grammar or vocabulary.
In sum, the Ideas & Issues series contains
motivating topics in a format which
serves up attractively arranged samples ofcontemporary English—a series of light,
tasty snacks but not really a meal.

Writers Needed!
On CUE seeks writers of feature articles, research notes,
opinion essays, book and conference reviews, and classroom
applications.
On CUE is now a refereed journal. All feature submissions arevetted by our professional review board.
Whether you are a previously unpublished author or aseasoned veteran, your submissions are welcomed andencouraged!

55

Book Review

to switch media frequently in mid-lesson.
Second, the “Language Practice” exercises(Pre-Intermediate only), not much different
from those in the units themselves, are placedat the back of the book. There seems to be noreason for this, except perhaps to maintainthe four-page unit format.
The design and instructions for the
exercises are simple and clear. However,with eight to twelve activities, along withlarge graphics, crammed into each fourpage spread, the slightly larger than B5-size
pages seem overcrowded. In addition, theexercises are very short, providing rather
limited practice, with tiny spaces for student
written responses.
The brief writing tasks provide very little
support. For example, the Pre-Intermediate“Sport and Money” unit merely asks
students to “Find out about a famous sports
personality and write a paragraph about him/her” (p. 39). and gives a short list of prompts,
such as “Name, Nationality, Sport,” etc. This

Summer 2005: Volume 13, Issue 2

On CUE

JALT's Asian Scholar Program
JALT’s Asian Scholar Program continues this year with another representative from Southeast Asia.
Mr. Chamroeun Koun will make a presentation on Sunday morning at the national JALT conference
on a topic many of us will ﬁnd relevant to the teaching situation here in Japan: “Students’ Beliefs
About Causes of Their Failure.” Please consider making time at Granship to attend this special
presentation which will be cosponsored by the CUE SIG and the Omiya Chapter.

Students’ Beliefs About Causes of Their Failure

Mr. Chamroeun Koun

Abstract

results also suggest implications for teachers tocope with the students’ attributions to enhancetheir motivation and expectancies for future
success.

Failure in English language learning is universal and it seems to be a serious issue for students
of English and in English education as a whole.
However, little attention has been paid to investigating the real causes of their failure.
This presentation will report on the ﬁndingsof a small-scale research study conducted withadult students aged 18 to 25 about their perceived
causes (attributions) of their failure in studying
English at a private language school in Cambodia. The instrument used forcollecting the data was a “semistructured interview.” The data
analysis was based on theinterpretive method paradigm
and the common categories of
attributions: ability, effort, taskdifficulty, and luck (Weiner,1979). The results indicated thatthose students ascribed both external and internal dimensionsof attributions to their failure.
For the external dimension, thecommon attributions were task
difﬁculty, teaching, and school
administration/management,
while ability and effort were themost commonly reported forthe internal dimension. Learning environment/pressure, family, motivation,
and anxiety were found in the study. The ﬁndings
suggest that these attributions are believed tohave a very strong relationship with the students’
expectations, motivation, and affective domains
for their future success in learning English. The

About the Presenter

Chamroeun Koun has a Bachelor’s Degree in
TEFL from the Institute of Foreign Languages(IFL), Royal University of Phnom Penh, and a
Masters in Applied Linguistics
from King Mongkut’s University of Technology Thonburi(KMUTT), Thailand. He has
several years of experience inteaching EFL students in Cambodia and Thailand, plus someexperience as a teacher trainer
for the TEFL1 Training Programat Spencer International, Chichester College, based in Phnom
Penh, Cambodia and Bangkok,
Thailand. He gave presentations
at CamTESOL workshop 2004
and CamTESOL conference
2005 on “Practical Issues in
Teaching.” Also, he has attended a series of ThaiTESOL conferences, and other workshops
on English language teaching in Cambodia,
Thailand, and Vietnam, and he has recently copresented at ThaiTESOL colloquium. Currently,
he is working as the Professional Development
Manager and teacher of English at the Australian
Centre for Education, Phnom Penh, Cambodia.
56