Self-assessment
and Learner

Strategy Training
in a Coordinated

Program: Using
Student and Teacher
Feedback to Inform

Curriculum Design
Heidi Evans Nachi
Kwansei Gakuin University,
School of Policy Studies
heidi@ksc.kwansei.ac.jp
This chapter investigates student and teacher attitudes regarding self-assessment instruments andprocedures in one university English Language Program (ELP). Students in thefinal semester of arequired four-semester ELP responded to a surveyabout self-assessment, and teachers answered asimilarsurvey. Descriptive statistical analyses of responses and a qualitative analysis of open-ended
question responses reveal some differences in attitudes among students; moreover, a comparisonof student and teacher responses reveals further differences in relation to materials design andmotivation. Theseresults may havesome relevance tofuture assessmentinstrument and curriculum
design.

*$«, h^^^omm^a^yM^hv^z^n^^^mm^&t^o^m^zwt^^^tmB(D^-^yjiz^^xmu^.A^mmizmzmi(DEL?iz.^'MtticLmmzy>^x(DTy'T-bizm

gu mMhmn<07>ir-h\z9x.it. m^mmmmm&^u, m®&&Kffl^<z>®%(Dn
M^f/rtcioT, ^<n%-X.1j\zmj&^<y)fr<DWm£jM}]^fr\zt£^tza SSfc, ^^.tfimo

0**ltlfeUfcic:5,**J©x1f^>i:ii«tJtft<Di8fl5C*V>r!E&Sffia!^WS^CiS:-3fc.

157•

A u t o n o m y You Asks

158

•

Nachi

Introduction

Query students in your classes or teaching colleagues about alternative forms of assessment,such as self-assessment, and you're likelyto hear a variety of responses:• Mostof the students don't take it seriously andjust write scores thatthey don't deserve toget. Student• I think students and teachers often evaluate different aspects of tasks, so although theirscores may besimilar, I think 'intentions' aredifferent, typically. Teacher•/ like self-assessment because we have time to review theclass activities andpersonal
work, so we can recognize what we need to do to improve for us. Student• Ingeneral, self-assessment depends on the individual student. It's effective for someanda total wasteof timefor others. Teacher• I think teacher and student scores will bedifferent because theylook at it (student
learning) from different angle. Student• I think thatself-assessment is one of the most effective methods teachers can use toencourage thoughtfulness and autonomy in students. Teacher
These responses, culled from student and teacher surveys I administered at the Japanese
university where I teach, exemplify some of the complex range of views towards self-assessmentthat can be found. Although self-assessment is believed to promote language learning, learnerawareness, and learner autonomy (Tudor, 1996), and self-assessment is widely used in current
language education programs, inter-group (between students and teachers) opinions and intragroup (within a group of teachers or within a group of students) opinions both overlap andconflict. What does variation in student attitudes towards self-assessment mean? What does
variation in teacher attitudes towards self-assessment mean? How can the answers to thesequestions impact curriculum development and materials design? Do these differences signal
problems within the framework? My aim in this chapter is to explore these issues and offersome answers which can positively influence future materials development.
Context

The following study was conducted in the English Language Program (ELP) at Kwansei Gakuin
University's School of Policy Studies. A brief explanation of the organization and aims of the
ELP can help clarify my motivation in doing this research. The ELP is a rigorous, content-based
English for Academic Purposes program enrolling approximately 1,000 students with TOEFL
scores ranging from 350 to 600. Throughout the required four semesters, students learn how
to formulate, support, and express various perspectives, in written and oral modes, on socio-

ACKNOWLEDCEMENTS

This research was supported by the Special Research Fund, Kwansei Gakuin University. I would
like to thank my collaboration partner; Mike Nix, for his valuable questions, comments, andinsights. I would also like to thank Professor Steven Ross for his assistance with the statisticalanalysis.

— LD SIG 2003 A n t h o l o g y

159

*

Self Assessment a n dpolitical and environmental topics through individual and group research, discussion, writing,and presentation projects. Commercially produced textbooks are not used in the ELP—allcourse materials used by ELP teachers are created by eight full-time Associate Lecturers ofEnglish using authentic sources for each skills class. Skills and topics are horizontally integrated
across classes, and vertically integrated from semester to semester. In this way, ELP courses
recycle and build upon skills and content, preparing students for success in upper divisioncontent courses and in their postgraduate careers.
Challenges

As a member of the full-time staff, I am involved in the design and implementation ofprogram-wide materials. I have witnessed both successes and challenges of using the samematerials for all students, and I've become increasingly interested in how self-assessment andlearner training materials in particular are used by a diverse teaching staff for a diverse student
cohort. Course materials, for example, can be used by more than a dozen teachers for over
400 students. I developed this research study to explore how student and teacher attitudes andfeedback in general might inform such curriculum design.

I have been fortunate to work in a context where promoting student autonomy—the ability
to work independently—is a central goal. By"independently" I mean that students can, on
their own and collaborativelyin groups, identify topics, carry out research, and discuss, present,
and write effectively about various topics. Autonomy is encouraged by giving students choicein their topics and assignments within the parameters of an overall class framework. Autonomy
is further developed by giving students responsibility for critiquing their own and their peers'work in various stages of completing a project. In the first year, self- and peer-assessment areclosely guided by instructors, and, in the second year, students are given more freedom in thecriteria used for evaluation. Self- and peer-assessment instruments are used as a pedagogicaltool to remind students of previously taught skills and topics, to facilitate goal-setting andencourage formulating plans to achieve those goals, to increase awareness and independence
as students learn to evaluate their and others' work, and to help students see the value of giving
and receiving feedback (Oscarson, 1989). Through training, practice, and discussion, teachers
help students understand these benefits to remind them of the value of alternative forms ofassessment.

At first glance, the ELP is a progressive program, embracing current methodology andpedagogy informed by'cutting-edge' research to deliver an education designed to foster learnerautonomy. However, since I joined the full-time staff more than 3 years ago, full- and part-timeteachers alike have voiced concerns during teachers' meetings about current self-assessment
materials and procedures, stating that (a) there seems to be little effort to coordinate selfassessment instruments for different skills classes, and (b) there are no explicit guidelines (forstudents or teachers) outlining how self-assessment should be carried out. Moreover, teachers
have reported a number of informal observations regardingthe use of self-assessment:• Since self-assessmenttraining is left up to individual teachers, students may receive
very different experiences.

•

Instruments which work well with higher proficiency students do not seem to work
as well with lower proficiency students.

•

Higher proficiency students and female students tend to score themselves lower.

•

Students don't seem to see the value of self-assessment.

— Autonomy You

Aski —

i 6o

•

Nachi

Hearing these claims both echoed my own classroom experiences and fueled my interest
to investigate them more deeply in an effort to improve existing materials. At this point,
responding to each claim in detail is beyond the scope of this chapter. Rather, my purpose is tobegin exploring student and teacher attitudes toward self-assessment in an attempt to better
understand how the stakeholders—teachers, students, and curriculum developers—perceive
what has come to be a critical component of the curriculum. Ultimately, I'm arguing that theopinions of all stakeholders need to be considered to promote thoughtful, meaningful selfassessment and learner strategy materials design. To this end, I have focused on the following
questions:
1. What attitudes do students and teacher have towards self-assessment in the ELP?

2.

Do students and teachers share similar attitudes towards self-assessment in the ELP?

3. What impact, if any, does gender have on student beliefs?

4. Do students perceive some value in completing self-assessment?
5. Do students assess themselves honestly?
Fostering Autonomy—Learner and Curriculum Issues

Thoughtful implementation of self-assessment is critical to effectively promote autonomous
learning. To begin with, teachers must be aware of themselves as learners, believe and trust
their students, and possess a sincere desire to foster learner independence (Breen & Mann,
1997). Teachers must also acknowledge the impact that learner diversity may have on classroom
behaviors and procedures. Studies show teacher-learner interaction and learner-learnerinteraction can be affected by many factors, such as gender. More specifically, it has been shownthat male students have more talk-time in the language classroom, and some evidence suggeststhat teachers expend more energy on male students (Shehadeh, 1999; Sunderland, 1992).
If male students talk more or if male students receive more teacher feedback, how might
this affect self-assessment? Is this an issue in Japan? Thomson's research on learners studying
Japanese takes into account gender and self-assessment, showingthat Asian females tended torate themselves lower, and ascribes this tendency to cultural background (Thomson, 1996).
Yet to better understand the interplay of gender (as well as culture) with self-assessment,more
research to investigate learner differences is needed (Vandrick, 2000). Ultimately, being mindfulof these issues means additional research, combined with carefully planned self-assessment
materials design and classroom procedures,is paramount: If students and teachers are
dissatisfiedwith current self-assessment procedures, then current practices will be limited in
making positive contributions towards learner autonomy.
Methods

Participants

Both students and teachers in the ELP were surveyed for this study. Sixty-five students, nearly
all of who were second-yearstudents, took part in the study. They were all enrolled in Animal
Issues, a Special Topics elective class taken during the last (fourth) semester of required English
courses. Twenty-eight students were female, and 36 male. The students werespread acrossthree, non-streamed classes, and represented a range of proficiencylevels. Eighteen out of 22
ELP part-time and full-time teachers completed surveys.

LD SIG 2003 A n t h o l o g y —

i6i

•

Self Assessment and

.

.

.

Course Description and Self-assessment Training

The Animal Issues course was a reading-based discussion course focusing on endangered
animals, wildlife trafficking, abandoned pets, and animal research among other topics.
Students completed readings and internet research to support in-class discussions, as well as
individual and small group project-based presentations.Assessment was based on completionof homework, vocabulary and reading comprehension quizzes, project work, presentations,and self- and peer-assessment scores.Students self-assessed discussion effort and participation
as wellas presentation preparation, and they also peer-assessed group members' presentation
performance. The self-assessment component accounted for 5% of the final grade.
Students completed a self-assessment form three times during the semester immediately
following in-class discussions (see Appendix A on the Autonomy You Ask! website). Students
were asked to focus on their preparation for discussions, as well as their effort and participation
during discussions,by responding to the following three prompts: (a) choose a numerical scorereflectingyour holistic effort and performance, (b) write a justification of the chosen score, and(c) write a plan outlining what and how you will improve for the next discussion.
To encourage active participation, some simple classroom practices were implemented.
For example, at the beginning of the course, students reviewed features of effective group
work and active classroom participation. Then, before all group work activities, they reviewed
these features. The purpose was twofold: first, to enable students to co-construct, with teacher
feedback, some standards for expected classroom behaviors, and second, to help students
apply these standards when choosing and justifying their own scores. In addition, eachmember selected (or was assigned) a group role (leader, note keeper, reporter, English Police).
Establishing these basic policies was instrumental in reminding students of their roles in theirlearning.
Activities designed to promote more thoughtful completion of self-assessment wereconducted throughout the semester (see Figure 1 over). Before completing self-assessment for
the first time, students brainstormed the rationale and potential benefits of self-assessment.
Student answers were then discussed by the whole class with teacher feedback and support.
Examples of good self-assessment forms were distributed and analyzed, and students
brainstormed additional information, creating a menu of feedback phrases and content to draw
on in future classes. These examples, in addition to classroom discussion, were aimed at giving
students the tools they needed to choose and justify their scores. To encourage students totake charge of their learning, they formulated a plan to improve their effort and participation
for the next class. Before beginning the next class discussion, students reviewed these plans
and were asked to keep them in mind during the day's activities. The self-assessment forms
were completed once again, collected at the end of class, and returned during the next classwith feedback from the teacher. Throughout the semester, we revisited the evaluation criteria,
reviewed good models, and discussed features of effective and less effective self-assessments.

I wanted to give useful feedback to students to further promote reflection and honesty. If
students expressed themselves well with concrete examples and details, I praised their efforts.
If student responses were vague and lacked detail, I wrote questions eliciting additional
information, and made brief suggestions on how they might expand their answers. If I
thought a student had given her/himself an overly high or low score, I wrote a question about
their score. I never changed their scores, but I wrote a quick note based on their in-classperformance, within our class-established evaluation framework, that might suggest their score
could be higher or lower. I felt I could only give this kind of feedback in extreme situations: forexample, for a student who did not interact and did homework in-class, yet scored themselves—

Autonomy You

Aski

—

162

Nach

Identify Goals & Needs

I
Why SA? Rationale

T
Analyze SA Models:
Language support &establish scoring criteria>•

Review comments&plan

Complete SA in Class:Justify score & make planT
Share & compareCollect teacher feedback

Return with comments

Figure 1 Self-assessment Process Model
10; or for a very active student who scored themselves 6 or less. When self-assessments werereturned at the beginning of the following class,students were encouraged to share their forms
with each other and ask me questions regarding my feedback.
SurveysSurveys were administered to students and teachers (see Appendices B and C on the Autonomy
You Ask! website). The student survey consisted of 15 statements and 6 open-ended questions,in English and Japanese. Using a 4-point Likert rating scale, students were asked to agree
or disagree with the 15 statements which were designed to investigate motivation, honesty,
and attitudes towards self-assessment classroom procedures and materials, as well as student
classroom behaviors. I decided to focus on these areas to explore whether or not students
perceived value in self-assessment and believed they were honest. Ultimately, I expected thatstudents (and teachers) who possessed positive attitudes towards self-assessment would agreewith most statements. The student survey was given to students on the last day of class.A
similar survey, made up of 12 questions and a comments section, was distributed to ELPteachers during the last week of class. Sharing 8 questions in common, the surveys aimed to

LD SIG 2003 A n t h o l o g y

i63

Self Assessment and.

uncover stakeholders' attitudes towards self-assessment instruments and procedures currently
in use in the ELP, with the ultimate goal of improving such practices and materials.
Results

Using descriptive statistical analysis, I examined student and teacher responses and attempted
to identify trends and differences between students and teachers. I was most interested inunderstanding, in more concrete terms, how students and teachers viewed self-assessment,
namely: (a) whether students perceivedsome value in self-assessment; (b) if students
completed self-assessment thoughtfully and honestly; and (c) on what points students andteachers broadly agreed or strongly disagreed.
Quantitative Analyses

Before reviewing the survey results, looking at student class scores and self-assessment scores
may help situate my research focus. Since the research serves to explore teacher observations
regarding self-assessment, I wanted to see how closely these claims matched the reality of myown classroom. Thus, I calculated average scores for each of the three times self-assessment wasadministered in class to see if any trends emerged.

Table 1 Average Student Self-assessment Scores
Female students

Male students

SA#1

7.8

7.7

SA#2

8.4

8.2

SA#3

8.3

8.7

Overall SA average

8.1

8.2

84.0

82.0

Overall course grade (out of100) average

Table 1 shows the average self-assessment scores for female and male students across thecourse. The average scores for all three self-assessments,as well as the overall average score, arevery similar. Females scored themselves slightly higher than males for self-assessments #1 and#2, and slightly lower than males for self-assessment #3. In addition, female and male students
received similar overall course grades, with female students earning slightly higher grades.
Bearing in mind student self-assessment scores and overall course performance, survey
responses are outlined below. Tables 2 and 3 below summarize student and teacher responses
for agree-disagree statements. For this survey, the four point Likert scale choices included: 1 =strongly disagree, 2 = disagree, 3 = agree, or 4 = strongly agree.

Autonomy You

Aski

164

N AC H I

Table 2 Survey Responses: Student Attitudes (N=65)
Question

Mean

SD

2.91

1. I understand why self-assessment is used.

F = 3.11

M = 2.83(n = 28)(n = 36)

2.73

2. Self-assessment encourages me to try harder.

.67
F = 2.96

3. Students should have the opportunity to do self-assessment.
4. Students choose their scores honestly.

.66

M = 2.61

3.08.65

F = 3.25

M = 3.03

2.63.78

F = 2.71

M = 2.64

2.48

5.

Student scores are similar to teacher scores.

.62

F = 2.68

6. I would like more self-assessment in my English classes.
7. I would like less self-assessment in my English classes.

8. Writing a justification of my score was useful and valuable.

2.19

.61

F = 2.25.66

F= 3.04 | M= 2.67

.76

.79

.75

M = 2.94

3.16.83

F = 2.93

M = 3.42

2.54.89

F = 2.39

14. If someone gave themselves a score that I thought was too
low, I could tell them to change it.

M= 1.97

2.77
F = 2.64

13. It's OK to show my score, justification, and plan to otherstudents.

M = 2.75

2.10
F = 2.32

12. If I don't finish my self-assessment in class and take ithome, I will probably forget to do it.

.76

2.79

10. If I write my self-assessment at home instead of in class,
I will spend more time on it and write a more thoughtful
11. I prefer to write my self-assessment in class.

M = 2.19

2.80

F = 2.93

self-assessment.

M = 2.39

2.19

F = 2.25

9. Writing a plan for the next discussion was useful andvaluable.

M = 2.39

M = 2.53

2.48.89

F = 2.50

M = 2.53

Table 2 shows that students moderately agree with all statements, reflecting some satisfaction
with the current system.The first three statements focus on how students view self-assessment,
and whether or not they see its value and are motivated by self-assessment. Statements 1 ("Iunderstand why SAis used"), 2 ("SA encourages me to try harder"), and 3 ("Students shouldhave the opportunity to do SA") havemeans of 2.91, 2.73, and 3.08, respectively. In otherwords, means which approach 3 (equivalent to "agree"on the Likert scale) indicate positive
attitudes towards self-assessment. Within these first three statements, however, is an interesting

trend: Female students tended to have slightly higher means than male students, which may
indicate that females agree more strongly with these statements than males. Statements 6("I would like more SA") and 7 ("I would like less SA"), which concern the amount of selfassessment used in classes, both have means of 2.19. Students moderately disagree with these

LD SIG 2003 A n t h o l o g y

165

•

Self Assessment andtwo statements, possibly indicating they regard the current amount of self-assessment as
adequate.

Statements 8 to 10 center on classroom procedures. Statements 8 ("Writing a justification
was useful") and 9 ("Writing a plan was useful") have means of 2.80 and 2.79,with female
scoresbeing slightly higher than male scores. Females' scores were, in fact, 3.04 and 2.93.Such
agreement represents some degree of satisfaction of current self-assessment instruments.
Statements 10 and 11 concern completing self-assessment at home or in class. Students tended
to disagree with 10 ("I will complete my SA more thoughtfully if I can do it at home") with amean of 2.10, representing moderate disagreement. In response to Statement 11 ("If I take my
SA home I will probably forget to do it."), the mean was 3.16, suggesting agreement, with 2.64
for females and 3.42 for males. This statement has the highest mean of all survey statements,
showing that students feel more strongly about this issue.
Several of the survey statements deal with issues of honesty. Statements 4 ("Students choose
their scores honestly") and 5 ("Student scores are similar to teacher scores.") are slightly lower
at 2.63 and 2.48, indicating less agreement with these statements. Females again tended tohave slightly higher means. These figures become more meaningful when student responses
to open-ended questions are considered. Likewise, the last three statements focus on honesty
operationalized in classroom procedures. After students complete their self-assessments,
they are asked to share their scores and explain their justifications to group members, as wellas encouraged to feedback on fellow group members' scores. For Statement 13 ("It's OK toshow my score to other students"), at 2.54, the mean rests in the middle. For 14 ("I could tellsomeone to change a score that is too low") the mean is 2.48 and for 15 ("I could tell someone
to change a score that is too high") the mean is 2.06. As the means approach 2.0 (disagree),it could be inferred that these students may not feel comfortable with these classroom
procedures.

Looking at the overall student means, as well as the means by gender, sheds some light on
student attitudes towards self-assessment in general, as well as materials, classroom procedures,and classroom behaviors. Comparing teacher responses to student answers shows students andteachers broadly agree on some points and disagree on other points (see Table 3 below).

Table 3 Survey Responses: Teacher Attitudes (N=18)
Question

Mean

SD

1. Self-assessment increases motivation.

2.67

.75

2. Self-assessment promotes reflection.

3.28

.45

3. Students should have the opportunity to do self-assessment.

3.44

.69

4. Students choose their scores honestly.

3.00

.47

5.

Student scores are similar to teacher scores.

2.72

.56

6. Writing the justification is useful and valuable.

3.33

.58

7. Writing a plan for the next discussion is useful and valuable.

3.17

.60

8. If students write self-assessment at home instead of class, they will
spend more time on it and write a more thoughtful self-assessment.

2.50

.50

Autonomy

You

Aski

—

166

N AC H I

9. Having student show their self-assessment to other students promotes
honesty and can be a valuable learning experience.

2.89

.66

10. Female students generally score themselves lower than male students.

2.78

.79

11. Lower level students tend to inflate their scores.

2.56

.69

12. Higher level students tend to score themselves lower than they should.

2.67

.58

Table 3 shows that teachers tended to agree more with statements than students on somepoints and less on other points, yet responses are generally more favorable, indicating teachers
agree more broadly with the statements than students. Statement 1 ("SA increases motivation")with a mean of 2.67 is similar to Statement 2 ("SA encourages me to try harder") on thestudent survey. The student mean at 2.91 is slightly higher than the teacher mean, showing aslight difference in attitudes.

Teachers moderately to strongly agreed with Statements 2 ("SA promotes reflection"),3 ("Students should have the opportunity to do SA"), and 4 ("Students choose their scoreshonestly"), showing positive attitudes towards self-assessment use in general. The teachers'
mean for Statement 4 at 3.00 is higher than the students' mean of 2.63, suggesting thatteachers do trust students and value self-assessment as a means of course evaluation. Teachers

also agreed more strongly than students with statements connected to the self-assessment
instrument itself (Statements 6 & 7), showing that teachers do feel asking students to justify
their scores and formulate plans is a critical part of self-assessment process.
The last four statements, which center on honesty and classroom procedures, show moderateagreement. In particular, teachers seem to subscribe to the practice of having students
share their self-assessments with group members (mean = 2.89). As expected, teachers also
moderately agreed that students do not always score themselves fairly.

Qualitative Analyses
The 6 open-ended questions on student surveys were coded to reveal patterns among
responses. I organized responses from all students question by question, and looked forpatterns in theme and vocabulary. I grouped responses together according to similar content
and totaled the responses, as shown in Table 4 below.Student responses are organized hereaccording to question. For Question 4,1 divided these responses by gender because I wanted
to see if gender had any impact on student perceptions of honesty. Since teachers claimed thatfemale students' scores were "lower than they should be" and lower than male students' scores, I
thought separating these responses by gender might shed some light on these claims.

Table 4 Survey Responses: Summary of Open-ended Student Responses
1.What do you like about self-assessment?
I don't like it.

1

I can find strong point.

6

I can evaluate myself without someone's evaluation.
I can find weak point.
I can improve for next class.
I can look back at myself / review myself.

9— LD SIG 2003 A n t h o l o g y

12
26
27

i67

Self Assessment and2. What do you not like about self-assessment?
It's troublesome and takes time.

It's too subjective; everyone has different standards.
Students lie and give themselves higher scores than they deserve.

835

3. What are the benefits of self-assessment?

We can know our strong points.
We can know our weak points.
It can help us improve.

3
10

16

We can think about what we did and look back.

7

4. Do you think there are any differences between women and men's scores?

female

male

Yes.

1

2

No.

7

23

It depends on the person, not if they are woman or man.
Women are more honest / thoughtful / strict.

6

5

I have no idea.

2

5

2

5. Do you think there are any differences between teacher and student scores?
No.

12

Yes. Students give themselves higher scores than they deserve.
Yes. Teachers and students are looking at different things; teachers andstudents have different viewpoints.

1214

6. How can teachers improve self-assessment?

Show more examples and give more support for what to write.

3

Give students more time to write.

4

Take more time to discuss self-assessment procedures.
Make time to compare with other students, and compare with teacher and6peer-assessment scores.

7

I don't know.

8

Use "point" / "concrete" / "detailed" system.

9

From the responses in Table 4, we can see that self-assessment clearly helps students 'look
back' and reflect on their performance and effort, even if they admit they do not always assignscores they deserve. Students view self-assessment as a method to inform them of their negative
or weak points more than their strong points. Overall,students recognize that differencesamong scores exist not only among students, but also between students and teachers. Based on
their experiences, students are in a position to deliver concrete feedback to impact instrument
design.
All comments from teacher surveys are listed in Table 5 below. In contrast to the student
survey, which included 6 open-ended questions, the teacher survey simply included a'Comments' section. With very busy schedules, I didn't want to push teachers to write morethan what they had time to thoughtfully complete.

Autonomy You

Aski

i 68

•

N Ac HI

Table 5 Survey Responses: Open-ended Teacher Comments

1.

I think it's quite valuable to ask students to assess themselves, though for Japanese
students, the notion of "self-assessment" is really unfamiliar. So they may need time tofeel comfortable with it.

2.

It's difficult to generalize! Generally, I think it's valuable, but not necessarily for the"score." I think it can help some, though not all, to promote responsibility in learning.
Self-assessment can promote honesty but can also promote group dishonesty. Students
are often reluctant to force another student to change a grade as it involves face loss
and confrontation.

4.

If self-assessment is done too often, it loses value.

5.

I think self-assessmentshould be done immediately after the activity while it is still
fresh. It may be useful to discuss a "group" self-assessment in order to hear otherstudents' ideas about your performance.

6.

I think teacher feedback (such as"Your scoreis a littletoo high") on student score—especiallywith display of self-assessment of students'work and scores to classmates—also
helps promote reflection, honesty, and bringing student scoresin-line with teacher scores.

7.

I think self-assessment can be a motivator, but it depends less on the students andmore on the teacher, e.g., how it has been introduced, what feedback is given.

8.

Women never grade themselves high enough. Self-assessment has value, but maybe
only for highly-motivated students. Though on the other hand, maybe it's justsomething they need more training in...

9.

I have found reflection and planning the next discussion is very useful with high level
students, but low level students don't reflect well. Perhaps there is a threshold level ofEnglish that you need!

10.

I think that self-assessment is one of the most effective methods teachers can use toencourage thoughtfulness and autonomy in students. Of course, the method andapproach need to be actively considered and evaluated, so that the process retains
credibility in the students' eyes.

11. I think it's only necessary to write a justification or a plan—not both.
12. In general, self-assessment depends on the individual student. It's effective for someand a total waste of time for others.

13. I think students and teachers often evaluate different aspects of tasks, so although theirscores may be similar, I think 'intentions' are different, typically.
The responses shown in Table 5 indicate a mixture of positive and negative opinions about
self-assessment, as well as comments regarding procedures in the current self-assessment
scheme. Broadly speaking, there is considerable overlap among these statements, showingthat teachers generally view self-assessment in positive terms, as a useful tool to promotereflection and motivation. Teachers also recognize limitations and challenges of using selfassessment, and see the importance of careful, thoughtful implementation.

LD SIG 2003 A n t h o l o g y169 * Self Assessment a n d . . .
Discussion

Self-assessment Scores and Gender

Are there significant differences between female and male students' self-assessment scores?
Rememberthat one area I wanted to investigate was whether or not teachers' claims that female
students score themselves lower than they deserve and that their scores are generally lower thanmale students' self-assessment scores are valid. (For my class, I collected self-assessment scores
only; I did not score students individually for each discussion. As a result, I cannot compare'their' scoreswith 'my' scores.) What wasstriking to me was that, contrary to teachers' claims(and my own observations as well), female and male students' self-assessment scores were quite
similar. The average male student's score of 8.2 is only slightly higher than the averagefemale
student's score of 8.1 (see Table 1 p. 163).
What can account for these results? It is possible that by the end of the last (fourth) semesterof ELP classes, students' scores have leveled out, because they have had lots of experienceand feedback using alternative forms of assessment. Moreover, the average score is slightly
above the 'average' of 7. As'veterans' of self-assessment, students may score themselves higher
because they understand their own learning processes better and have become accustomed tocompleting self-assessment. Finally, since the course was an elective, and students were enrolled
by choice, it's possible they were more motivated during discussions because of the content, so
perhaps their scores reflect this active participation.

But what do students have to say about this issue? Do students believe gender has any
influence on scores?According to Table 4 above, students do not presume that gender has any
bearing on self-assessment scores. While three students do believe scores may be different,
40 students do not perceive differences stemming from gender. Rather, students ascribe score
differences arising from students as individuals, not their gender. I still believe this issue needs
further investigation, given that more female than male students (9 to 5) believe females are
more honest, thoughtful, and strict (see Question 4, Table 4 above). Clearly, these accuracy andhonesty issues deserve further consideration.
Student and Teacher Attitudes

What differences in attitudes emerged between students and teachers? From students'
perspectives, self-assessment is a tool that allows them to "look back" and "think about how
to improve for the next class." I interpreted these responses to mean students see value in selfassessment as a method to reflect on their effort and participation and create a concrete plan
to improve their performance. These responses also nicelymirror Dam and Legenhausen's
claim (as cited in Cotterall, 2000, p. 112) that "Evaluation has a retrospectiveand prospectivefunction, in which the learning experiences of the past are reflected upon and transformed intoplans for future action." Essentially, students expressed the same idea, though not so eloquently.
What is interesting, however, is how students viewself-assessment as a wayto understand their
negative points more than recognize their successes. Identifying these weak points seems to be
the primary benefit of self-assessment for students.

From the data, students aren't aware of all the benefitsof using self-assessment, as outlined
by Oscarson (1989). Oscarson believes that training students to evaluate themselves benefits
their learning process and raises their awareness of not only their own performance, but also
fosters goalsetting and reminds them of previously learned content; that relying on students toprovide insight into their own performance gives a more accurate picture of a learner's abilities
and shares some of the assessment burden with the teacher; and ultimately that self-assessment—

Autonomy You

Aski

—

i7Q

•

Nachi

fosters independent learning beyond the classroom context. There is some indication thatstudents understand that assessment is a shared process based on the claim that self-assessment

allows students to "evaluate myself without someone's evaluation," which I interpret as
their ability to provide anotherperspective, in addition to teacher and peerevaluation (seeQuestions 1 and 3,Table 4 above). Gaps in their overall understanding about why selfassessment is such a critical component of their learning canbe overcome with teacher supportand improved classroom procedures to make self-assessment more meaningful and valuable.
Based on the surveyresults and my experiences working closely with other ELP teachers,
teachers typically support Oscarson's reasons forusing self-assessment. I interpret teacher survey
responses as a broad endorsementof self-assessment, when the materials are appropriate andprocedures promote autonomy. Teachers value using self-assessment and are committed to its
purposeful useas a tool to encourage reflection and foster autonomy. Clearly, teachers understandthe reasons for usingself-assessment, and, with improved materials, should be better equipped tohelp students more fullyunderstand the rationalebehind using self-assessment
Students and teachers hold slightlydifferent viewswith regard to implementation of selfassessment. Teachers view writing a justification (mean = 3.33) and plan (3.17) to be usefuland valuable while student means are slightlylower at 2.80 and 2.79, respectively. Moreover,whereas teachers somewhat agree with asking students to share self-assessments (mean = 2.89),
students are less keen on it (mean = 2.54). These are hardly great differences, but I think it's
important to note that perhaps some current classroom practices may need to be revised, as I
will discuss more fully in the conclusion.
Honesty

Are students honest? Responses from the open-ended questions certainly make the statistical data
more meaningful. In response to "Students choose their scores honestly,"the student mean was
2.6. Byitself, this figure has little meaning. However, looking at the extracts from Table 4 below,
we can see that clearly students believe that students are dishonest. It's not clear from the data
who is lying. Are they lying? Or are other students lying? Regardless, student responses suggestthat they believe scores are not (always) honest reflectionsof student effort and participation.

2.What do you not like about self-assessment?
It's troublesome and takes time.

8

It's too subjective; everyone has different standards.

8

Students lie and give themselves higher scores than they deserve.

35

5. Do you think there are any differences between teacher and student scor es?
No.

12

Yes. Students give themselves higher scores than they deserve.

12

Yes. Teachers and students are looking at different things; teachers andstudents have different viewpoints.

14

LD SIG 2003 A n t h o l o g y —

i7i

*

Self Assessment a n dContrasting the results of the above two questions poses an interesting contradiction. To
put it simply, 35 studentsclaim that students lieand award themselves high scores. Yet with
Question 5, only 12students noted that studentsaward themselves scores that are higher thanthey deserve. Shouldn't these numbers be similar? How can we account for this difference?
Perhaps a hint of the answer can be found in the claim by 14studentsthat students andteachers look at different features, and if this is the case, then we can't expect scores to be equal
because students and teachers have different perspectives. To truly understand these figures,
however, follow-up investigation is needed.
Limitations

The results could be more powerful if data were subjected to more rigorous statistical analysis.
For example, correlation between gender and achievement or proficiency with survey answers
could be explored. Also, student and teacher responses could be compared quantitatively using
discriminant analysis, a type oft-test which compares groups of different sizes to determine
which features serve to distinguish group membership. These analyses might reveal trends not
immediately evident in the present data. Finally, the number of cases is too small to generalize
to a larger population. If this study were replicated using an improved survey with a 6-point
Likert scale for a larger sample, perhaps the results would be more reliable, powerful, andgeneralizable to multiple contexts.

At this point, the results should be regarded as initial and applicable for one university context.
Based on these preliminary findings, more questions have been raised than answered! Further
investigation is needed to develop a deeper understanding and clearer picture of how these issues
play out beyond this particular context. Conducting a longitudinal study,for example, that tracks
changes in students' scoresmay reveal a deeper understanding of the relationship between selfassessment, gender, and proficiency, as wellas show how experience impacts self-assessment
procedures.Such a study would naturally include revised assessmenttools and procedures, withfollow-up surveys and interviews to investigate learner attitudes more fully.
Conclusions

The present study shows that attitudes towards self-assessment among teachers and students inthe ELP are complex and wide-ranging. Teachers regard self-assessment slightly more positively
than students. However, quantitative and qualitative results seem to confirm that students
understand the aims and benefits of self-assessment in a broad sense. Evidence of this can beseen in the fact that students regard self-assessment as a tool to "look back on" their effort anda method to improve "for the next time," which is aligned with teacher comments. However,students also expressed concernover issues of fairness and accuracy. Students are well awareof their power to score themselves higher than theydeserve, and they feel that some students
take advantage of this power. Moreover, students (and teachers) believe that students andteachers arelooking at learning from different angles, using different knowledge and criteria,
so student self-assessmentscores may very well be different from teacher-assessed scores.

Ultimately, teachers can help students understand self-assessment and its benefits by providing
opportunities for discussion and giving feedback.
Re-designing Self-assessment

One of my main purposes for investigating student and teacher attitudes towards self-assessment

isto stimulate curriculum renewal. Based on the research results, I would now like to suggest
a number of waysin which teachers in the ELP can improve self-assessment instruments and— Autonomy You

Aski —

172«

NACH I

procedures to help students have more meaningful experiences using self-assessment. Thesesuggestions, based on myunderstanding of current pedagogical principles and informed by
student and teacher feedback, represent a starting point for curriculum innovation formy
teaching context. I hope that they may alsobe of some use to teachers in other contexts.
1. Teachers should spend more time facilitating discussions about self-assessment tohelp students discover the rational for self-assessment. This is an on-going processthat needs to be re-visited as students become more proficient and experienced with
self-assessment. This includeshelping students understand the benefits of multiple
assessment efforts, such as self,peer, and teacher assessment.
2. Self-assessment instruments should combine a global, holistic rating with specificperformance-related ratings directly connected to current course skills and goals.
Performance-related ratings givestudents and teachers a more concrete view ofstudent performance and more fully inform holistic scores.
3. As students gain experience, they should be allowed to design or co-design selfassessment instruments.

4. Explicit evaluation standards and criteria for holistic and performance-related
ratings should be co-constructed to help students choose their scores. Theseevaluation standards should be re-visited and revised to reflect changes in
expectations as students' proficiency and experience increase.
5. Assessment instruments should aim to accommodate different perspectives among
students, peers, and teachers, so students should only be responsible for assessing
features which they can realistically assess. Teachers should take caution assessing
features for which they lack insight (i.e., student effort) and stick with "external"
features. However, it is likely that there will be overlap among various featuresassessed by students and their peers and teachers.
6. If a self-assessment instrument does not have a 'free' comments section, but students

would like to write comments anyway, they should be encouraged and given time towrite in English or Japanese if they feel it will benefit their learning.

7. Students should have the option to maintain a 'private' place to keep their comments,
such as a personal notebook or journal which is not collected by the teacher orshown to other students.

8. Students need models and good examplesespecially in the early stages of selfassessment training. Analyzing models in class maybenefit some students. Students
of all levels may need additional language support to express their thoughts. Suchlanguage support requires lots of practice and should be revisited again and again.
9. Instruments and procedures shouldbe realistic, achievable, and transparent.Students
should have time to discuss materials with each other and with their teacher before writing.

10. Completing self-assessment canbe an individual and a social effort. Students should havethe opportunity to compareand discuss their self-assessment forms with other students(iftheywish to do so).However, students shouldn't be forced to advise group members
to change scores theyperceive to be too highor too low. Grade deflation or inflation
issues should be addressed by the teacher at first, and as students gain experience and get
to knowtheir peers better, it maybe possible to askstudents to feedback on each others'
self-assessment scores. The keyis to tread lightlyhere and provide options.

LD SIG 2003 A n t h o l o g y173* Self Assessment andIn the process of conducting this research, I have gained a number of insights into my own
and my students' and colleagues' understanding ofELP self-assessment procedures. Materials
developers can improve self-assessment instruments byreading books about curriculum
design, subscribing to teaching and research publications, conducting action research, anddiscussing these issues with other teachers. But teachers should also survey students (and otherteachers) and ask them directly what theythink about materials and classroom procedures.
Somewhere, between our experience using self-assessment in the classroom and ourunderstanding of pedagogy and learning principles, we should consider our students' voices
in self-assessment design and implementation. If we're using self-assessment to encourage
students to become autonomous learners, then it simply makes sense that self-assessment
materials and procedures should reflect their input.
Critical Reader Response i

nobuyuki takaki

What is most meaningful about this project is that it presents an 'institutionalized' selfassessment and learner training program at the university level where the faculty'share' thesame syllabus and the same self-created materials. This seems innovative for two reasons: First,
assessment of students' abilities has been traditionally left up to individual teachers, who rarely
communicate among themselves concerning their subject matter teaching (Takaki, 2002);
second, learner training has been traditionally neglected in Japanese universities.

Unfortunately, however, it is not easy to get a good picture of the project's setting, such as
the ratio of full-time vs. part-time and native vs. non-native speaker staff. This is an important
issue because involving part-time staff or teachers with mixed nationalities in such an ELP cancause various conflicts. It would have been also useful if Heidi had specifically discussed theparticular local constraints and conflicts that the faculty faced in implementing their program.

It seems that this researchshould be regarded as a pilot study by the faculty, for the data collected
withinsuch a limitedtime wouldbe unsatisfactory to make suggestions about self-assessment. Why
I say this isthat self-assessment in Japan hasstartedfairly recently in a restricted and superficial
senseat best without contributing much to improving the classroomenvironment.
Heidi has not discussed teachers' reluctance in Japanese school culture to verbalize beliefs
concerning classes, although she has citedsome comments on the project voiced both by
the teachers and the students. It seems to me that Heidi needs to consult more with both thestudents and the colleagues about their views and their changing understandings of'selfassessment.' Moreover, since Heidi claims that the research is both quantitative and qualitative,
she should give follow-up interviews to some of the examinees, citing some of their comments.
Additionally, the gender issue in the survey seems out of focus and premature in her present
paper, as it is briefly mentioned but not developed as a discussion.
One problem with Japanese students is that they have not been sufficiently educated to takeresponsibility in their own learning. A program likeHeidi's should be designed to help themwith their awareness-raising on this point. Thus, the 'Self-Assessment' form (see AppendixAon the Autonomy You Ask! website) should have been handed to the students on Day One of thecourse (if it was not), encouraging them to set their own goals in their learning in the course
before discussing their self-assessment.

It is wonderful, on the other hand, that the students were given an opportunity to discuss
the feedback from the teacher, but, to my disappointment, no example feedback discussions— Autonomy You

Aski

174

*

NACH

are shown. And lastly, I would want Heidi to cite what thefaculty has discussed in theircollaborative reflection sessions, and I would also like to know how Heidi's project has
influenced thestaff's development as teachers, curriculum designers, and textbook writers.
Despite myvarious criticisms, Heidi's project isvery welcome as a possible contribution forfinding innovative ways to evaluate students and teachers in Japan.

Critical Reader Response 2
Stephen H. Brown

Evaluation plays a pivotal role in the development of learner autonomy. The functionof evaluation is on the one hand to ensure that work undertaken is discussed andrevised, and on the other to establish a basis of experience and awareness that can beused in planning further learning. (Dam, 1995,p. 49)

Heidi Evans Nachi'sfocus on self-assessment touches on a keyarea of autonomous learningin practice, asking some important questions about the attitudes of students and teachers.

One of the things that struck me throughout Heidi's chapter was the barely-concealedtension stemming from the dual function of self-assessment in the ELP (the programme she
describes), shared with many educational institutions worldwide. On the one hand, there isself-assessment as an instrument of Assessment (with a capital A, i.e., grade generation for
official purposes); on the other, self-assessment is an important pedagogical tool—what I
prefer to call, in common with Dam above, (self-)evaluation, (i.e., a tool for reflection on what
has passed and for futureplanning). This second function is an essential element in fostering
autonomy, evaluating what has been done in order to plan what comes next in the learning
cycle. The challenge for a programme like the ELP is to find a way of reconciling this with theadministrative, grade-generating function.
The results of Heidi's survey seem to suggest that her students understand the intended
pedagogical function of self-assessment in their programme and value it, judging by the
numbers responding with comments such as "I can look back at myself" and "I can improvefor next class" to the question What doyou like about self-assessment? The concerns they haveseem to be related to their awareness of the other, administrative function (e.g., 35 responsesof "Students lie and give themselves higher scores than they deserve"). The danger is that theseconcerns might impinge upon their view of the whole evaluation process.
One of the potential effects of anygrading system isthat learners areinclined to think horizontally("Howam I doingcompared to the others?"), ratherthan vertically ("Where am I/wenow compared
to whereI/wewereone month ago, or whereI/wewant to be at the end of the semester?")—whichis really howwewant our learners to be thinking. In the real world, studentsare concerned with howothers sees them, with theirperformance in relation to other learners—a natural human concern,and one which is also compounded by their previouseducationalexperience.
Perhaps it'spossible to separate the two functions and have the reflection / planningevaluationin the form of,say, worksheets, diaries or groupreviews (something I'vetried in my own classes,
encouraging students to focus on theirprogress, rather than on the final grade). The self-assessment
to generategradeswould then be a separate process—still subject to the same inclinationto
compare with others,perhaps, but tire process of reflective evaluation wouldbe better protected.
And that might offera better environmentto "include our students'voices" in the process.
— LD SIG 2003 A n t h o l o g y —