Accuracy in Speaking Self-Assessment among Japanese-speaking English Learners and Its Implications

This paper investigates the accuracy of self-assessment of speaking proficiency among undergraduates at a university in Japan. The value and utility of criterion-referenced self-assessment is discussed in light of dominant cultural discourses emphasising the importance of modesty, moderation and group harmony, which encourages Japanese learners of English to self-assess towards norms of “average” or “below average”. The importance of criterion-referencing is discussed, and limitations in the data and criteria used in the study are explored. The self-assessment protocol is described, and the results of the self-assessments are compared with scores from assessments by trained and experienced raters conducting simulated IELTS speaking tests. The reliability of these expert assessments is explored using a measure of inter-rater reliability described in the paper. The study finds that the assessments were within the boundaries for inter-rater reliability in 10 of 11 cases. The data also suggests, in line with the literature, that self-assessment can be reliable – 8 out of 10 completed self-assessments were within the same boundaries – but becomes problematic at lower proficiency levels due to increasing inaccuracy. The implications of accurate self-assessment are discussed, consideration is given to how knowledge of one's own competences might interact with assessment and motivation, and the practical, affective and ethical ramifications of self-assessment and the challenges it presents to teachers, learners and curricula are examined. The paper concludes with the observation that further investigation of the reliability, scope and purpose of assessment is necessary. Key terms: assessment, self-assessment, inter-rater reliability, criterion-referencing, IELTS, self-regulated learning Literature Review The ability to perform accurate self-assessment is a core component of self-regulated learning (Clark, 2012), which may be more familiar to English teachers as learner autonomy. Under models of self-regulated learning, self-assessment is one strand of feedback in a model of formative assessment along with feedback from both peers and teachers. Teachers are well placed to help learners who know and understand their own abilities, the goals of the learning encounter they are engaged in, and hold positive beliefs about their self-efficacy – the belief that they can take actions which will contribute to a positive outcome (Gallagher, 2012, p. 314). Previous studies, such as that by Munoz and Albarez (2007) and the meta-analysis by Blanche and Merino (1989) indicate that self-assessment is positively linked to motivation, and that learners can self-assess with relative accuracy. However, learners both overand underestimate their skills. Additionally, accuracy varies positively with proficiency, and the self-assessment tools used require careful design. Culture may play a significant confounding role in such studies with Japanese learners, as dominant discourses in Japanese culture emphasise modesty (謙遜 – kenson), moderation (控えめ – hikaeme) (Brown, 2004) and group harmony (和 – wa). These have been shown to encourage learners – regardless of their actual ability – to report their ability to learn a foreign language as “average” or “below average” (Brown, 2004). To appropriate the Japanese proverb, the nail that sticks up will hammer itself down. Two additional concepts further complicate discussion of norm-referenced self-assessment in the Japanese context, and are related to wa: Honne (本音) – behaving in ways and saying things that represent one's actual beliefs, dispositions and attitudes; and tatemae (建前) – behaving in ways and saying things that one believes to be appropriate in the particular social circumstance. These are particularly troubling concepts in the investigation of self-assessment, since the decision to say what a person believes is expected (rather than proffer their real opinion) is a private one, and seems to be linked to avoiding disapprobation rather than any misalignment between self-perception and reality (Yamagishi et al., 2012). Therefore, establishing the veracity of participants' reports in such studies is dependent upon an unverifiable factor. Unfortunately, a lengthier discussion of cultural variation in the