The Japanese Skills Test as an On‐Demand Placement Test: Validity Comparisons and Reliability

Abstract: This study and analysis are designed to find a common, practical measurement tool that may be used for intra- and interprogram evaluation of Japanese programs, especially in the higher education context. First, three previous methods of evaluation (the Oral Proficiency Interview, the Nihongo Noryoku Shiken or Japanese Language Proficiency Test, and the Simple Performance-Oriented Test) are analyzed for strengths and weaknesses as common, practical measurement tools. The Japanese Skills Test (JSKIT) is suggested as superior to these three, especially in distinguishing first- and second-year ability in the language. The JSKIT is evaluated on the basis of being both valid and reliable as a test. This is supported by data collected at a nine-week summer intensive language program during the summers of 2000 and 2001, at entry and exit points. Key words: articulation, placement, testing, validation Language: Japanese (ProQuest: ... denotes non-USASCII text omitted.) Introduction As the number of learners of Japanese in the United States increases, the need for a common, practical measurement tool to compare abilities and skills of learners from different programs and institutions is becoming critical. According to the Moderna Language Association, more than 66,000 students were studying Japanese in a higher education setting in the fall of 2006, when Japanese was ranked as the sixth most commonly taught language, following Spanish, French, German, American Sign Language, and Italian (Furman, Goldberg, & Lusin, 2007). With increased enrollment in Japanese language courses and a growing variety of program types and curricula, instructors of Japanese face greater challenges when placing students who move from one program to another (Wetzel, 1997). A recent survey of college-level East Asian language programs indicates that only 19% of instructors use a standardized test for assessment and/or placement purposes, while 36% of department heads say their institutions use standardized examinations (Walker & Li, 2002). Japanese language professionals rightly are concerned with the articulation between high school and college programs, and so the Japanese Skills Test QSKIT) was developed to meet the need for a reliable, cost-effective, and easily administered assessment tool that can be used to place learners within a program and evaluate learner outcomes in programs at different institutions.1 The present study describes this test and reports on a study to examine the validity and reliability of the JSKIT, Part 1, by analyzing data obtained from participants in a large-scale, multilevel summer intensive Japanese language program in the United States. This article is divided into four sections. In Section 1, three tests commonly used in the United States to measure learner proficiency in Japanese-the ACTFL Oral Proficiency Interview (OPI), Nihongo Noryoku Shiken (or Japanese Language Proficiency Test, JLPT), and Simple Performance-Oriented Test (SPOT)-are reviewed in terms of their usability as placement tests. Section 2 describes the JSKIT. Section 3 describes the procedures and participants of the reliability and validity studies of the JSKIT as a placement test. The last section discusses the findings and limitations of the study and describes future directions for the development of the JSKIT. Common Proficiency Tests in Japanese: The OPI, Nihongo Noryoku Shiken, and SPOT and Their Suitability as Placement Tests Currently, a handful of widely recognized proficiency tests in Japanese are available in the United States. These include the OPI developed by ACTFL, the Simulated Oral Proficiency Interview (SOPl) available from the Center for Applied Linguistics, the Nihongo Noryoku Shiken (or JLPT) offered by the Japan Foundation, the SAT II Subject Test in Japanese administered by the Educational Testing Service, the Standardsbased Measurement of Proficiency from the University of Oregon's Center for Applied second Language Studies, and the SPOT created at the University of Tsukuba. …