Empirical Validation of Listening Proficiency Guidelines.

Abstract: Because listening has received little attention and the validation of ability scales describing multidimensional skills is always challenging, this study applied a multistage, criterion-referenced approach that used a framework of aligned audio passages and listening tasks to explore the validity of the ACTFL and related listening proficiency guidelines. Rasch measurement and statistical analyses of data generated in seven separate language studies resulted in significant differences in listening difficulty between the proficiency levels tested and confirmed the validity of the ACTFL proficiency assessment for listening.Key words: ACTFL Proficiency Guidelines, multistage assessment, proficient listening, scale validation, testing listeningIntroductionThe Interagency Language Roundtable (ILR) language proficiency scales (2012) grew out of the U.S. government's pragmatic need to appropriately assign language learners with varying ability levels to jobs of varying linguistic difficulty. By using a standard set of ability descriptions, the personnel office could thus make overseas assignments and know that a Level 1 speaker (Intermediate) would have sufficient language skills to survive in the target culture, a Level 2 (Advanced) speaker would have limited working proficiency and could be employed in areas in which he or she would be interacting with the target culture, and a Level 3 (Superior) would be able to function professionally with educated speakers in the target culture. One of the characteristics of the scales was to functionally describe, in a way that laypeople could understand, what language learners could do in real-world contexts.This practical view to language learning and eventual language use led to the first widespread deployment of speaking assessments that were used to match civil servants with various assignments throughout the world. However, language consists of more than speaking, and it soon became apparent that there was a need to describe language use for the other skills, particularly because progress toward native? like proficiency in listening does not always progress in the same way and at the same rate. For example, in situations where learn- ers have rehearsed the language they may need, they can produce utterances that they would not necessarily understand when lis- tening to speakers using natural language in the target culture. Similarly, in other situa- tions, learners can understand language at a much higher level than the speech they can produce. Thus, while closely related to pro- ficiency scales and assessments for speaking, the listening proficiency scale was devel- oped to describe listener functions apart from speaking in a hierarchical, criterion? referenced manner. The ability to measure listening has important implications ranging from the pragmatic need to monitor com- munication for national security to the lofty ideal of understanding another culture as goodwill ambassadors.However, the listening proficiency scale, along with other proficiency scales, has long been under attack in academia. The genesis of these criticisms can be found in the fact that the scales were never in- tended to model language acquisition, are agnostic in their view of language theories, and focus instead on real?world functional language use. Thus, academics intent on developing assessments that operationalize theoretical constructs of acquisition have concluded that the scales violate a tenet of sound assessment practice. Namely, they believe that not having a theoretical frame- work of language learning that defines a language ability construct a priori to con- structing a test weakens any validity claims the resulting scale might have (Bachman, 1990; Bachman & Palmer, 1996). Even those sympathetic to using the proficiency scale with the productive skills of speaking and writing may harbor some hesitations in applying the scales to the receptive skills. …