Outcomes Assessment for Beginning and Intermediate Spanish: One Program's Process and Results

Abstract: Outcomes assessment is a process by which an academic unit defines and articulates its program goals and assesses its attainment of those goals. This article chronicles one language division's efforts at outcomes assessment for beginning and intermediate Spanish. The evidence used for program assessment consisted of WebCAPE placement scores, student satisfaction surveys, learner portfolios, and oral proficiency interviews (OPIs). The results of the project suggest that the target goals for student progress were being met and that students were generally satisfied that the program met its stated goals. The results also suggest areas of improvement to the means by which outcomes are measured as well as areas of improvement to the curriculum. Key words: intermediate Spanish, oral proficiency, outcomes assessment, portfolio assessment, written proficiency Language: Spanish Introduction Outcomes assessment is a multilayered process by which an academic unit defines and articulates its program goals and assesses its attainment of those goals. The articulation of program goals takes place not only among the faculty within the academic unit, but also, potentially, prepares the unit for accreditation review. In order to assess the effectiveness of a program, it must be established whether the programs stated goals are being met. If the program's goals require development, then outcomes assessment must begin with articulation of these goals. Only then may the academic unit begin to address issues of the adequacy of instruments for evaluating student performance and achievement. This article chronicles one language division's efforts to reexamine goal statements for the purpose of articulation and outcomes assessment. The core language requirement at the Saint Louis University consists of a three-semester sequence. In Spanish, these courses consist of two semesters of beginning study (Spanish 110 and 115) and one semester of intermediate (Spanish 210). All of these courses meet three times per week with a 1-hour lab. Early efforts at outcomes assessment focused on an analysis of Spanish Computer Adaptive Placement Exam (SCAPE) data (Brigham Young University, 2000-2001), a survey of factors influencing student decisions, and a correlation of course grades with SCAPE scores. In 1997, ACTFL proficiency levels were identified and assessment practices were established for intermediate and advanced levels. For Spanish 210, Intermediate-Mid was the target proficiency level identified for oral work and Intermediate-High was identified as the target level for written work. Assessment data consisted of learner portfolios and oral proficiency interviews (OPIs). These levels were chosen in a departmental effort to articulate target proficiency levels before the present project was undertaken. Then, in fall 2000, a more comprehensive and systematic approach to outcomes assessment was initiated. This later stage of the Spanish Division's efforts is the focus of this article. Review of the Literature Some scholars have expressed reservations about the limitations of the ACTFL Proficiency Guidelines (ACTFL, 1986). Bachman and Savignon (1986), for example, pointed out that the guidelines are based on an overly narrow view of proficiency and have questioned their applicability to the academic setting. Wherritt (1990) summarized the criticism of the OPI for the limited types of discourse it elicits. Although the ACTFL Guidelines have several limitations, they are, as Lee and VanPatten (2003) indicated, useful for establishing rough program goals. As Clifford (2003) indicated, proficiency testing using the ACTFL Guidelines is "at best a macro-diagnostic assessment and determines only 'what' types of communication tasks can or cannot be accomplished, in what types of contexts, at a specified level of accuracy" (p. 481). Gradman and Reed (1997) also acknowledged the utility of adopted ACTFL Proficiency Guidelines: The view of "proficiency" offered in [ACTFL listening, reading, and writing] tests emphasizes not what an individual knows about the target language (e. …