Qualitative and Quantitative Measures of Second Language Writing: Potential Outcomes of Informal Target Language Learning Abroad

Abstract: This research describes a method applied at a U.S. university in a thirdyear Russian language course designed to facilitate Advanced and Superior second language writing proficiency through the forum of argumentation and debate. Participants had extensive informal language experience living in a Russian-speaking country but comparatively little formal language instruction, especially with regard to writing. The authors quantitatively analyzed weekly position papers and ACTFL pre- and post- Written Proficiency Test compositions using prescribed complexity measures and compared them with qualitative ACTFL proficiency ratings. Preliminary findings underscore the value to both the writer and instructor of applying both quantitative and qualitative measures; they likewise point out challenges associated with teaching students whose language has become automatized, albeit oftentimes error-full, in part owing to the nature of past language study. Key words: Russian, content-based instruction, curriculum development, second language acquisition, teaching methods, writing proficiency Introduction Matters of oral proficiency have received considerable attention in second language acquisition research, including the effects of study abroad on facilitating and even expediting language uptake. Less examined in the literature, nevertheless noteworthy of attention, is the area of second language (L2) writing, particularly in regard to assessment measures. While oral skills typically exceed other language skills at the ACTFL Novice and Intermediate levels, writing skills become increasingly important as one progresses to the Advanced level and beyond, in part owing to domain-specific expectations associated with such levels. Yet for many, written production never goes beyond the ''speech-written-down style of writing,'' which entails a degree of ''differentiation,'' i.e., a style of language unique from oral production in its syntactic complexity and increased usage of subordinated and embedded clauses (Weissberg, 2006, p. 10). If viewed from the perspective of the ACTFL Written Proficiency Test (WPT) Rating Grid by Sub-Level, then such a process of differentiation becomes particularly evident at the Intermediate-High/Advanced-Low threshold where writers are expected to demonstrate with increasing accuracy and consistency the ability to connect sentences into paragraphs using cohesive elements and devices. Sentence cohesion and paragraph-level discourse become increasingly sophisticated and consistent as one progresses stepwise within the Advanced level, thus contributing to a complexity that distinguishes itself from oral communication (see ACTFL, 2001). One method used for assessing writing proficiency involves counting the total number of connectors per total number of words in a composition. Such measures provide fine-grained, quantitative analyses that, when coupled with qualitative descriptors associated with the holistic ACTFL rating scale, stand to enhance overall rater objectivity. In essence, the WPT can serve as a valuable set of bookends at the beginning and end of a course; however, the gradual and often subtle language gain that transpires in-between those assessments likewise calls for an instrument capable of detecting nuanced degrees of change that help explain general movement in proficiency, e.g., participants' use of transition words, connectors, coordinate, participial, and adverbial clauses. This article describes one method of facilitating foreign language uptake in writing applied at a U.S. university in a third-year Russian language course comprising students with extensive informal language experience in countries of the former Soviet Union but with comparatively little exposure to the formal elements of the language, in particular its writing system. The course was designed to push students' written proficiency from the Intermediate and Advanced levels to the Advanced and Superior levels, respectively. …