# The Assessment of Written Phrasal Constructs and Grammar of Deaf and Hard of Hearing Students with Varying Expressive Language Abilities

Authors: Lisa M. Bowers; Hannah Dostal; Kimberly A. Wolbers; Shannon C. Graham
Journal: Education Research International
 (2018)

---

## Abstract

The purpose of this study was to examine the written phrasal constructs and grammar usage of deaf and hard of hearing students with varying expressive language skills. Twenty-nine d/hh middle school students attending a residential school for the deaf were divided into three language groups: students using spoken English, ASL/English bilinguals, and language delayed learners. Personal narrative writing samples were collected at the beginning, middle, and end of the academic year. The samples were divided into T-units and coded for language variables, including word efficiency ratio (WER) scores according to the Structural Analysis of Written Language (SAWL) and phrasal errors. The repeated measures ANOVA for WER III showed a statistically significant main effect with no between-subjects factor, demonstrating that students from all three language groups made positive gains in their written outcomes over one academic year. There was a reduction in phrasal errors over the course of the year for all language groups. Differences in word efficiency ratio scores by language groups are discussed. Findings from this study suggest that SAWL is an effective tool in assessing the grammaticality of written compositions for d/hh students with varying language abilities over time. Instructional implications are discussed.


---

## Body

## 1. Introduction

Deaf and hard of hearing (d/hh) students are a heterogeneous group that present with language and literacy abilities both unique to this population and to each individual. While each child differs in the severity of hearing loss, type of amplification, modality, and communication philosophy, overall, children with hearing loss will face challenges in acquiring language. When a child with hearing loss has limited access to language, deficits in linguistic competence can directly influence areas of reading and writing development [1]. While the purpose of this paper is to examine d/hh middle school students’ use of incoherent structures and agrammatical elements in written expression, it is important to first explore the relationship between language acquisition, linguistic competence, and literacy skills.Models of literacy acquisition emphasize the integral relationship between linguistic proficiency, language comprehension, and literacy skills, as many oral language skills contribute to the literary abilities of children with typical hearing [2]. Children with early and immediate access to language, either visually or orally, begin at birth to acquire the skills needed to eventually understand and use language to communicate. Language skills then provide a foundation for the process of learning how to read and write, with the implication that children entering school with weak or delayed language abilities are more likely to experience deficits in literacy [3]. Linguistic competence, or our implicit knowledge of language rules, is acquired with exposure to and use of language. While most monolingual children with typical hearing primarily use auditory information to acquire language and linguistic competence, there are a number of diverse pathways for d/hh children to learn language.It is important to understand how young d/hh children access language, as a child’s preschool language and vocabulary skills have been shown to correlate with later reading comprehension [4, 5]. Access to language for d/hh infants and children varies widely, as do the communication modality approaches utilized with d/hh children and their families. For example, recent advances in hearing loss identification, technology, and early intervention have provided easier access to listening and spoken language for many d/hh infants and children [6]. Early intervention models often emphasize the use of appropriate hearing aid amplification and/or cochlear implantation, so infants and children can learn to listen to sounds and use spoken language [7] and, with more regularity, parents of d/hh children are choosing to participate in communication approaches that incorporate oral communication [8]. However, even with the great strides that have been made with early identification, technology, and listening/spoken language approaches [9, 10], some d/hh children that use spoken English and hearing technology continue to lag behind their peers in vocabulary, reading, and writing knowledge [11–13].Another language option for d/hh children is American Sign Language (ASL), a visual language that has its own unique syntax and grammatical markers [14, 15]. A subgroup of children who are deaf and have full and complete access to a visual language such as ASL at birth achieve language milestones that are parallel to those of hearing children acquiring spoken language [16, 17]. Even though this subgroup of deaf children may have limited access to English, ASL proficiency is significantly correlated with high literacy attainment in English [18], including written narratives [19]. Thus, fluent and frequent exposure to and use of ASL can provide another route to English for language and literacy purposes; however, only a limited number of deaf children are exposed to ASL at a young age, especially with a parent fluent in ASL [20]. Given the potential for limited language accessibility regardless of communication approach [21], it is no surprise that d/hh children as a group often demonstrate delays in language acquisition [22, 23] and literacy outcomes [24, 25].Although there is a small corpus of research suggesting that the linguistic proficiency of d/hh students may influence the unique characteristics found in their writing [13, 18], there are only a few assessment options that specifically target written composition reported in the literature. Norm-referenced standardized tests are often used to assess the writing skills of d/hh students [26]; however, the practice of evaluating the writing outcomes of d/hh students by these standardized assessments alone has been criticized, as they may not capture the unique strengths, weaknesses, and characteristics of d/hh students’ writing [27].One example of an alternative method of measuring writing assessment was presented by Musselman and Szanto [28]. Their study demonstrated that spontaneous writing samples may elicit higher performance levels and provide more diagnostically relevant information for d/hh students than norm-referenced assessments. Writing skills of 69 d/hh adolescents (mean age = 16.3 years; mean unaided hearing levels = 110 dB) were assessed using both the Test of Written Language, Second Edition (TOWL-2) [29] and a written letter. The students were further divided into two language groups: Auditory/Oral (A/O; N=15) and Signing N=54. Both the TOWL-2 and the letter writing assessment were scored using a linguistic scoring system, including holistic score, number of words, number of T-units (defined as an independent clause and all subordinate clauses), number of different words, type-token ratio, and mean words per T-unit. Findings demonstrated a significant difference by communication mode, with students in the A/O group demonstrating significantly higher scores on the TOWL-2 and letter holistic scores, number of words, and number of different words. Students from both language groups obtained higher scores on the letter writing task than the TOWL-2 subtest and demonstrated more variability within the group.The Structural Analysis of Written Language (SAWL) [30] is another alternative assessment measure designed to analyze the use of appropriate grammar of written language of d/hh writers over time. The purpose of SAWL is to objectively evaluate the written language improvement of d/hh students by analyzing the grammaticality of students’ T-units. Instead of a traditional “correct”/“incorrect” scoring system utilized by most standardized assessments, T-units are coded as perfect, flawed, or perfect word strings within T-units, to show progression. Thus, a perfect T-unit (e.g., I played some games) contains zero language errors. A flawed T-unit (e.g., We got snack and drink) contains all the main critical structures but has one or more surface errors. Errors that render a T-unit flawed include tense errors, subject-verb agreement, article error, plurality errors, and incorrect prepositions. If the T-unit is missing a critical structure (e.g., subject, main verb construction, and preposition), then it qualifies as neither perfect nor flawed. Within these T-units, strings of three or more perfect words are given credit (e.g., Then my mom). Preliminary studies reported by White [30] found adequate construct validity, interscorer reliability, and test-retest reliability with reasonable scoring time. Overall, the SAWL presents as an objective measure of written expression that was designed to be sensitive to gradual changes over time in students’ writing samples.Towards the goal of writing assessments documenting change over time as opposed to just a static measure of writing ability (i.e., standardized tests), it is important to consider the instructional approach utilized to improve written composition. Strategic and Interactive Writing Instruction (SIWI) is one writing approach specifically designed to improve the writing outcomes of d/hh children. SIWI draws upon several evidence-based practices in writing instruction, such as the incorporation of strategy instruction for writing process and skill development, the writing of authentic and purposeful pieces, and the use of model texts [31]. Two specific aspects of SIWI believed to contribute to students’ English and/or ASL linguistic competence are (1) interactive writing and (2) the language zone.d/hh students who participate in SIWI benefit linguistically from being conversational partners during meaningful writing activities [32]. Interactive writing, as well as other forms of dialogic pedagogy, has the potential to promote greater linguistic competence with the language of instruction, whether English or ASL, among d/hh students. During SIWI, students often engage in group guided and interactive writing activities to gain experience with new skills or new text types and then slowly transition to more independent writing as they grow in confidence and knowledge. Group guided writing is inherently interactive, meaning students must communicate with one another to make decisions about planning, writing, editing, and publishing. This support is provided in either English or ASL depending on the linguistic needs of the child.Secondly, during SIWI, teachers and students often work within a space separate from the board where they are writing English called the “language zone.” This space is where participants clarify or complicate expressions or raise metalinguistic awareness of languages. Understandably, interactive writing can present challenges when students do not have competency in either English or ASL to communicate their ideas clearly and effectively. Drawing on techniques that have long been used with d/hh children for repairing expressive or receptive communication breakdowns [33], there are procedures within SIWI such as role play, drawing, using objects/pictures, and gesturing that can be used to ensure students understand their peers and the teacher. As is often the case with children who are delayed in developing language in English and/or ASL, contributions offered during guided writing may be incomplete or incomprehensible. Over time, students grow in their abilities to express themselves more clearly and with greater complexity.In summary, d/hh students comprise a heterogeneous group with various linguistic abilities. Differences in linguistic ability make the assessment and instruction of writing outcomes difficult for practitioners and related professionals that work with d/hh students, particularly with the limited functional use of standardized assessment scores. The current study is an attempt to evaluate the clarity and grammaticality of written English expressions from middle school d/hh students with varying expressive language abilities using an alternative assessment, the SAWL, over one academic year in which students participated in SIWI. Additional patterns of linguistic growth in written composition, including unintelligible phrasal constructions, total number of words, and total number of unique words, were also analyzed. The two research questions guiding the present study were as follows:When assessing the writing skills of d/hh students with varying language abilities,(1)
to what extent are different patterns of grammatical growth captured by the SAWL in perfect T-units (word efficiency ratio (WER) I), perfect and flawed T-units (WER II), and perfect word strings (WER III)?(2)
to what extent do students demonstrate differences in patterns of linguistic growth, as measured by unintelligible phrasal constructions (i.e., phrasal errors (PE)), total number of words, and total number of unique words?


## 2. Methods

### 2.1. Participants

Twenty-nine d/hh middle school students in grades 6–8 (mean age = 13.2) attending a residential school for the deaf in the United States participated in the study. All students were taught by one primary classroom teacher. In collaboration with the classroom teacher, the researchers divided the student participants into language groups by their mode of communication and observed their expressive linguistic competence. These groups included (1) students who primarily used spoken English to communicate, (2) developing bilinguals who used ASL and written English, and (3) language delayed learners (i.e., students experiencing language delays with no clear primary language). Students who consistently used English as their primary receptive and expressive language were assigned to theEnglish group N=13. While students in the English group used speech as their primary form of communication, some students would also periodically or regularly supplement unintelligible speech with manually coded signs. Students who demonstrated appropriate ASL grammar and/or incorporated ASL features and signs into their daily communication were considered to be developing ASL-English bilinguals N=9. Lastly, students whose signed or spoken expressions were unintelligible due to the use of fragmented and nonsensical productions were considered to be language delayed learners (LDL) in both English and ASL N=7. Placement of students into these language groups was finalized after a two-phase review, which first included an initial analysis of student language histories as they related to the three expressive language categories presented above. Then, the researchers reviewed videotaped classroom instruction and individual student interviews to cross-check the groupings established by the teacher. Demographic data by full group and language groups including unaided hearing threshold and aided hearing threshold are provided in Table 1. It important to note the simultaneous communication (sim-com), or signing and using spoken language at the same time, was the communication methodology utilized at the school. Thus, students in all three groups had exposure to both spoken English and sign in the academic setting. Additionally, most of the students in all language groups were exposed to ASL in the residential school environment.Table 1
Student demographics by full group and language groups.
Full group
English
Bilingual
LDL
N
29
13
9
7
Unaided hearing
88 dB
80 dB
98 dB
89 dB
Aided hearing
35 dB
38 dB
35 dB
32 dB


### 2.2. Writing Samples: Data Collection and Scoring

Personal narrative writing samples were collected at the beginning, middle, and end of the academic year for a total of 87 writing samples. All students were read a prompt that asked them to write about a true event that happened in the past (e.g., visiting a special place and what they did over the break). Students were given no time limit and no assistance during the writing process. The researchers segmented the writing samples into T-units [34]. Two research members double coded 20% of the samples for an interrater agreement of 0.86. Data were collected for number of words and number of unique vocabulary words for all writing samples.


### 2.3. Perfect and Flawed Phrasal Constructions: Structural Analysis of Written Language (SAWL) Scoring

The writing samples were also scored for word efficiency ratio (WER) according to the SAWL [30]. WER I is the total number of words in perfect T-units divided by the total word count. WER II is the total number of words in perfect and flawed T-units divided by the total word count. WER III is the total number of words in perfect and flawed T-units and perfect word strings divided by the total word count. Approximately 5% of the samples were coded by all research members for training purposes. Differences in coding were discussed, and agreed upon codes were included in the analyses. An additional 25% of the samples were coded in rotating pairs. Averages were included in the final analysis. Pearson’s correlation for the interrater agreement was 0.97.


### 2.4. Unintelligible Phrasal Constructions: Phrasal Error (PE) Scoring

Three native English-speaking researchers (two who were also fluent in ASL) identified confused statements that were deemed unintelligible in both English and ASL and coded the statements as phrasal errors. A consensus was reached on all phrasal error codes. An example of a phrasal error isCat you ah is so is pay so is so you is you yes is pay is yes.


### 2.5. Writing Approach

Three personal narrative writing samples were collected from all participants over one academic year. During this time, students participated in SIWI, a framework for approaching writing instruction with d/hh students. The researcher observed the teacher’s instruction seven different times throughout the year. For purposes of measuring instructional fidelity, the teacher’s use of 27 actionable principles was measured on a 4-point rubric scale. The teacher’s scores ranged from 3.809 to 4.0 per observation which shows her consistency in demonstrating the instructional principles associated with SIWI.


### 2.6. Data Analysis

Operational definitions of all measurement data and examples are included in Table2. The analysis of SAWL data involved a repeated measures ANOVA with a between-subjects factor to investigate the presence of perfect and flawed grammar in writing and to examine outcomes among students in different language groups. The within-subjects variable was time (e.g., WER I, WER II, and WER III at the beginning, middle, and end of the year samples), and the between-subjects factor was language group (i.e., English, bilingual, and language delayed learners). Additionally, there were repeated measures ANOVA and descriptive statistics for number of T-units, phrasal errors (PEs), number of words, and number of unique words for the beginning, middle, and end of the academic year samples by language group.Table 2
Assessment measures, definitions, and examples.
Assessment measure
Definition
Example
T-unit
An independent clause and all subordinate clauses
The boy is running
WER I
Perfect T-unit: no grammatical errors
The boy is running
WER II
Flawed T-unit: contains all the main critical structures but has one or more surface errors (e.g., tense errors, subject-verb agreement, article error, plurality errors, and incorrect prepositions)
The boy is run
WER III
Perfect word string: 3+ consecutive words within a T-unit not deemedperfect or flawed
The boy is running an girl (4-word perfect word string)
Phrasal error (PE)
Confused statements that were unintelligible in English or ASL
Cat you ah is so is pay so is you is you yes is pay yes
Total # of words
All words in the writing sample
The boy is running to the store (7 words)
Total # of unique words
All unduplicated words in a writing sample (includes base, derived, and inflected words)
The boy is running to the store (6 unique words)


## 2.1. Participants

Twenty-nine d/hh middle school students in grades 6–8 (mean age = 13.2) attending a residential school for the deaf in the United States participated in the study. All students were taught by one primary classroom teacher. In collaboration with the classroom teacher, the researchers divided the student participants into language groups by their mode of communication and observed their expressive linguistic competence. These groups included (1) students who primarily used spoken English to communicate, (2) developing bilinguals who used ASL and written English, and (3) language delayed learners (i.e., students experiencing language delays with no clear primary language). Students who consistently used English as their primary receptive and expressive language were assigned to theEnglish group N=13. While students in the English group used speech as their primary form of communication, some students would also periodically or regularly supplement unintelligible speech with manually coded signs. Students who demonstrated appropriate ASL grammar and/or incorporated ASL features and signs into their daily communication were considered to be developing ASL-English bilinguals N=9. Lastly, students whose signed or spoken expressions were unintelligible due to the use of fragmented and nonsensical productions were considered to be language delayed learners (LDL) in both English and ASL N=7. Placement of students into these language groups was finalized after a two-phase review, which first included an initial analysis of student language histories as they related to the three expressive language categories presented above. Then, the researchers reviewed videotaped classroom instruction and individual student interviews to cross-check the groupings established by the teacher. Demographic data by full group and language groups including unaided hearing threshold and aided hearing threshold are provided in Table 1. It important to note the simultaneous communication (sim-com), or signing and using spoken language at the same time, was the communication methodology utilized at the school. Thus, students in all three groups had exposure to both spoken English and sign in the academic setting. Additionally, most of the students in all language groups were exposed to ASL in the residential school environment.Table 1
Student demographics by full group and language groups.
Full group
English
Bilingual
LDL
N
29
13
9
7
Unaided hearing
88 dB
80 dB
98 dB
89 dB
Aided hearing
35 dB
38 dB
35 dB
32 dB


## 2.2. Writing Samples: Data Collection and Scoring

Personal narrative writing samples were collected at the beginning, middle, and end of the academic year for a total of 87 writing samples. All students were read a prompt that asked them to write about a true event that happened in the past (e.g., visiting a special place and what they did over the break). Students were given no time limit and no assistance during the writing process. The researchers segmented the writing samples into T-units [34]. Two research members double coded 20% of the samples for an interrater agreement of 0.86. Data were collected for number of words and number of unique vocabulary words for all writing samples.


## 2.3. Perfect and Flawed Phrasal Constructions: Structural Analysis of Written Language (SAWL) Scoring

The writing samples were also scored for word efficiency ratio (WER) according to the SAWL [30]. WER I is the total number of words in perfect T-units divided by the total word count. WER II is the total number of words in perfect and flawed T-units divided by the total word count. WER III is the total number of words in perfect and flawed T-units and perfect word strings divided by the total word count. Approximately 5% of the samples were coded by all research members for training purposes. Differences in coding were discussed, and agreed upon codes were included in the analyses. An additional 25% of the samples were coded in rotating pairs. Averages were included in the final analysis. Pearson’s correlation for the interrater agreement was 0.97.


## 2.4. Unintelligible Phrasal Constructions: Phrasal Error (PE) Scoring

Three native English-speaking researchers (two who were also fluent in ASL) identified confused statements that were deemed unintelligible in both English and ASL and coded the statements as phrasal errors. A consensus was reached on all phrasal error codes. An example of a phrasal error isCat you ah is so is pay so is so you is you yes is pay is yes.


## 2.5. Writing Approach

Three personal narrative writing samples were collected from all participants over one academic year. During this time, students participated in SIWI, a framework for approaching writing instruction with d/hh students. The researcher observed the teacher’s instruction seven different times throughout the year. For purposes of measuring instructional fidelity, the teacher’s use of 27 actionable principles was measured on a 4-point rubric scale. The teacher’s scores ranged from 3.809 to 4.0 per observation which shows her consistency in demonstrating the instructional principles associated with SIWI.


## 2.6. Data Analysis

Operational definitions of all measurement data and examples are included in Table2. The analysis of SAWL data involved a repeated measures ANOVA with a between-subjects factor to investigate the presence of perfect and flawed grammar in writing and to examine outcomes among students in different language groups. The within-subjects variable was time (e.g., WER I, WER II, and WER III at the beginning, middle, and end of the year samples), and the between-subjects factor was language group (i.e., English, bilingual, and language delayed learners). Additionally, there were repeated measures ANOVA and descriptive statistics for number of T-units, phrasal errors (PEs), number of words, and number of unique words for the beginning, middle, and end of the academic year samples by language group.Table 2
Assessment measures, definitions, and examples.
Assessment measure
Definition
Example
T-unit
An independent clause and all subordinate clauses
The boy is running
WER I
Perfect T-unit: no grammatical errors
The boy is running
WER II
Flawed T-unit: contains all the main critical structures but has one or more surface errors (e.g., tense errors, subject-verb agreement, article error, plurality errors, and incorrect prepositions)
The boy is run
WER III
Perfect word string: 3+ consecutive words within a T-unit not deemedperfect or flawed
The boy is running an girl (4-word perfect word string)
Phrasal error (PE)
Confused statements that were unintelligible in English or ASL
Cat you ah is so is pay so is you is you yes is pay yes
Total # of words
All words in the writing sample
The boy is running to the store (7 words)
Total # of unique words
All unduplicated words in a writing sample (includes base, derived, and inflected words)
The boy is running to the store (6 unique words)


## 3. Results

### 3.1. Research Question 1: Perfect and Flawed Phrasal Constructions—SAWL

Descriptive statistics for WER I, WER II, and WER III for the whole group and by language groups are provided in Table3. Results of the repeated measures ANOVA for WER I, or total number of words in perfect T-units divided by the total word count, demonstrated a nonsignificant main effect (F(2, 52) = 1.68, p=0.2). The repeated measures ANOVA for WER II, or the total number of words in perfect and flawed T-units divided by the total word count, also demonstrated a nonsignificant main effect (F(2, 52) = 1.87, p=0.16). Sphericity cannot be assumed for WER III, or the total number of words in perfect and flawed T-units and perfect word strings divided by the total word count; therefore, the Greenhouse-Geisser correction procedure was used. The repeated measures ANOVA for WER III showed a statistically significant main effect: F(1.54, 40.01) = 6.989, p<0.01, and partial η2 = 0.212. Effect size is provided using the partial eta-squared (partial η2); it is described as small when less than 0.06, medium when greater than or equal to 0.06 and less than 0.14, and large when greater than or equal to 0.14 [35]. The effect size for WER III, therefore, was large. Further analysis of the repeated measures ANOVA for WER III showed that the within-subjects by between-subjects interaction effect was not significant (F(3.08, 40.01) = 1.935; p=0.138), demonstrating that all language groups made similar positive gains in the condition over time. Means and standard errors for WER I, WER II, and WER III are displayed in Figures 1–3, respectively.Table 3
Means (M) and standard errors (SE) for word efficiency ratio (WER) at the beginning, middle, and end of the year by language groups.
WER I
WER II
WER III
BEG
MID
END
BEG
MID
END
BEG
MID
END
English
M
0.17
0.21
0.21
0.61
0.71
0.69
0.77
0.84
0.86
SE
0.05
0.06
0.06
0.08
0.05
0.06
0.05
0.03
0.04
Bilingual
M
0.26
0.23
0.35
0.71
0.79
0.77
0.82
0.92
0.89
SE
0.05
0.06
0.05
0.07
0.02
0.05
0.05
0.03
0.03
LDL
M
0.01
0.03
0.05
0.13
0.14
0.26
0.22
0.32
0.53
SE
0.01
0.01
0.03
0.07
0.06
0.07
0.11
0.06
0.09
Total
M
0.16
0.17
0.21
0.53
0.60
0.61
0.65
0.74
0.79
SE
0.03
0.03
0.04
0.06
0.06
0.05
0.06
0.05
0.04Figure 1
Means and standard errors for WER I (perfect T-unit: no grammatical errors) at the beginning, middle, and end of the year by language groups.Figure 2
Means and standard errors for WER II (flawed T-unit: contains all the main critical structures but has one or more surface errors) at the beginning, middle, and end of the year by language groups.Figure 3
Means and standard errors for WER III (perfect word string: 3+ consecutive words within a T-unit not deemed perfect or flawed) at the beginning, middle, and end of the year by language groups.


### 3.2. Research Question 2: Unintelligible Phrasal Constructions—Phrasal Errors (PEs), Total Number of Words, and Total Number of Unique Words

T-units in each writing sample were analyzed for the inclusion of Phrasal Errors (PEs), or confused statements that were deemed unintelligible. Descriptive statistics for average T-units per sample and percentage of PEs per T-unit (i.e., number of PEs divided by number of T-units) are included in Table4. Means and standard deviation (SD) data are reported for the beginning, middle, and end of the year samples for all participants N=29 and by language groups (i.e., English, bilingual, and LDL). Results of the repeated measures ANOVA for PEs, or total number of phrasal errors divided by the total number of T-units, demonstrated a nonsignificant main effect (F(2, 52) = 1.31; p=0.27). There was a significant main effect for change in the total number of T-units (F(2, 52) = 4.27; p<0.05), with an average of 15 T-units at the beginning of the year and 23 T-units per writing sample at the end of the year for the total group.Table 4
Means and standard deviations for average T-units per sample and phrasal errors (PEs) as a percentage of T-unit for the beginning, middle, and end of the year writing samples by language groups.
Average T-unit
Average # of PE per T-units
BEG
MID
END
BEG
MID
END
Language group
EnglishN=13
13 (13)
12 (8)
18 (10)
4.4% (5.2%)
2.8% (6.5%)
1.2% (2.9%)
BilingualN=9
27 (12)
20 (14)
42 (26)
2% (3.2%)
0.7% (2%)
0% (0%)
LDLN=7
3 (2)
15 (10)
9 (3)
21.9% (36.8%)
14.9% (19.3%)
14% (22.7%)
TotalN=29
15 (14)
16 (11)
23 (21)
7.9% (19.3%)
5.0% (11.5%)
4.0% (12.2%)At the beginning of the year, 7.7% (19.3% SD) of all T-units included PEs, 5.0% (11.5% SD) at the middle, and 4.0% (12.2% SD) at the end of the year. While all language groups demonstrated both an increase in overall number of T-units and a decrease in PEs over the course of the year, there were differences in the percentage of PEs by group. The English group demonstrated a decrease from 4.4% (SD 5.2%) to 1.2% (SD 2.9%) by the end of the year. The bilingual language group demonstrated the lowest percentage of PEs (2%, 3.2% SD) at the beginning of the year and decreased to 0% PEs by the end of the year. The language delayed learners demonstrated the highest percentage of PEs at the beginning of the year, with 21.9% (36.8%) of T-units including a PE and a decrease to 14% (SD 22.7%) by the end of the academic year.Data were also analyzed for number of words and number of unique vocabulary words for all writing samples by language groups. Table5 includes descriptive statistics for total number of words and unique vocabulary words for the beginning, middle, and end of the year writing samples for all students and by language groups. There was a significant main effect for change in the total number of words (F(2, 52) = 4.26; p<0.05) and number of unique vocabulary words (F(2, 52) = 5.15; p<0.01). Specifically, the average number of words increased from 122 at the beginning of the year to 200 at the end of the year, and the average number of unique words increased from 60 to 89, respectively, for the total group of students.Table 5
Mean, standard deviation (SD), and minimum and maximum for total number of words and unique vocabulary words for the beginning, middle, and end of the year writing samples by language group.
Beginning
Middle
End
Mean (SD); min–max
Mean (SD); min–max
Mean (SD); min–max
Total words
English (13)
91 (99); 17–365
100 (63); 46–239
156 (101); 37–356
Bilingual (9)
251 (127); 43–409
183 (88); 80–356
381 (290); 131–970
LDL (7)
10 (6); 3–18
81 (62); 14–176
50 (25); 18–91
All students (29)
122 (133); 3–409
121 (81); 14–356
200 (214); 18–970
Unique vocabulary words
English (13)
49 (40); 15–153
55 (24); 29–110
73 (38); 28–157
ASL (9)
114 (49); 28–170
94 (38); 50–171
155 (88); 68–310
Language delayed (7)
10 (6); 3–17
47 (27); 12–88
32 (14); 17–58
All students (29)
60 (55); 3–170
65 (35); 12–171
89 (72); 17–310


## 3.1. Research Question 1: Perfect and Flawed Phrasal Constructions—SAWL

Descriptive statistics for WER I, WER II, and WER III for the whole group and by language groups are provided in Table3. Results of the repeated measures ANOVA for WER I, or total number of words in perfect T-units divided by the total word count, demonstrated a nonsignificant main effect (F(2, 52) = 1.68, p=0.2). The repeated measures ANOVA for WER II, or the total number of words in perfect and flawed T-units divided by the total word count, also demonstrated a nonsignificant main effect (F(2, 52) = 1.87, p=0.16). Sphericity cannot be assumed for WER III, or the total number of words in perfect and flawed T-units and perfect word strings divided by the total word count; therefore, the Greenhouse-Geisser correction procedure was used. The repeated measures ANOVA for WER III showed a statistically significant main effect: F(1.54, 40.01) = 6.989, p<0.01, and partial η2 = 0.212. Effect size is provided using the partial eta-squared (partial η2); it is described as small when less than 0.06, medium when greater than or equal to 0.06 and less than 0.14, and large when greater than or equal to 0.14 [35]. The effect size for WER III, therefore, was large. Further analysis of the repeated measures ANOVA for WER III showed that the within-subjects by between-subjects interaction effect was not significant (F(3.08, 40.01) = 1.935; p=0.138), demonstrating that all language groups made similar positive gains in the condition over time. Means and standard errors for WER I, WER II, and WER III are displayed in Figures 1–3, respectively.Table 3
Means (M) and standard errors (SE) for word efficiency ratio (WER) at the beginning, middle, and end of the year by language groups.
WER I
WER II
WER III
BEG
MID
END
BEG
MID
END
BEG
MID
END
English
M
0.17
0.21
0.21
0.61
0.71
0.69
0.77
0.84
0.86
SE
0.05
0.06
0.06
0.08
0.05
0.06
0.05
0.03
0.04
Bilingual
M
0.26
0.23
0.35
0.71
0.79
0.77
0.82
0.92
0.89
SE
0.05
0.06
0.05
0.07
0.02
0.05
0.05
0.03
0.03
LDL
M
0.01
0.03
0.05
0.13
0.14
0.26
0.22
0.32
0.53
SE
0.01
0.01
0.03
0.07
0.06
0.07
0.11
0.06
0.09
Total
M
0.16
0.17
0.21
0.53
0.60
0.61
0.65
0.74
0.79
SE
0.03
0.03
0.04
0.06
0.06
0.05
0.06
0.05
0.04Figure 1
Means and standard errors for WER I (perfect T-unit: no grammatical errors) at the beginning, middle, and end of the year by language groups.Figure 2
Means and standard errors for WER II (flawed T-unit: contains all the main critical structures but has one or more surface errors) at the beginning, middle, and end of the year by language groups.Figure 3
Means and standard errors for WER III (perfect word string: 3+ consecutive words within a T-unit not deemed perfect or flawed) at the beginning, middle, and end of the year by language groups.


## 3.2. Research Question 2: Unintelligible Phrasal Constructions—Phrasal Errors (PEs), Total Number of Words, and Total Number of Unique Words

T-units in each writing sample were analyzed for the inclusion of Phrasal Errors (PEs), or confused statements that were deemed unintelligible. Descriptive statistics for average T-units per sample and percentage of PEs per T-unit (i.e., number of PEs divided by number of T-units) are included in Table4. Means and standard deviation (SD) data are reported for the beginning, middle, and end of the year samples for all participants N=29 and by language groups (i.e., English, bilingual, and LDL). Results of the repeated measures ANOVA for PEs, or total number of phrasal errors divided by the total number of T-units, demonstrated a nonsignificant main effect (F(2, 52) = 1.31; p=0.27). There was a significant main effect for change in the total number of T-units (F(2, 52) = 4.27; p<0.05), with an average of 15 T-units at the beginning of the year and 23 T-units per writing sample at the end of the year for the total group.Table 4
Means and standard deviations for average T-units per sample and phrasal errors (PEs) as a percentage of T-unit for the beginning, middle, and end of the year writing samples by language groups.
Average T-unit
Average # of PE per T-units
BEG
MID
END
BEG
MID
END
Language group
EnglishN=13
13 (13)
12 (8)
18 (10)
4.4% (5.2%)
2.8% (6.5%)
1.2% (2.9%)
BilingualN=9
27 (12)
20 (14)
42 (26)
2% (3.2%)
0.7% (2%)
0% (0%)
LDLN=7
3 (2)
15 (10)
9 (3)
21.9% (36.8%)
14.9% (19.3%)
14% (22.7%)
TotalN=29
15 (14)
16 (11)
23 (21)
7.9% (19.3%)
5.0% (11.5%)
4.0% (12.2%)At the beginning of the year, 7.7% (19.3% SD) of all T-units included PEs, 5.0% (11.5% SD) at the middle, and 4.0% (12.2% SD) at the end of the year. While all language groups demonstrated both an increase in overall number of T-units and a decrease in PEs over the course of the year, there were differences in the percentage of PEs by group. The English group demonstrated a decrease from 4.4% (SD 5.2%) to 1.2% (SD 2.9%) by the end of the year. The bilingual language group demonstrated the lowest percentage of PEs (2%, 3.2% SD) at the beginning of the year and decreased to 0% PEs by the end of the year. The language delayed learners demonstrated the highest percentage of PEs at the beginning of the year, with 21.9% (36.8%) of T-units including a PE and a decrease to 14% (SD 22.7%) by the end of the academic year.Data were also analyzed for number of words and number of unique vocabulary words for all writing samples by language groups. Table5 includes descriptive statistics for total number of words and unique vocabulary words for the beginning, middle, and end of the year writing samples for all students and by language groups. There was a significant main effect for change in the total number of words (F(2, 52) = 4.26; p<0.05) and number of unique vocabulary words (F(2, 52) = 5.15; p<0.01). Specifically, the average number of words increased from 122 at the beginning of the year to 200 at the end of the year, and the average number of unique words increased from 60 to 89, respectively, for the total group of students.Table 5
Mean, standard deviation (SD), and minimum and maximum for total number of words and unique vocabulary words for the beginning, middle, and end of the year writing samples by language group.
Beginning
Middle
End
Mean (SD); min–max
Mean (SD); min–max
Mean (SD); min–max
Total words
English (13)
91 (99); 17–365
100 (63); 46–239
156 (101); 37–356
Bilingual (9)
251 (127); 43–409
183 (88); 80–356
381 (290); 131–970
LDL (7)
10 (6); 3–18
81 (62); 14–176
50 (25); 18–91
All students (29)
122 (133); 3–409
121 (81); 14–356
200 (214); 18–970
Unique vocabulary words
English (13)
49 (40); 15–153
55 (24); 29–110
73 (38); 28–157
ASL (9)
114 (49); 28–170
94 (38); 50–171
155 (88); 68–310
Language delayed (7)
10 (6); 3–17
47 (27); 12–88
32 (14); 17–58
All students (29)
60 (55); 3–170
65 (35); 12–171
89 (72); 17–310


## 4. Discussion

In order to provide d/hh students with an optimal learning environment, it is critical to accurately assess their understanding of language and literacy skills. The purpose of this paper was to investigate the writing skills of d/hh students with varying language abilities using alternative assessment measures to capture patterns of grammatical and linguistic growth in written expression. This study found the SAWL to be a measure that was sensitive to differences in writing outcomes produced by students with varying language and literacy skills. Thus, by analyzing the grammatical errors according to the different levels of the SAWL’s word efficiency ratio (WER), it is possible to assess students’ use of correct written English grammaticality [30]. There was a significant increase in the average total number of words in perfect and flawed T-units and perfect word strings participants included in the personal narrative writing all of students N=29 over one academic year. The SAWL was shown to be sensitive to changes happening at the phrasal and T-unit levels, and thus, it makes an adequate measure for students who are currently performing at very different levels—some writing near-perfect English and others at the beginning levels of English knowledge. Improvements in linguistic written outcomes were also demonstrated by the decrease of phrasal errors (PEs) and significant increases in the use of unique vocabulary words in their written composition.The repeated measures ANOVA for WER III was statistically significant, and all three groups (English, bilingual, and LDL) showed the same pattern of improvement. Further analysis of the SAWL descriptive statistics helps us to illustrate some logical changes in performance. The English group demonstrated an 8% gain (0.61 to 0.69) in perfect and flawed T-units (i.e., WER II) from the beginning to the end of the academic year and a 9% gain (0.77 to 0.86) in perfect, flawed, and three word strings (i.e., WER III). The bilingual group made the largest gains in WER I scores. Approximately 0.26 of T-units were perfect at the beginning of the year, and 0.35 were perfect at the end. The language delayed learning group made the largest gains in their WER III scores, with an increase from 0.22 at the beginning of the year to 0.53 at the end, meaning they were writing substantially more perfect word strings by the end of one year. While all three groups demonstrated significant changes in grammatical growth, these changes were unique to the individual groups, impacting WER I (i.e., perfect T-units), II (i.e., flawed T-units), and III (i.e., perfect 3+ word strings) differently.Different patterns of grammatical growth by language profile were captured using the SAWL, which may not have been identified with other norm-referenced assessment measures. For example, the LDL group showed the most progress in WER III (i.e., perfect word strings; a 21% increase) as compared to a 4% gain in perfect T-units (WER I), suggesting that analysis at the phrasal level can provide valuable information about the LDL groups’ development of personal narrative writing. In contrast, WER I gains over the year for the bilingual group provided more diagnostically relevant information, as they began the year with a higher percentage of perfect T-units and complete independent clauses. This indicates that the SAWL was sensitive to change in linguistic proficiency at the word string and sentence level by group and provided fine grain specificity that found increases in skills that may not have been measured by standard assessments. For a group of students with highly diverse writing abilities, it may be beneficial to use the WER III analysis to measure language changes at all levels (i.e., perfect, flawed, and 3+ word strings).While SAWL demonstrated the ability to capture meaningful increases in grammatical accuracy, the researchers also identified confused statements that were deemed unintelligible in both English and ASL and coded the statements as phrasal errors. Thus, an additional analysis of the presence of phrasal errors (PEs) was conducted. Although not statistically significant, there was an overall reduction in PEs for all students after one academic year of instruction, on average. Further analysis showed that, by the end of one year, PEs were eliminated in the bilingual group, and there was a reduction in PEs in the English and LDL groups. Overall, WER I (perfect T-units) scores were consistent with the presence and/or absence of PEs. For example, the bilingual group demonstrated the lowest number of PEs in their writing (2%), which is consistent with their high number of perfect T-units (0.26). PEs were present in 20–25% of T-units produced by students in the LDL group, which is also consistent with the group’s low number of perfect T-units (i.e., WER I score of 0.01 at the beginning of the year). Thus, at the beginning of the year, students in the LDL group averaged one of every four to five T-units that were unintelligible in both English and ASL as well as demonstrated very few perfect T-units. Students in this group did not demonstrate language competence in either English or ASL and that was also evident in their written expression. Students in the LDL group demonstrated both a decrease in PEs and an increase in WER I, II, and III. Students in the English group demonstrated more PEs than the bilingual group but less than the LDL group at the beginning of the year (4.4%), which is consistent with their perfect T-unit production (0.17). These findings point to the importance of language competence and how it serves as the foundation for literacy skills, including writing.On both the PE and SAWL analyses, students in the spoken English group scored lower than students in the bilingual group but higher than the students in the LDL group. In this study, the spoken English group included students who, despite using English as their primary form of communication, also demonstrated limited language proficiency. This could be in part due to the use of only one type of educational setting—a residential school for the deaf—for this study. It is possible that deaf peers who also used spoken English as their primary mode of communication were mainstreamed in general education classrooms or chose to attend schools for the deaf using a listening and spoken language approach and thus were not included in the participant pool for this study. Previous findings in the literature point to the variability in performance of d/hh children, including those learning English as their first language [36]. What is important to note is that, as a group, the students using English as their primary language demonstrated gains in the total number of words in perfect and flawed T-units and perfect word strings and decreases in unintelligible phrases over one year. It is clear as students work to develop English proficiency, there can also be robust literacy growth.To account for writing sample productivity, average number of T-units, number of words, and number of unique vocabulary words were analyzed by language group over time. Results demonstrated an increase in the average number of T-units and unique vocabulary words for all students. The English group increased from an average of 13 T-units to an average of 18 T-units, the bilingual group from 27 to 42, and the LDL group from 3 to 9 T-units. In addition, overall, there was an increase in the number of unique words that were used per writing sample, with an average of 60 unique vocabulary words in writing samples from the beginning of the year and 89 at the end. Increases in unique vocabulary words were also demonstrated by language groups, from 49 to 73 in the English group, 114 to 155 in the bilingual group, and 10 to 32 in the LDL group. Thus, decreases in confused and unintelligible phrases occurred as students produced lengthier and more semantically rich writing samples. Knowing that linguistic proficiency (i.e., oral language skills) is related to reading achievement [2], it is possible that linguistic competency gains acquired over the year may have influenced the students’ literacy gains.SIWI, the instructional approach implemented throughout the academic year, includes a “language zone,” a tool that emphasizes metalinguistic awareness of languages for clarification and production of complete thoughts for writing. One example of the use of the language zone comes from a later elementary classroom of d/hh children using a listening and spoken language communication approach. When the teacher asked for a detail to add to their co-constructed text, one child raised his hand and said, “Yellow hot.” They employed various approaches in the language zone to clarify the meaning of the expression such as drawing, role play, and use of pictures. Once members arrived at a point of shared understanding, the teacher modeled back complete and complex language associated with the now grounded concept: “Billy’s mom put oil in the pan and fried chicken.” For some students, this process of clarifying meaning and then pairing model language occurs in ASL, and then the class works together to translate the concept into equivalent written English. This translation process can help students with the metalinguistic awareness necessary to recognize the differences between English and ASL—differences that are sometimes missed when signs and oral language are combined (i.e., sim-com). Additionally, for students using listening/spoken language approaches to learn English, the language zone helps to explicitly teach components of language that can be missed using auditory information alone.SIWI may offer a platform for effectively teaching writing to students who are developing competency in English and/or ASL. In particular, the interactive nature of SIWI ensures that students are actively engaging with peers and their teacher in the thinking, problem-solving, and decision-making phases of the writing process. Research on dialogic modes of instruction suggests that language is critical to mediating understanding and transforming action [37]. In the case of SIWI, students are apprenticed with the cognitive tasks and thinking needed for writing. At the same time, we believe dialogic pedagogy also plays an instrumental role in furthering students’ language development. Through classroom dialogue, students are exposed to others’ language associated with cognitive activity such as problem-solving [38]. When students do not have sufficient language through which to communicate about the cognitive tasks at hand, as is the case with many d/hh students, they experience frequent expressive and receptive communication breakdowns. During SIWI, the act of collaborative writing with the goal that all participants are involved in the writing decisions and the process of co-constructing text means that communication breakdowns must be continually repaired. Repairing communication breakdowns with the application of language zone strategies keeps language delayed students on track with the classroom dialogue and creates language learning opportunities. This study demonstrated that students engaged in SIWI with diverse linguistic abilities show writing gains using cohesive text in English with less phrasal errors. Previous work has also demonstrated that students have shown expressive language gains in ASL clarity and complexity after using SIWI [39]. Thus, we believe that having a strong language foundation in English and/or ASL is an important link to d/hh students’ literacy success.


## 5. Limitations and Future Directions

Findings from this study suggest that SAWL presented as a useful tool for tracking growth in language competence and clarity during personal narrative writing. Future research should focus on both clarity and complexity data across genres, as students who take more risks to write complex phrases may demonstrate greater difficulties with English grammar than if they had chosen simple sentence structures. Additionally, examining language clarity in other genres beyond personal narrative would also be important in order to understand students’ development and proficiency of language use when writing with different purposes that require different writing conventions. Collecting and analyzing these data in tandem could lead to a more comprehensive understanding of language development and use. The complexity of written English may be demonstrated through a composite index (i.e., number of independent and dependent clauses divided by the number of independent clauses) or by calculating the number of words per T-unit. Additionally, data analysis for this study was conducted by a research team. Studies should be conducted to assess the feasibility of practicing speech-language pathologists, teachers, and related professionals to use the SAWL as part of a comprehensive diagnostic assessment battery for d/hh students.While the instructional approach SIWI was implemented throughout the academic year, no conclusions can be made regarding the effectiveness of instructional approach due to the lack of a comparison group. It is essential that future studies are experimental in nature and replicate the data collection and analyses conducted here with both experimental and comparison groups in order to infer with greater confidence that growth in English linguistic competence can be attributed to the instruction provided. Nevertheless, the changes in students’ written English clarity and grammaticality demonstrated in this study present as promising findings, as these have traditionally been challenging areas to invoke change.


## 6. Conclusions

Deaf and hard of hearing (d/hh) students have unique language histories that can impact literacy achievement [40]. Some d/hh children learn English via listening and spoken language approaches while others acquire ASL naturally in the early years of life; however, many d/hh children, for a number of reasons, experience delays in developing either speech or sign as their first language [1]. A portion of these children experience extreme language deprivation and arrive at school with limited or unintelligible language through which to communicate and learn. Regardless of previous language exposure, it is important to use a written language assessment that is sensitive to change over time to measure writing outcomes. The SAWL is a tool that captures improvements in writing ability at the phrasal and clausal (i.e., T-unit) level. Additionally, substantial linguistic gains can be made when teachers have specialized methods for responding to language needs [41]. Since a strong foundation in expressive/receptive language has a positive impact on written English competence, it is important to provide instructional approaches and implement an assessment protocol that captures written language improvements. Findings from this study suggest that improvements in writing outcomes can be documented using the SAWL by students who are developing competency in English and/or ASL.



---

*Source: 2139626-2018-03-13.xml*