DOD Schools: Additional Reporting Could Improve Accountability for Academic Achievement of Students with Dyslexia
Released: 2007-12-06T00:00:00

Letter

Background

DOD oversees a worldwide school system to meet the educational needs of military dependents and others, such as the children of DOD’s civilian employees overseas. The Department of Defense Education Activity (DODEA) administers schools both within the United States and overseas. In school year 2006-07, DODEA had schools within 7 states, Puerto Rico, Guam, and in 13 foreign countries. DOD has organized its 208 schools into three areas: the Americas (65), Europe (98), and Pacific (45). Almost all of the domestic schools are located in the southern United States. The overseas schools are mostly concentrated in Germany and Japan, where the U.S. built military bases after World War II. Given the transient nature of military assignments, these schools must adapt to a high rate of students transferring into and out of their schools. According to DOD, about 30 percent of its students move from one school to another each year. These students may transfer between DOD schools or between one DOD school and a U.S. public school.

Although DOD is not subject to the No Child Left Behind Act of 2001 (NCLBA), it has its own assessment and accountability framework. Unlike public schools, DOD schools receive funding primarily from DOD appropriations rather than through state and local governments or Department of Education grants. U.S. public schools that receive grants through the NCLBA must comply with testing and reporting requirements designed to hold schools accountable for educating their students and making adequate yearly progress. DOD has adopted its own accountability framework that includes a 5-year strategic plan, an annual report that measures the overall school system’s progress, and data requirements for school improvement plans. The strategic plan sets the strategic direction for the school system and outlines goals and performance measures to determine progress. In annual reports, DOD provides a broad overview of its students’ overall progress, including the results of standardized tests. On DOD’s Web site, DOD publishes more detailed test score results for each school at each grade level. DOD also requires each school to develop its own improvement plan that identifies specific goals and methods to measure progress. School officials have the flexibility to decide what goals to pursue but must identify separate sources of data to measure their progress in order to provide a more complete assessment. For example, if a school chooses to focus on improving its reading scores, it must identify separate assessment tests or other ways of measuring the progress of its students.

DOD is subject to many of the major provisions of IDEIA and must include students with disabilities in its standardized testing. However, unlike states and districts subject to NCLBA, DOD is not required to report publicly on the academic achievement of these students. States and public school districts that receive funding through IDEIA must comply with various substantive, procedural, and reporting requirements for students with disabilities. For example, they must have a program in place for evaluating and identifying children with disabilities, developing an individualized education program (IEP) for such students, and periodically monitoring each student’s academic progress under his or her IEP. Under IDEIA, children with disabilities must be taught, to the extent possible, with non-disabled students in the least restrictive environment, such as the general education classroom, and must be included in standardized testing unless appropriate accommodations or alternate assessments are required by their IEPs. Although DOD schools do not receive funding through IDEIA, they generally are subject to the same requirements concerning the education of children with disabilities. However, unlike states and districts that are subject to NCLBA, DOD schools are not required to report publicly on the performance of children with disabilities on regular and alternate assessments.

Definitions of dyslexia vary from broad definitions that encompass almost all struggling readers to narrow definitions that only apply to severe cases of reading difficulty. However, DOD and others have adopted a definition developed by dyslexia researchers and accepted by the International Dyslexia Association, a non-profit organization dedicated to helping individuals with dyslexia. This definition describes dyslexics as typically having a deficit in the phonological component of language, the individual speech sounds that make up words, which typically causes difficulty with accurate or fluent word recognition, poor spelling ability, and problems in reading comprehension that can impede growth of vocabulary. Recent research has identified a gene that may be associated with dyslexia and has found that dyslexia often coincides with behavior disorders or speech and language disabilities and can range from mild to severe. Nevertheless, the percentage of people who have dyslexia is unknown with estimates varying from 3 to 20 percent, depending on the definition and identification method used. Research promotes early identification and instruction for dyslexics to help mitigate lifelong impacts.

DOD Offers Professional Development to Teachers of Struggling Readers and Has Used Funds Designated for Dyslexia to Support That Effort

DOD offers professional development to all staff to help them support students who struggle to read, including those who may have dyslexia, and used designated funds to supplement existing training efforts across its schools. This professional development prepares teachers to assess student literacy skills and provides strategies to help instruct struggling readers. DOD used funds designated to support students with dyslexia for the development of two new online training courses containing modules on dyslexia, for additional seats in existing online courses, and for additional literacy assessment tools.

Professional Development for Staff Working with Struggling Readers Mainly Focuses on General Literacy Assessment and Instruction

DOD offers professional development to all staff who teach struggling readers, including students who may have dyslexia, primarily through online courses. The department offers online training courses through a professional development series known as Scholastic RED. These courses are DOD’s primary professional development on literacy for general education teachers. According to DOD, the department began offering the courses during the 2003-04 school year. DOD officials told us that since that time about half of the nearly 8,700 teachers in DOD schools have taken at least one Scholastic RED online course. Of the school principals who responded to our survey, almost all indicated that some of their staff members, including administrators and general and special education teachers, had participated in Scholastic RED training. Beyond Scholastic RED courses, DOD officials we interviewed told us that general education teachers also receive literacy development through instructional training in subject areas other than reading. For example, professional development on teaching at the middle school level may include guidance on how to enhance students’ reading skills through the study of a particular science.

Most professional development for staff working with struggling readers focuses on the assessment of student literacy skills and presents strategies for instructing students who struggle to read, some of whom may have dyslexia. Scholastic RED online courses train teachers in five basic elements of reading instruction: phonemic awareness, comprehension, phonics, fluency, and vocabulary. Research suggests that both phonics and phonemic awareness pose significant challenges to people who have dyslexia. According to course implementation materials, the training is designed to move beyond online course content and allow participants the opportunity to apply new skills in site-based study groups as well as in the classroom. Some principals and teachers indicated their schools follow this model with groups of teachers meeting to discuss best practices for applying Scholastic RED knowledge and resources in their classrooms.

DOD districts and schools sometimes offer their own literacy training through a localized effort or initiative. Professional development unique to a DOD district or school may be offered by a district’s special education coordinator. For example, the special education coordinator in a domestic district told us she offers literacy training to all staff, explaining that she tries to create a broader base of professionals who can more accurately identify and instruct students who are struggling readers. Regarding overseas schools, administrators in Korea told us they offer in-service workshops to help teachers improve student literacy, reading comprehension, and writing.

DOD designed and provided additional training on literacy instruction for most special education teachers and other specialists under a special education initiative. The training provided these staff members with courses on how students develop literacy skills and how to teach reading across all grade levels. According to a 2004-05 DOD survey on the initiative, over half of special educators and other specialists said they had completed this training. Since the 2003-04 school year, special education teachers and other specialists have received training on topics such as the evaluation of young children’s literacy skills and adjusting instruction based on student performance. The department also provided speech and language pathologists specialized training to help them assist struggling readers, including guidance on basic elements of literacy instruction and development, such as phonological awareness and vocabulary development.

DOD offers another literacy professional development program for special education teachers and other specialists known as Language Essentials for Teachers of Reading and Spelling (LETRS). According to the department, LETRS is designed to give teachers a better understanding of how students learn to read and write, showing instructors how to use such knowledge to improve targeted instruction for every type of reader. According to our survey results, about 10 percent of schools had staff who had taken this course. The LETRS course is based on the concept that once teachers understand the manner in which students approach reading and spelling tasks, they can make more informed decisions on instructional approaches for all readers. Much like the other literacy training DOD offers, LETRS modules contain reading instruction approaches on areas that may present challenges for those who have dyslexia: phonemic awareness, vocabulary, and reading comprehension.

Overall, DOD staff told us the literacy training the department offered was useful for them, with some indicating they wanted additional training. In responding to our survey, more than 80 percent of the principals who said their staff used Scholastic RED courses rated them as very useful for specialized instruction. Principals we interviewed told us their teachers characterize Scholastic RED concepts as practical and easy to apply in the classroom. While teachers we interviewed told us Scholastic RED training is helpful, some special education teachers indicated the course material is basic and better-suited to meet the developmental needs of general education teachers than special education teachers. For example, one special education teacher we spoke to said Scholastic RED courses do little to enhance the professional skills of special education teachers because many of these teachers have already received advanced training on reading interventions. Special education teachers did indicate, however, that training offered through the department’s special education initiative has provided them with identification strategies and intervention tools to support struggling readers. Regarding the impact of the initiative’s training, a DOD survey of special education teachers and other specialists found that over half of respondents said they had seen evidence of professional development designed to maximize the quality of special education services, and most had completed some professional development. The department did report, however, that respondents working with elementary school students frequently requested more training in areas such as phonemic awareness, while respondents working with high school students requested more professional development in a specific supplemental reading program used at DOD schools: Read 180. Moreover, teachers we interviewed in both foreign and domestic locations said they would like additional training on identifying and teaching students with specific types of reading challenges, including dyslexia. For example, one special education teacher we interviewed told us this specific training could help general education teachers to better understand the types of literacy challenges struggling readers face that in turn could help teachers better understand why students experience difficulties with other aspects of coursework.

Designated Funds Were Used for Online Courses That Include a Component on Dyslexia and for Additional Tools to Assess Student Literacy

DOD reported it had fully obligated the $3.2 million designated for professional development on dyslexia, with about $2.9 million for online courses and literacy assessment tools. Between fiscal years 2004 and 2006, the conference committee on defense appropriations designated a total of $3.2 million within the operation and maintenance appropriation for professional development on dyslexia. As of September 2007, DOD reported it had obligated these funds for professional development in literacy, including online training courses containing components on dyslexia. Reported obligations also included tools to help teachers identify and support students who struggle to read, some of who may have dyslexia. DOD obligated the remaining designated funds for general operations and maintenance purposes. All related obligations, as reported by the department, are outlined in table 1.

The online training included two newly developed courses that may be too new to evaluate and the purchase of extra seats in existing Scholastic RED training courses. The first of the new training courses to be fully developed was Fundamentals of Reading K-2. According to DOD, this course was designed to present teachers with strategies for instructing struggling readers in the early K-2 grade levels and contains six modules on the components of reading, including a specific module on dyslexia. The K-2 course was first made available in January 2006 to teachers who participated in a pilot project. DOD then opened the course to all teachers in February 2007. According to our survey results, 29 percent of the schools serving grades K-2 had used the course by the end of the school year. Nearly half of those school principals who indicated their staff used the course, however, did not indicate the extent to which it had been helpful in supporting struggling readers. It is possible the course is still too new for DOD schools to evaluate as some principals indicated on our survey that they had not heard of the course or they were not aware it was available to their staff. The second of the new online training courses, Fundamentals of Reading Grades 3-5, is not fully developed for use at this time. According to DOD officials, the course will be available to all staff in the 2007-08 school year and will also contain six modules on the components of reading, including a module on dyslexia. Additionally, DOD reported purchasing another 1,100 seats in selected Scholastic RED online training courses. The department also added a page entitled, Help your Students with Dyslexia to its main online resource site that is available to all teachers.

DOD reported also using designated funds to purchase electronic literacy assessment tools and other instruments that were widely used in DOD schools, one of which received mixed reviews on its usefulness. DOD reported obligating about one-third of the designated funds for the Dynamic Indicators of Basic Early Literacy Skills (DIBELS) assessment tool. The DIBELS assessment allows a teacher to evaluate a student’s literacy skills in a one-on-one setting through a series of one-minute exercises that can be administered via pen and paper or through the use of a hand-held electronic device. By using the exercises, teachers can measure and monitor these students’ skill levels in concepts such as phoneme segmentation fluency, a reading component that often gives dyslexics significant difficulty. DIBELS was used to help identify struggling readers in at least half of the schools serving grades K-2, according to our survey results, and DOD plans to begin use of the assessment in additional locations during the 2007-08 school year.

However, school officials and teachers had mixed reactions regarding the ease and effectiveness of using DIBELS to help identify struggling readers. In responding to our survey, about 40 percent of principals whose schools used DIBELS to help identify struggling readers indicated it was very or extremely useful, about 30 percent indicated it was moderately useful, and about 20 percent indicated it was either slightly or not at all useful. Several principals we surveyed indicated that they liked the instant results provided by the DIBELS assessment. For example, one principal called the assessment a quick and easy way to assess reading skills, saying it provides teachers with immediate feedback to help inform decisions about instruction. Others indicated the assessment is time-consuming for teachers. One kindergarten teacher we interviewed said that it is challenging to find the time to administer the test because it must be individually administered. Another principal expressed concern about the difficulty in using the electronic hand-held devices, saying the technology poses the greatest challenge to teachers in using the DIBELS assessment. According to DOD officials, the agency is currently evaluating its use of DIBELS, searching for other assessment tools, and will use the results to determine whether to continue using DIBELS or replace it with another tool. DOD purchased four other instruments to aid teachers in the evaluation of literacy skills; however, the tools are targeted to specific reading problems. According to DOD officials, they selected these tools because they measure specific skills associated with dyslexia. Table 2 shows reported use of each literacy assessment tool across DOD schools.

DOD Screens Students to Identify Those Who Struggle to Read and Provides Them with Supplemental Instruction

DOD schools identify students who have difficulty reading and provide them with supplemental reading services. DOD uses standardized tests to determine which students are struggling readers, although these tests do not screen specifically for dyslexia. DOD then provides these students with a standard supplemental reading program. For those children with disabilities who meet eligibility requirements, DOD provides a special education program in accordance with the requirements of IDEIA and department guidance.

DOD Schools Identify and Support Struggling Readers, Although the Schools Do Not Specifically Screen for Dyslexia

Schools primarily determine students’ reading ability and identify those who struggle through the use of standardized assessments. DOD uses several standardized assessments, including the TerraNova Achievement Test, and identifies those students who score below a certain threshold as having the most difficulty with reading and in need of additional reading instruction. DOD requires that schools administer these reading assessments starting in the third grade. However, some schools administer certain assessments as early as kindergarten. For example, some schools used Dynamic Indicators of Basic Early Literacy Skills (DIBELS) to identify struggling readers in grades K-2. In an effort to systematically assess students in kindergarten through second grade, DOD plans to identify assessment tools designed for these grades during school year 2007-08 and require their use throughout the school system. In addition to assessments, schools also use parent referrals and teacher observations to identify struggling readers. Several school officials with whom we spoke said that parent feedback about their children to school personnel and observations of students by teachers are both helpful in identifying students who need additional reading support. Like many public school systems in the United States, DOD school officials do not generally use the term “dyslexia.” However, DOD officials told us they provided an optional dyslexia checklist to classroom teachers to help determine whether students may need supplementary reading instruction and if they should be referred for more intensive diagnostic screening. According to our survey results, 17 percent of schools used the checklist in school year 2006-07.

DOD schools provide a supplemental reading program for struggling readers, some of whom may have dyslexia, a program that has some support from researchers and has received positive reviews from school officials, teachers, and parents we interviewed. The program, called READ 180, is a multimedia program for grades 3 through 12. It is designed for 90- minute sessions during which students rotate among three activities: whole-group direct instruction, small-group reading comprehension, and individualized computer-based instruction. The program is designed to build the reading skills, such as phonemic awareness, phonics, vocabulary, fluency, and comprehension. In responding to our survey, over 80 percent of the school principals indicated it was very helpful in teaching struggling readers. Several school administrators stated that it is effective with students due to the nonthreatening environment created by its multimodal instructional approach. Several teachers said the program also helped them to monitor student performance. Several parents told us that the program increased their children’s enthusiasm for reading, improved their reading skills, and boosted their confidence in reading and overall self- esteem. Some parents stated that their children’s grades in general curriculum courses improved as well since the children were not having difficulty with course content but rather with reading. At the secondary level, however, school officials stated that some parents chose not to enroll their child in READ 180 because of the stigma they associate with what they view as a remedial program. According to the Florida Center for Reading Research, existing research supports the use of READ 180 as an intervention to teach 6th, 7th, and 8th grade students comprehension skills, however; the center recommends additional studies to assess the program’s effectiveness.

Certain districts and schools have implemented additional strategies for instructing struggling readers such as using literacy experts, offering early intervention reading programs, and prioritizing reading in annual improvement plans. In the Pacific region and the Bavaria district, literacy experts work in collaboration with classroom teachers and reading specialists to design appropriate individualized instruction for struggling readers and monitor student performance. All of the elementary schools in the Pacific region offer reading support to struggling readers. Some schools offer early reading support in grades K-2. Certain districts offer early intervention to first and second graders in small groups of five and eight students, respectively. Some schools in Europe provide intensive instruction to students in first grade through Reading Recovery, a program in which struggling readers receive 30-minute tutoring sessions by specially trained teachers for 12 to 20 weeks. According to the Department of Education’s What Works Clearinghouse, Reading Recovery may have positive effects in teaching students how to read. Several superintendents and principals we interviewed said that improved reading scores was one of the school’s goals in their annual school-improvement plan, which is in line with DOD’s strategic plan milestone of having all students in grades three, six, and nine read at their grade level or higher by July 2011. For example, to improve reading scores, officials in the Heidelberg District developed a literacy program requiring each school to identify all third grade students who read below grade level and develop an action plan to improve their reading abilities.

DOD Schools Provide Special Education Services for Eligible Children with Disabilities

Those students whose performance does not improve through their enrollment in supplemental reading programs or who have profound reading difficulties may be eligible to receive special education services. DOD provides this special education program in accordance with the requirements of department guidance and the IDEIA, although DOD is not subject to the reporting and funding provisions of the act. According to our survey results, almost all schools provided special education services in the 2006-2007 school year.

The level of special education services available to students with disabilities varies between districts and schools, and may affect where some service-members and families can be assigned and still receive services. DOD established the Exceptional Family Member Program to screen and identify family members who have special health or educational needs. It is designed to assist the military personnel system to assign military service members and civilian personnel to duty stations that provide the types of health and education services necessary to meet their family members’ needs. In general, parents with whom we spoke said that they were pleased with the services their children received in DOD schools at the duty locations where they were assigned.

DOD conducts a comprehensive multidisciplinary assessment to evaluate whether a student is eligible to receive special education services under any of DOD’s disability categories, and most parents we interviewed were complimentary of the program. A student who is identified as having a disability receives specific instruction designed to meet the student’s academic needs. A team comprised of school personnel and the student’s parents meets annually to assess the student’s progress. While the majority of parents we interviewed were complimentary of DOD’s special education program, a few expressed concern that their children were not evaluated for special education eligibility early enough despite repeated requests to school personnel that their children needed to be evaluated for a suspected disability. According to DOD officials, department guidance requires school officials to look into parent requests, but officials do not have to evaluate the child unless they suspect the child has a disability. However, they must provide parents with written or oral feedback specifying why they did not pursue the matter.

Students with dyslexia may qualify for special education services under the specific learning disability category, but students must meet specific criteria. To qualify as having a specific learning disability, students must have an information-processing deficit that negatively affects their educational performance on an academic achievement test resulting in a score at or near the 10th percentile or the 35th percentile for students of above average intellectual functioning. There must also be evidence through diagnostic testing to rule out the possibility that the student has an intellectual deficit.

DOD schools provide children with disabilities instruction through two additional programs that have some research support. Fifteen percent of our survey respondents were principals of schools that used the Lindamood Phoneme Sequencing Program (LiPS), a program that helps students in grades prekindergarten through 12 with the oral motor characteristics of individual speech sounds. According to the What Works Clearinghouse, one research study it reviewed in 2007 suggested the LiPS program may have positive effects on reading ability. Our survey results indicated that 37 percent of schools serving grades 7 through 12 used a program called Reading Excellence: Word Attack and Rate Development Strategies that targets students who have mastered basic reading skills but who are not accurate or fluent readers of grade-level materials. According to a Florida Center for Reading Research report, there is research support for the program, but additional research is needed to assess its effectiveness.

DOD Assesses All Students Using Standardized Tests but Does Not Report Separately on the Academic Achievement of Those with Disabilities

DOD assesses the academic achievement of all students using standardized tests. The department administers the TerraNova Achievement Test to students in grades 3 through 11. Test scores represent a comparison between the test taker and a norm group designed to represent a national sample of students. For example, if a student scored at the 68th percentile in reading, that student scored higher than 68 percent of the students in the norm group–the national average is the 50th percentile. DOD uses these scores to compare the academic achievement of its students to the national average. In addition, DOD schools participate in the National Assessment of Educational Progress (NAEP), known as the nation’s report card, which provides a national picture of student academic achievement and a measure of student achievement among school systems. According to an agency official, DOD administers NAEP to all of its fourth and eighth grade students every other year. The NAEP measures how well DOD students perform as a whole relative to specific academic standards.

Overall, DOD students perform well in reading compared to the national average and to students in state public school systems, as measured by their performance on standardized tests. The latest available test results showed that DOD students scored above average and in some cases ranked DOD in the top tier of all school systems tested. According to the 2007 TerraNova test results, DOD students scored on average between the 60th and 75th percentile at all grade levels tested. The 2007 NAEP reading test results ranked the DOD school system among the top for all school systems. Specifically, on the eighth grade test, DOD tied for first place with two states among all states and jurisdictions and on the fourth grade test, tied with one state for third place.

All students, including those with disabilities, participate in DOD’s systemwide assessments using either the standard DOD assessment or alternate assessments. In some cases, students who require accommodations to complete the standard assessment may need to take the test in a small group setting, get extended time for taking the test, or have directions read aloud to them. Some students with severe disabilities may take an alternate assessment if required by the student’s individualized education program. An alternate assessment determines academic achievement by compiling and assessing certain documentation, such as a student’s work products, interviews, photographs, and videos. According to an official from DODEA’s Office of System Accountability and Research, DOD provides an alternate assessment to fewer than 200 of its roughly 90,000 students each year.

For use within the department and in some districts and schools, DOD disaggregates TerraNova test scores for students with disabilities. DOD officials reported that they disaggregate scores for the entire school system, each area, and each district, in order to gauge the academic performance of students with disabilities. DOD’s policy states that DOD shall internally report on the performance of children with disabilities participating in its systemwide assessments. According to DOD officials, they use the data to determine progress toward goals and to guide program and subject area planning. According to our survey results, over 90 percent of DOD schools disaggregate their test scores by gender and race and about 85 percent disaggregate for students with disabilities for internal purposes. Some school officials told us they use test data in order to track students’ progress, assess the effectiveness of services they offer students, identify areas of improvement, and assess school performance. For example, one Superintendent who shared her disaggregated data with us showed how third-grade students with disabilities made up over half of those who read below grade level in her district.

DOD does not generally report disaggregated test scores for students with disabilities. DOD’s annual report provides data at each grade level, and test scores posted on its Web site provide data for each school. DOD also reports some results by race and ethnicity for the NAEP test. However, DOD does not disaggregate its TerraNova test data for students with disabilities or other subgroups. A primary goal of its strategic plan is for all students to meet or exceed challenging academic content standards, and DOD uses standardized test score data to determine progress towards this goal. Disaggregating these data provides a mechanism for determining whether groups of students, such as those with disabilities, are meeting academic proficiency goals. However, unlike U.S. public school systems that are subject to the No Child Left Behind Act, DOD is not required to report test scores of designated student groups. According to DOD officials, they do not report test results for groups of fewer than 20 students with disabilities because doing so may violate their privacy by making it easier to identify individual students. Where there are groups of 20 or more students with disabilities, DOD officials said they do not report it publicly because it might invite comparisons between one school and another when all of them do well compared to U.S. public schools. DOD officials did not comment on any negative implications of such comparisons.

Conclusions

On the whole, DOD students perform well in reading compared with public school students in the United States, and in some cases DOD ranks near the top of all school systems, as measured by students’ performance on standardized tests. DOD has programs and resources in place to provide supplemental instruction to students who have low scores on standardized tests or who otherwise qualify for special education services, some of whom may have dyslexia. The department generally includes these students when administering national tests. Nevertheless, by not reporting specifically on the achievement of students with disabilities, including those who may have dyslexia, DOD may be overlooking an area that might require attention and thereby reducing its accountability. Without these publicly reported data, parents, policymakers, and others are not able to determine whether students with disabilities as a whole are meeting academic proficiency goals in the same way as all other students in the school system. For example, high performance on the part of most DOD students could mask low performance for students with disabilities.

Recommendation

To improve DOD’s accountability for the academic achievement of its students with disabilities, including certain students who may have dyslexia, we recommend that the Secretary of Defense instruct the Director of the Department of Defense Education Activity to publish separate data on the academic achievement of students with disabilities at the systemwide, area, district, and school levels when there are sufficient numbers of students with disabilities to avoid violating students’ privacy.

Agency Comments

We provided a draft of this report to DOD for review and comment. DOD concurred with our recommendation. DOD’s formal comments are reproduced in appendix II. DOD also provided technical comments on the draft report, which we have incorporated when appropriate.

We will send copies of this report to the Secretary of Defense, the Director of the Department of Defense Education Activity, and other interested parties. We will also make copies available to others upon request. In addition, the report will be available at no charge on GAO’s Web site at http://www.gao.gov.

Please contact me at (202) 512-7215 if you or your staff have any questions about this report. Contact points for our offices of Congressional Relations and Public Affairs may be found on the last page of this report. Major contributions to this report are listed in appendix III.

Appendix I: Objectives, Scope, and Methodology

Our objectives were to determine: 1) what professional development DOD provides its staff to support students with dyslexia and how the fiscal year 2004-to-2006 funds designated for this purpose were used, (2) what identification and instructional services DOD provides to students who may have dyslexia, and (3) how DOD assesses the academic achievement of students with disabilities, including dyslexia. To meet these objectives, we interviewed and obtained documentation from DOD and others, conducted a Web-based survey of all 208 DOD school principals, and visited or interviewed by phone officials and parents in six school districts. We conducted our work between January 2007 and October 2007 in accordance with generally accepted government auditing standards.

Interviews and Documentation from DOD and Others

To obtain information on how schools support students with dyslexia we interviewed officials from the Department of Defense Education Activity (DODEA) and the Department of Education, as well as representatives from the International Dyslexia Association and the National Association of State Directors of Special Education. We obtained several DODEA reports including: a 2007 report to Congress on DODEA’s efforts to assist students with dyslexia, a 2006 evaluation of DODEA’s English and language arts instruction, and a 2005 survey of DODEA special education personnel. We reviewed relevant federal laws, regulations, and DOD guidance, and also obtained information on DOD’s obligation and disbursement of funds designated for professional development on dyslexia. We also reviewed the DODEA web site for schools’ student performance data to determine how DOD assesses the academic achievement of students with disabilities. We also obtained summary reports on the scientific evidence regarding the effectiveness of DODEA’s supplemental reading programs from the Department of Education’s What Works Clearinghouse and the Florida Center for Reading Research, two organizations that compile and evaluate research on reading.

Survey of Principals

To gather information concerning dyslexic students in DoDEA schools, including how DoDEA schools identify dyslexic students and the instructional services provided to such students, we designed a Web-based survey. We administered the survey to all 208 DODEA school principals between May 10, 2007 and July 6, 2007, and received completed surveys from 175 school principals—an 84 percent response rate. In order to obtain data for a high percentage of DOD schools, we followed up with principals through e-mail and telephone to remind them about the survey. We also examined selected characteristics to ensure that the schools responding to our survey broadly represent DODEA’s school levels, geographic areas, and special education population. Based on our findings, we believe the survey data are sufficient for providing useful information concerning students with dyslexia.

Nonresponse (or, in the case of our work, those DOD school principals that did not complete the survey) is one type of nonsampling error that could affect data quality. Other types of nonsampling error include variations in how respondents interpret questions, respondents’ willingness to offer accurate responses, and data collection and processing errors. We included steps in developing the survey, and collecting, editing, and analyzing survey data to minimize such nonsampling error. In developing the web survey, we pretested draft versions of the instrument with principals at various American and European elementary, middle, and high schools to check the clarity of the questions and the flow and layout of the survey. On the basis of the pretests, we made slight to moderate revisions of the survey. Using a web- based survey also helped remove error in our data collection effort. By allowing school principals to enter their responses directly into an electronic instrument, this method automatically created a record for each principal in a data file and eliminated the need for and the errors (and costs) associated with a manual data entry process. In addition, the program used to analyze the survey data was independently verified to ensure the accuracy of this work.

Site Visits and Phone Interviews

We visited school officials and parents of struggling readers in two of the three areas (the Americas and Europe) overseen by DODEA and contacted schools in the third area (the Pacific) by phone. For each location we interviewed the district Superintendent or Assistant Superintendent, school principals, teachers, and special education teachers. At each location we also interviewed parents of struggling readers. Each group had between two and seven parents, and in some cases we interviewed a parent individually. To see how DOD schools instruct struggling readers we observed several reading programs during classroom instruction including Read 180, Reading Recovery, and Reading Improvement, as well as the use of literacy tools such as the Dynamic Indicator of Basic Literacy Skills. We selected 6 of DOD’s 12 school districts, 2 from each area, using the following criteria: (1) geographic dispersion, (2) representation of all military service branches, (3) variety of primary and secondary schools, and (4) range in the proportion of students receiving special education services.

Appendix II: Comments from the Department of Defense

Appendix III: Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

Harriet Ganson, Assistant Director, and Paul Schearf, Analyst-In-Charge, managed this assignment. Farah Angersola and Amanda Seese made significant contributions throughout the assignment, and Rebecca Wilson assisted in data collection and analysis. Kevin Jackson provided methodological assistance. Susan Bernstein and Rachael Valliere helped develop the report’s message. Sheila McCoy provided legal support.