No Child Left Behind Act: Education Assistance Could Help States Better Measure Progress of Students with Limited English Proficiency
Released: 2007-03-23T00:00:00

Letter

Background

Students with limited English proficiency are a diverse and complex group. They speak many languages and have a tremendous range of educational needs and include refugees with little formal schooling and students who are literate in their native languages. Accurately assessing the academic knowledge of these students in English is challenging. If a student responds incorrectly to a test item, it may not be clear if the student did not know the answer or misunderstood the question because of language barriers.

Title I of NCLBA requires states to administer tests in language arts and mathematics to all students in certain grades and to use these tests as the primary means of determining the annual performance of states, districts, and schools. These assessments must be aligned with the state’s academic standards—that is, they must measure how well a student has demonstrated his or her knowledge of the academic content represented in these standards. States are to show that increasing percentages of students are reaching the proficient level on these state tests over time. NCLBA also requires that students with limited English proficiency receive reasonable accommodations and be assessed, to the extent practicable, in the language and form most likely to yield accurate data on their academic knowledge. In addition, for language arts, students with limited English proficiency who have been in U.S. schools for 3 years or more must generally be assessed in English. Finally, NCLBA also created a new requirement for states to annually assess the English language proficiency of students identified as having limited English proficiency.

Accurately assessing the academic knowledge of students with limited English proficiency has become more critical because NCLBA designated specific groups of students for particular focus. These four groups are students who (1) are economically disadvantaged, (2) represent major racial and ethnic groups, (3) have disabilities, and (4) are limited in English proficiency. These groups are not mutually exclusive, so that the results for a student who is economically disadvantaged, Hispanic, and has limited English proficiency could be counted in three groups. States and school districts are required to measure the progress of all students in meeting academic proficiency goals, as well as to measure separately the progress of these designated groups. To make adequate yearly progress, each district and school must generally show that each of these groups met the state proficiency goal and that at least 95 percent of students in each group participated in these assessments. Students with limited English proficiency are a unique group under NCLBA because once they attain English proficiency they are no longer counted as part this group, although Education has given states some flexibility in this area.

Recognizing that language barriers can hinder the assessment of students who have been in the country for a short time, Education has provided some testing flexibility. Specifically, Education does not require students with limited English proficiency to participate in a state’s language arts assessment during their first year in U.S. schools. In addition, while these students must take a state’s mathematics assessment during their first year, a state may exclude their scores in determining whether it met its progress goals.

Title III of NCLBA focuses specifically on students with limited English proficiency, with the purpose of ensuring that these students attain English proficiency and meet the same academic standards as other students. This title holds states and districts accountable for student progress in attaining English proficiency by requiring states to establish goals to demonstrate annual increases in both the number of students attaining English proficiency and the number making progress in learning English. States must establish English language proficiency standards that are aligned with a state’s academic standards in order to ensure that students are acquiring the academic language they need to successfully participate in the classroom. Education also requires that a state’s English language proficiency assessment be aligned to its English language proficiency standards. While NCLBA requires states to administer academic assessments to students in some grades, it requires states to administer English language proficiency assessments annually to all students with limited English proficiency, from kindergarten to grade 12.

Students with Limited English Proficiency Performed below Progress Goals in 2004 in Almost Two- Thirds of States

In nearly two-thirds of the 48 states for which we obtained data, students with limited English proficiency did not meet state proficiency goals in the 2003-2004 school year. Students with limited English proficiency met goals in language arts and mathematics in 17 states. In 31 states, these students missed the goals either for language arts or for both language arts and mathematics (see fig. 1). In 21 states, the percentage of proficient students in this group was below both the mathematics and the language arts proficiency goals.

We found that the percentage of elementary school students with limited English proficiency achieving proficient scores on the state’s mathematics assessment was lower than that for the total student population in 48 of 49 states that reported to Education in school year 2003-2004. We also found that, in general, a lower percentage of students with limited English proficiency achieved proficient test scores than other selected student groups. All of the 49 states reported that these students achieved lower rates of proficiency than white students. The performance of limited English proficient students relative to the other student groups varied. In 37 states, for example, economically disadvantaged students outperformed students with limited English proficiency, while students with disabilities outperformed these students in 14 states.

Selected States Considered Language Issues when Developing Academic Assessments, but Validity and Reliability Concerns Remain

Officials in the 5 states we studied reported that they have taken steps to address challenges associated with academic assessments of students with limited English proficiency. However, Education’s peer reviews of 38 states found a number of concerns in assessing these students. Our group of experts indicated that states are generally not taking the appropriate set of comprehensive steps to create valid and reliable assessments for students with limited English proficiency. To increase validity and reliability, most states offered accommodations to students, such as providing extra time to complete the test and offering native language assessments. However, offering accommodations may or may not improve the validity of test results, as research in this area is lacking.

States Reported Efforts to Improve Validity of Assessment Results for Students with Limited English Proficiency

Officials in 5 states we studied reported taking some steps to address challenges associated with assessing students with limited English proficiency. Officials in 4 of these states reported following generally accepted test development procedures, while a Nebraska official reported that the state expects districts to follow such procedures.

Officials in California, New York, North Carolina, and Texas told us that they try to implement the principles of universal design, which support making assessments accessible to the widest possible range of students. This is done by ensuring that instructions, forms, and questions are clear and not more linguistically complex than necessary. In addition, officials in some states reported assembling committees to review test items for bias. For example, when developing mathematics items, these states try to make language as clear as possible to ensure that the item is measuring primarily mathematical concepts and to minimize the extent to which it is measuring language proficiency. A mathematics word problem involving subtraction, for example, might refer to fish rather than barracuda. Officials in 3 of our study states told us they also used a statistical approach to evaluate test items for bias related to students with limited English proficiency.

Both Education’s Peer Reviews and Our Group of Experts Raised Concerns Regarding State Efforts to Ensure Valid and Reliable Assessment Results

Education’s completed NCLBA peer reviews of 38 states found that 25 did not provide sufficient evidence on the validity or reliability of results for students with limited English proficiency. For example, in Idaho, peer reviewers commented that the state did not report reliability data for students with limited English proficiency. As of March 2007, 18 states have had their assessment systems fully approved by Education.

Our group of experts indicated that states are generally not taking the appropriate set of comprehensive steps to create valid and reliable assessments for these students and identified essential steps that should be taken. These experts noted that no state has implemented an assessment program for students with limited English proficiency that is consistent with technical standards. They noted that students with limited English proficiency are not defined consistently within and across states, which is a crucial first step to ensuring reliability. If the language proficiency levels of these students are classified inconsistently, an assessment may produce results that appear inconsistent because of the variable classifications rather than actual differences in skills. Further, it appears that many states do not conduct separate analyses for different groups of limited English proficient students. Our group of experts indicated that the reliability of a test may be different for heterogeneous groups of students, such as students who are literate in their native language and those who are not. Further, these experts noted that states are not always explicit about whether an assessment is attempting to measure skills only (such as mathematics) or mathematics skills as expressed in English. According to the group, a fundamental issue affecting the validity of a test is the definition of what is being measured.

The expert group emphasized that determining the validity and reliability of academic assessments for students with limited English proficiency is complicated and requires a comprehensive collection of evidence rather than a single analysis. In addition, the appropriate combination of analyses will vary from state to state, depending on the characteristics of the student population and the type of assessment. The group indicated that states are not universally using all the appropriate analyses to evaluate the validity and reliability of test results for students with limited English proficiency. These experts indicated that some states may need assistance to conduct appropriate analyses. Finally, they indicated that reducing language complexity is essential to developing valid assessments for these students, but expressed concern that some states and test developers do not have a strong understanding of universal design principles or how to use them to develop assessments that eliminate language barriers to measuring specific skills.

Accommodations Can Increase Validity of Assessment Results, but Research on Appropriate Use Is Limited

The majority of states offered some accommodations to try to increase the validity and reliability of assessment results for students with limited English proficiency. These accommodations are intended to permit students to demonstrate their academic knowledge, despite limited language ability. Our review of state Web sites found documentation on accommodations for 42 states. The number of accommodations offered varied considerably among states. The most common accommodations were allowing the use of a bilingual dictionary and reading test items aloud in English (see table 1). Some states also administered assessments to small groups of students or individuals, while others gave students extra time to complete a test.

According to our expert group and our review of literature, research is lacking on what specific accommodations are appropriate for students with limited English proficiency, as well as their effectiveness in improving the validity of assessment results. A 2004 review of state policies found that few studies focus on accommodations intended to address the linguistic needs of students with limited English proficiency or on how accommodations affect the performance of students with limited English proficiency. In contrast, significantly more research has been conducted on accommodations for students with disabilities, much of it funded by Education. Because of this research disparity, our group of experts reported that some states offer accommodations to students with limited English proficiency based on those they offer to students with disabilities, without determining their appropriateness for individual students. They noted the importance of considering individual student characteristics to ensure that an accommodation appropriately addresses the needs of the student.

Native Language and Alternate Assessments May Improve the Validity of Results but Are Challenging to Implement

In our survey, 16 states reported that they offered statewide native language assessments in language arts or mathematics in some grades for certain students with limited English proficiency in the 2004-2005 school year. For example, New York translated its statewide mathematics assessments into Spanish, Chinese, Russian, Korean, and Haitian-Creole. In addition, 3 states were developing or planning to develop a native language assessment.

Our group of experts told us that this type of assessment is difficult and costly to develop. Development of a valid native language assessment involves more than a simple translation of the original test. In most situations, a process of test development and validation similar to that of the nontranslated test is recommended. In addition, the administration of native language assessments may not be practicable, for example, when only a small percentage of limited English proficient students in the state speak a particular language or when a state’s student population has many languages. Members of our expert group told us that native language assessments are generally an effective accommodation only for students in specific circumstances, such as students who are instructed in their native language or are literate in their native language.

Thirteen states offered statewide alternate assessments (such as reviewing a student’s classroom work portfolio) in 2005 for certain students with limited English proficiency, as of March 2006. Our expert group noted that alternate assessments are difficult and expensive to develop, and may not be feasible because of the amount of time required for such an assessment. Members of the group also expressed concern about the extent to which these assessments are objective and comparable and can be aggregated with regular assessments.

Most States Implemented New English Language Proficiency Assessments but Faced Challenges Establishing Their Validity

Many states implemented new English language proficiency assessments for the 2005-2006 school year to meet Education’s requirement for states to administer English language proficiency tests that meet NCLBA requirements by the spring of 2006. These assessments must allow states to track student progress in learning English. Additionally, Education requires that these assessments be aligned to a state’s English language proficiency standards. Education officials said that because many states did not have tests that met NCLBA requirements, the agency funded four state consortia to develop new assessments that were to be aligned with state standards and measure student progress.

In the 2005-2006 school year, 22 states used assessments or test items developed by one of four state consortia, making this the most common approach taken by states. Eight states worked with test developers to augment off-the-shelf English language proficiency assessments to incorporate state standards. Officials in 14 states indicated that they are administering off-the-shelf assessments. Seven states, including Texas, Minnesota, and Kansas, created their own English language proficiency assessments. Officials in these states said they typically worked with a test developer or research organization to create the assessments.

Officials in our study states and test developers we interviewed reported that they commonly apply generally accepted test development procedures to develop their assessments, but some are still in the process of documenting their validity and reliability. A 2005 review of the documentation of 17 English proficiency assessments used by 33 states found that the evidence on validity and reliability was generally insufficient. The study, which was funded by Education, noted that none of the assessments contained “sufficient technical evidence to support the high-stakes accountability information and conclusions of student readiness they are meant to provide.”

Education Has Provided Assistance, but States Reported Need for Additional Guidance and Flexibility

Education has offered states a variety of technical assistance to help them appropriately assess students with limited English proficiency, such as providing training and expert reviews of their assessment systems. However, Education has issued little written guidance on how states are expected to assess and track the English proficiency of these students, leaving state officials unclear about Education’s expectations. While Education has offered states some flexibility in how they incorporate these students into their accountability systems, many of the state and district officials we interviewed indicated that additional flexibility is needed to ensure that academic progress of these students is accurately measured.

Education Has Provided a Variety of Support on Assessment Issues but Little Written Guidance on Assessing Students with Limited English Proficiency

Education offers support in a variety of ways to help states meet NCLBA’s assessment requirements for students with limited English proficiency. The department’s primary technical assistance efforts have included the following: Title I peer reviews of states’ academic standards and assessment systems: During these reviews, experts review evidence provided by the state about the validity and reliability of these assessments. Education shares information from the peer review to help states address issues identified during the review.

Title III monitoring visits: Education began conducting site visits to review state compliance with Title III requirements in 2005. As part of these visits, the department reviews the state’s progress in developing English language proficiency assessments that meet NCLBA requirements.

Comprehensive centers: Education has contracted with 16 regional comprehensive centers to build state capacity to help districts that are not meeting their adequate yearly progress goals. At least 3 of these centers plan to assist individual states in developing appropriate goals for student progress in learning English. In 2005, Education also funded an assessment and accountability comprehensive center, which provides technical assistance related to the assessment of students, including those with limited English proficiency.

Ongoing technical assistance for English language proficiency assessments: Education has provided information and ongoing technical assistance to states using a variety of tools and has focused specifically on the development of the English language proficiency standards and assessments required by NCLBA.

While providing this technical assistance, Education has issued little written guidance on developing English language proficiency assessments that meet NCLBA’s requirements and on tracking the progress of students in acquiring English. Education issued some limited nonregulatory guidance on NCLBA’s basic requirements for English language proficiency standards and assessments in February 2003.

However, officials in about one-third of the 33 states we contacted expressed uncertainty about implementing these requirements. They told us that they would like more specific guidance from Education to help them develop tests that meet NCLBA requirements, generally focusing on two issues. First, some officials said they were unsure about how to align English language proficiency standards with content standards for language arts, mathematics, and science, as required by NCLBA. Second, some officials reported that they did not know how to use the different scores from their old and new English language proficiency assessments to track student progress. Without guidance and specific examples on both of these issues, some of these officials were concerned that they will spend time and resources developing an assessment that may not meet Education’s requirements. Education officials told us that they were currently developing additional nonregulatory guidance on these issues, but it had not yet been finalized.

Education Has Offered Different Accountability Options for Students with Limited English Proficiency, but State Officials Reported Additional Flexibility Is Needed

Education has offered states several flexibilities in tracking academic progress goals for students with limited English proficiency to support their efforts to develop appropriate accountability systems for these students. For example, students who have been in U.S. schools for less than a year do not have to meet the same testing requirements as other students. Another flexibility recognizes that limited English proficiency is a more transient quality than being of a particular race. Students who achieve English proficiency leave the group at the point when they demonstrate their academic knowledge in English, while new students with lower English proficiency are constantly entering the group (see fig. 2). Given the group’s continually changing composition, meeting progress goals may be more difficult than doing so for other student groups, especially in districts serving large numbers of these students.

Consequently, Education allowed states to include, for up to 2 years, the scores of students who were formerly classified as limited English proficient when determining whether a state met its progress goals for students with limited English proficiency.

Several state and local officials in our study states told us that additional flexibility would be helpful to ensure that the annual progress measures provide meaningful information about the performance of students with limited English proficiency. Officials in 4 of the states we studied suggested that certain students with limited English proficiency should be exempt from testing or have their test results excluded for longer periods than is currently allowed. Several officials voiced concern that some of these students have such poor English skills or so little previous school experience that assessment results do not provide any meaningful information. Instead, some of these officials stated that students with limited English proficiency should not be included in academic assessments until they demonstrate appropriate English. However, the National Council of La Raza, a Hispanic advocacy organization, has voiced concern that excluding too many students from a state’s annual progress measures will allow some states and districts to overlook the needs of these students.

With respect to including the scores of students previously classified as limited English proficient for up to 2 years, officials in 2 of our 5 study states, as well as one member of our expert group, thought it would be more appropriate for these students to be counted in the limited English proficient group throughout their school careers—but only for accountability purposes. They pointed out that by keeping students formerly classified as limited English proficient in the group, districts that work well with these students would see increases in the percentage who score at the proficient level in language arts and mathematics. An Education official explained that the agency does not want to label these students as limited English proficient any longer than necessary. Education officials also noted that including all students who were formerly limited English proficient would inflate the achievement measures for this group.

District officials in 4 states argued that tracking the progress of individual students in this group is a better measure of how well these students are progressing academically. Officials in one district pointed to a high school with a large percentage of students with limited English proficiency that had made tremendous progress with these students, doubling the percentage of students achieving academic proficiency. The school missed the annual progress target for this group by a few percentage points, but school officials said that the school would be considered successful if it was measured by how much individual students had improved. In response to educators and policymakers who believe such an approach should be used for all students, Education initiated a pilot project in November 2005, allowing a limited number of states to incorporate measures of student progress over time in determining whether districts and schools met their annual progress goals.

Prior Recommendations and Agency Response

We made several recommendations to Education in our July 2006 report. Specifically, we recommended that Education support additional research on appropriate accommodations for these students and disseminate information on research-based accommodations to states. We also recommended that Education determine what additional technical assistance states need to implement valid and reliable academic assessments for these students and provide such assistance. Further, we recommended that Education publish additional guidance with more specific information on the requirements for assessing English language proficiency and tracking student progress in learning English. Finally, we recommended that Education explore ways to provide states with additional flexibility in terms of holding states accountable for students with limited English proficiency.

Education agreed with our first three recommendations and has taken a number of steps to address them. In recognition of the challenges associated with assessing students with limited English proficiency and in response to GAO’s report, Education initiated the LEP (Limited English Proficient) Partnership in July 2006. Under the partnership, Education has pledged to provide technical assistance and support to states in the development of assessment options for states to use in addressing the needs of their diverse student populations. Education’s partners in this effort include the National Council of LaRaza, Mexican American Legal Defense and Educational Fund, Council of Chief State School Officers, Comprehensive Center on Assessment and Accountability, and the National Center on English Language Acquisition. All states have been invited to participate in this effort. The partnership held its first meeting in August 2006. In October 2006, officials from all the states came together to discuss areas for which they need additional technical assistance. As a result of these meetings, Education is supporting a variety of technical assistance projects, including the development of a framework on English language proficiency standards and assessments, the development of guides for developing native language and simplified assessments, and the development of a handbook on appropriate accommodations for students with limited English proficiency. Education officials told us that they are planning the next partnership meeting for the summer of 2007 and expect to have several of these resources available at that time.

Education did not explicitly agree or disagree with our recommendation to explore additional options for state flexibility. Instead, the agency commented that it has explored and already provided various types of flexibility regarding the inclusion of students with limited English proficiency in accountability systems. However, in January 2007, Education issued a blueprint for strengthening NCLBA, which calls for greater use of growth models and the recognition within state accountability systems of schools that make significant progress in moving students toward English proficiency.

Mr. Chairman, this completes my prepared statement. I would be happy to respond to any questions you or other members of the subcommittee may have.

GAO Contacts

For further information regarding this testimony, please contact me at (202) 512- 7215. Individuals making key contributions to this testimony include Harriet Ganson, Bryon Gordon, Shannon Groff, Krista Loose, Michelle St. Pierre, Sheranda Campbell, and Nagla’a El Hodiri.

Related GAO Products

No Child Left Behind Act: Education’s Data Improvement Efforts Could Strengthen the Basis for Distributing Title III Funds. GAO-07-140. Washington, D.C.: December 7, 2006.

No Child Left Behind Act: Education Actions Needed to Improve Local Implementation and State Evaluation of Supplemental Educational Services. GAO-06-758. Washington, D.C.: August 4, 2006.

No Child Left Behind Act: Assistance from Education Could Help States Better Measure Progress of Students with Limited English Proficiency. GAO-06-815. Washington, D.C.: July 26, 2006.

No Child Left Behind Act: States Face Challenges Measuring Academic Growth That Education’s Initiatives May Help Address. GAO-06-661. Washington, D.C.: July 17, 2006.

No Child Left Behind Act: Improved Accessibility to Education’s Information Could Help States Further Implement Teacher Qualification Requirements. GAO-06-25. Washington, D.C.: November 21, 2005.

No Child Left Behind Act: Education Could Do More to Help States Better Define Graduation Rates and Improve Knowledge about Intervention Strategies. GAO-05-879. Washington, D.C.: September 20, 2005.

No Child Left Behind Act: Most Students with Disabilities Participated in Statewide Assessments, but Inclusion Options Could Be Improved. GAO-05-618. Washington, D.C.: July 20, 2005.

Head Start: Further Development Could Allow Results of New Test to Be Used for Decision Making. GAO-05-343. Washington, D.C.: May 17, 2005.

No Child Left Behind Act: Education Needs to Provide Additional Technical Assistance and Conduct Implementation Studies for School Choice Provision. GAO-05-7. Washington, D.C.: December 10, 2004.

No Child Left Behind Act: Improvements Needed in Education’s Process for Tracking States’ Implementation of Key Provisions. GAO-04-734. Washington, D.C.: September 30, 2004.

This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.