Framework for High-Quality

**English Language Proficiency Standards and Assessments**

**DRAFT**

**October 2007**

Prepared for the U.S. Department of Education, LEP Partnership, Office
of the Deputy Secretary of Education, by the Assessment and
Accountability Comprehensive Center

![](media/image1.png){width="3.25in" height="0.7409722222222223in"}

![](media/image2.jpeg){width="1.875in" height="0.4777777777777778in"}

### CONTENTS

Introduction p. 2

Section 1: English Language Proficiency Standards p. 6

Section 2: English Language Proficiency Assessments p. 29

Recommendations for Application of the Framework p. 54

Resources p. 57

References p. 64

Acknowledgements p. 68

INTRODUCTION

Addressing the needs of the Nation\'s more than five million limited
English proficient (LEP)[^1]^,^ [^2] students (National Clearinghouse
for English Language Acquisition, 2006) is central to meeting the goals
of the No Child Left Behind Act of 2001 (NCLB). Improving instruction
and closing the achievement gap for English language learners (ELLs) is
highly dependent on the development and proper utilization of
high-quality systems of State standards and assessments. We need to be
able to measure what ELLs know and do not know, both in terms of English
language skills and academic content knowledge, so that educators have
the data required to inform classroom instruction for both English
language acquisition and academic content.

**NCLB Requirements Regarding English Language Proficiency Standards and
Assessments**

Title III of the Elementary and Secondary Education Act of 1965 (ESEA),
as amended by NCLB, requires each State educational agency (SEA) to
submit a plan to the Secretary of Education describing how the agency
will establish standards and objectives for raising the level of English
proficiency that are derived from the four recognized domains of
language---listening, speaking, reading, and writing---and
comprehension. These standards and objectives are to align[^3] with the
achievement of challenging academic content and academic achievement
standards for all students that States have implemented under Title I of
the ESEA.

The general requirement for testing for English language proficiency is
nearly identical under both Titles I and III. Both require local
educational agencies (LEAs) and SEAs to provide for an annual assessment
of English language proficiency in the four language domains of
listening, speaking, reading, and writing. Title III also requires LEAs
and SEAs to report student progress in English comprehension.
Comprehension can be demonstrated through reading and listening.

English language proficiency (ELP) standards are meant to guide the
design and administration of local language education programs. ELP
assessments are a primary tool for identifying ELLs and measuring
students' progress in their English language acquisition. Such
assessments also are used to hold States and LEAs accountable under
Title III for helping ELLs attain English language skills and knowledge,
including the language necessary for achievement in the core academic
content areas.

NCLB is clear that the main purposes of ELP standards, assessments, and
language instruction educational programs are to help ensure that ELLs
attain proficiency in English and meet the same academic content and
student achievement standards as all students are expected to meet.

**Purpose of the Framework**

States have requested assistance from the U.S. Department of Education
(Department) on how to evaluate the technical quality of ELP standards
and assessments. Key issues where they believe assistance would be
beneficial include how to demonstrate the validity and reliability of
the systems of standards and assessments and the alignment of ELP
standards to ELP assessments as well as to the achievement of
challenging content standards in the core subject areas required to be
assessed under NCLB. In October 2006, Deputy Secretary Raymond Simon set
as a Department priority that the LEP Partnership provide resources to
help States with their ELP standards and assessment implementation,
issues, and challenges (for more information on the LEP Partnership see
http://www.ed.gov/about/inits/ed/lep-partnership).

The purpose of this *Framework for High-Quality ELP Standards and
Assessments* (Framework) is to provide States with information on
suggested elements, possible considerations, and resources in order to
help States evaluate and improve the quality of their current ELP
standards and assessments. This Framework also will help States identify
their technical assistance needs related to ELP standards and
assessments and, therefore, help the Department provide States with the
assistance they need to improve and implement their Title III standards
and assessments effectively. This Framework is applicable both to States
that have established their own ELP standards and assessments, and to
States that are members of a consortium.[^4] This Framework, however,
does not establish any legally binding requirements on States or their
local educational agencies (LEAs).

The Framework is intended to guide the establishment of rigorous, valid,
and accessible State ELP standards and assessments that support
effective instruction. This Framework does not address the validity and
reliability of State accountability systems (e.g., for State
accountability workbooks) or how the results/uses of ELP assessments
ought to contribute to State accountability requirements under Title
III; rather, this Framework addresses the validity and reliability of
ELP standards and assessments for placement and proficiency
determinations.

# 

# **Development of the Framework** 

On June 6, 2007, the Department published a Federal Register notice
announcing a series of public meetings and a comment period during which
stakeholders were invited to submit recommendations regarding the
content of the *Framework for High-Quality ELP Standards and
Assessments*.

Specifically, the Department invited comment on the following four
questions:

> 1\. What are the critical elements that States should examine to
> ensure that their ELP standards promote effective instruction to raise
> LEP students' level of English proficiency? (Section 3113(b)(2))
>
> 2\. What are the critical elements that States should examine to
> ensure that their ELP assessments provide a valid and reliable
> assessment of English language proficiency? (Section
> 3122(a)(3)(A)(ii))
>
> 3\. What are the critical elements that States should examine to
> ensure that their ELP standards are aligned with their ELP
> assessments? (Sections 3113(b)(2) and (3)(D) and 3122(a)(3)(A)(ii))
>
> 4\. What are strategies that States can use to ensure that their ELP
> standards are aligned with the achievement of challenging State
> academic content standards and student academic achievement standards
> they have adopted under Title I? (Section 3113(b)(2))

In addition to inviting public comment, the Department's public meetings
featured roundtable discussions with experts in the fields of standards
and assessments, English language acquisition, applied linguistics, and
language research. A list of the roundtable participants, as well as
transcripts of the roundtable discussions and public meetings, is
available at: http://www.ed.gov/about/inits/ed/lep-partnership/.

Throughout the process of developing this Framework, the Assessment and
Accountability Comprehensive Center, working with the Department,
received valuable recommendations not only from technical experts in
standards, assessment, psychometrics, English language acquisition,
applied linguistics, and language research, but also from parents,
teachers, administrators, researchers, and others with experience and
expertise in Title I and Title III programs.

**Next Steps**

This first draft of the Framework will be shared with additional experts
and practitioners, as well as with States, for review and comment and to
solicit States to volunteer to undertake self-review of their State ELP
standards and assessments using the Framework. The purpose of this
self-review will be to serve as a means of improving the Framework as a
tool for States, not to judge the quality of the States' standards and
assessments as part of any formal review. The Department plans to
provide technical assistance to States willing to undertake such a
review and will revise the Framework document, in spring 2008, as
necessary, based on the outcomes from the self-review.

### Overview of Structure and Content of the Framework

The Framework document is divided into two main sections. Section 1
focuses on ELP standards. Section 2 focuses on ELP assessments. Within
these two major sections of the Framework, the document outlines general
issues and possible considerations for evaluating and improving the
quality of State ELP standards and assessments. The Framework includes a
set of suggested elements necessary for high-quality ELP standards and
assessments, possible considerations and strategies to address each
suggested element, and concludes with a list of resources to consult for
more information.

**SECTION 1: ENGLISH LANGUAGE PROFICIENCY STANDARDS**

The cornerstone of any substantive education reform lies in the creation
and implementation of rigorous standards. The impact of rigorous
standards is undeniable: they can provide a clear direction for what
students should know and be able to do and establish clear expectations
for schools, teachers, parents, and students.

Various States, researchers, technical assistance providers, and
educators refer to standards for English language acquisition in
different ways. During the Department's public meetings, references were
made to English language development (ELD) standards and to English as a
Second Language (ESL) standards, as well as standards for English
language learners (ELLs). For the purposes of the Framework, we refer to
the English language acquisition standards as **English language
proficiency (ELP) standards**, which is consistent with the language
used in Title III of the ESEA. ELP standards are distinguished from
English language arts (ELA), or reading standards, as defined by Title I
of the ESEA, which define *content* knowledge and skills rather than
*language* knowledge and skills.[^5] While we recognize that there are
important relationships between English language acquisition and
academic content knowledge---and that the learning of both happens
simultaneously for ELLs---we intend this Framework to focus specifically
on ELP standards and assessments.

ELP standards should define the knowledge and skills necessary for
students to attain English language proficiency and to participate
effectively in the academic content areas in English. Therefore, ELP
standards should define, in addition to the language skills and
knowledge specific to the needs of ELLs, the "academic language"
necessary for all students to access content in ELA, reading,
mathematics, science, social studies, and other academic content areas.
Broadly defined, academic language is the language (e.g., lexical,
grammatical, and discourse features specific to and reflective of a
particular context or content area) students need to meaningfully engage
with academic content and achieve proficiency in English in the academic
setting (Sato, 2007).[^6] In this sense, ELP standards are not limited
to the ESL teacher or classroom but are appropriate for all educators of
ELLs. ELP standards also should be well grounded in the theory and
research relevant to the language acquisition process.

# **Section 1 Overview**

This section presents suggested elements necessary for the evaluation
and implementation of high-quality English language proficiency (ELP)
standards. The suggested elements and possible considerations are based
on research-supported theory (including second language acquisition,
English language development/acquisition, and measurement) and applied
research relevant to the development of standards that will facilitate
student progress toward and attainment of proficiency in English,
including English language skills and knowledge necessary for
achievement in the academic content areas. Although there is overlap
among the suggested elements, they are listed separately in this
Framework to facilitate State processes for addressing the elements and
to reflect their individual contribution to a valid and reliable ELP
standards system. Although these elements are listed separately, States
should keep in mind the inter-relationships among these suggested
elements for establishing a comprehensive and cohesive plan for
addressing the needs of their English language learners (ELLs).

Generally, as States evaluate and implement their ELP standards, they
should consider the following:

-   The characteristics of the State's ELL population;

-   The process and people involved in developing, implementing, and
    evaluating the ELP standards;

-   The degree to which the ELP standards fit into the State's larger
    plan for educating all of its students;

-   The degree to which the ELP standards address academic language; and

-   The theoretical and research/empirical foundation for ELP standards.

# **Issues Relevant to the Development of High Quality English Language Proficiency Standards** 

Three issues are described briefly below and addressed where relevant in
the Suggested Elements and Possible Considerations presented in Section
1 of this Framework. States should keep these factors in mind as they
evaluate and improve their English language proficiency standards. Doing
so will better ensure the development and implementation of high-quality
standards that facilitate the proficiency and achievement of these
students, meet the requirements of Title III, and help address the
persistent achievement gap.

# The Nature of the Domain: English Language Proficiency

Clear models or frameworks of how students acquire language (i.e.,
English for students who are speakers of other languages) and clear
definitions of what constitutes proficiency in English are necessary for
establishing high-quality ELP standards. Researchers have a
long-standing debate on the very nature of language acquisition and
proficiency (Del Vecchio & Guerrero, 1995; Menken, 2000; Walqui, 2000).

In this Framework, States are encouraged to re-examine the theoretical
(e.g., theories of second language acquisition, English language
development/acquisition, sociocultural development,
sociolinguistics)[^7] and research bases for their definition of English
language proficiency, as well as evaluate the breadth, depth,
complexity, and articulation of the associated skills and knowledge
expected of students as they progress toward and achieve "proficiency"
(See relevant Suggested Elements and Possible Considerations.). Clarity
of the theoretical and research-based underpinnings of the State's
definition of English language proficiency and the manner and
purposefulness with which the State uses such a foundation to drive its
standards development and implementation are integral to ensuring
high-quality standards.

# The Heterogeneity of the English Language Learner Population

English language learners are diverse in their educational history,
level of literacy, background experiences, sociocultural practices, and
socioeconomic status. This diversity[^8] may affect the ways in which
different groups of ELLs access and interpret academic content in
curriculum, instruction, and assessment (Abedi, 2004; Abedi & Dietel,
2004; Kopriva, 2000; Solano-Flores & Trumbull, 2003). Some ELLs also
will be eligible for a range of special education services, if they have
been identified as having learning or other disabilities.

In this Framework, States are encouraged to involve experts who have
familiarity with the range of diversity represented in their ELL
population (e.g., in standards review, professional development
regarding the standards and their instruction) in the development and
implementation of ELP standards. These experts should be included among
other experts (e.g., applied linguists, experts in second language
acquisition, English language development) at appropriate points in the
design, development, review, and implementation of the State's ELP
standards (See relevant Suggested Elements and Possible
Considerations.). Consideration of the heterogeneity of the State's ELL
population should *not be interpreted* to suggest different standards or
achievement expectations for different ELL subgroups; only one set of
ELP standards should be developed for all of the State's ELLs. Given
that standards are major levers for improving student achievement by
articulating goals and focusing instruction, the standards should be
accessible to all students. That is, the State's ELP standards should be
conceptualized and designed so that they guide and support instruction
that can be differentiated, as necessary, to meet the varying needs of
the State's ELLs.

# State Context Factors: History, Policies, Resources

There are a number of political and resource issues that affect the
development and implementation of standards in States. These contextual
factors, specific and unique for each State, include system readiness,
familiarity with the population, past and current practices and
programs, and structures and resources to support the instruction and
assessment of the population. For example, States that have a history of
large numbers of ELLs (e.g., California, Texas, Florida) will have
different policies, practices/structures, and resources available to
support the achievement of their ELLs compared to states that have
recently experienced significant increases in ELLs ("rapid-growth"
States such as South Carolina, Kentucky, and Indiana have experienced an
increase in their PreK-12 public school ELL population by more than 400%
in the 2004-2005 school year from the 1994-1995 school year \[National
Clearinghouse for English Language Acquisition, 2006\].).

Overview of Suggested Elements for High-Quality English Language
Proficiency Standards

Below is an overview of the suggested elements related to high-quality
ELP standards. As mentioned previously, suggested elements are listed
separately in this Framework to facilitate State processes for
addressing the elements and to reflect their individual contribution to
a valid and reliable ELP standards system. Although these elements are
listed separately, States should keep in mind the inter-relationships
among these suggested elements for establishing a comprehensive and
cohesive plan for addressing the needs of their ELLs. Each suggested
element is described in greater detail in the following section.

1.1: The State has a comprehensive and coherent plan for its English
language proficiency (ELP) standards.

1.2: The State's ELP standards are developed on sound theoretical and
research bases relevant to English language acquisition and English
language proficiency.

1.3: The purpose of the State's ELP standards (i.e., to guide
instruction and assessment) is clearly stated.

1.4: The ELL population for which the State's ELP standards are intended
is clearly described, taking into account the heterogeneity of the
population (i.e., major relevant demographic, socio-cultural, and
language characteristics).

1.5: The State's ELP standards include language skills and knowledge
necessary for students' achievement of challenging State academic
content.

1.6: The structure of the State's ELP standards (format,
organization/hierarchy, levels of detail) is appropriate for the
instructional and assessment (including reporting) purposes and uses of
the ELP standards.

1.7: The State's ELP standards are stated with sufficient clarity and
definition to guide curriculum and assessment development for the ELL
population.

1.8: The State's ELP standards are articulated vertically and
horizontally.

1.9: The State ELP standards, including those in the four recognized
language domains of listening, speaking, reading, and writing, are
aligned[^9] with *the achievement of* challenging State academic content
and student academic achievement standards (based on Title III, Section
3113); that is, the State ELP standards are *linked* to the State
academic content standards.

1.10: The State has ensured alignment between its ELP standards and
language proficiency level descriptors.

1.11: The State's system of ELP standards and assessments is aligned.

1.12: The State provides guidance and training to LEAs---both ESL
teachers and content area teachers---on the ELP standards and their
purpose and use.

1.13: The State monitors and evaluates the implementation of ELP
standards in LEAs, schools, and classrooms.

**SECTION 1: ENGLISH LANGUAGE PROFICIENCY STANDARDS**

**Suggested Elements for High-Quality English Language Proficiency
Standards**

+-------------------------------+--------------------------------------+
| **Suggested Element**[^10]    | **Possible Considerations for        |
|                               | Addressing the Suggested             |
|                               | Element**[^11]                       |
+===============================+======================================+
| 1.1                           | The State has ELP standards that:    |
|                               |                                      |
| The State has a comprehensive | -   Are appropriate for each grade   |
| and coherent plan for its     |     range (across grades K-12);      |
| English language proficiency  |                                      |
| (ELP) standards to ensure     | -   Are appropriate for each of the  |
| that its standards:           |     four language domains:           |
|                               |     listening, speaking, reading,    |
| -   Are appropriate for       |     and writing;                     |
|     grades K-12;              |                                      |
|                               | -   Are appropriate for each         |
| AND                           |     language proficiency level       |
|                               |     within each grade range defined  |
| -   Include specific language |     by the State;                    |
|     skill and knowledge       |                                      |
|     expectations for each     | -   Incorporate language skills      |
|     grade range and language  |     consistent with higher-order     |
|     proficiency level defined |     thinking skills and              |
|     by the State;             |     understanding[^12] necessary for |
|                               |     achievement in the content       |
| AND                           |     areas;                           |
|                               |                                      |
| -   Cover each of the four    | -   Address the needs of the State's |
|     language domains:         |     English language learners        |
|     listening, speaking,      |     (ELLs); and                      |
|     reading, and writing;     |                                      |
|                               | -   Are based on relevant theory     |
| AND                           |     and/or research.                 |
|                               |                                      |
| -   Can be applied to *all*   | The State's plan for the development |
|     eligible public school    | and validation of its ELP standards  |
|     students and students in  | is comprehensive and                 |
|     the State.                | coherent---documentation of the plan |
|                               | is complete, accurate, and clearly   |
|                               | written so that the range of users   |
|                               | and the general public can readily   |
|                               | understand the State's plan for its  |
|                               | ELP standards. Related materials     |
|                               | also include a description of the    |
|                               | relationship between the State's ELP |
|                               | standards and the State's academic   |
|                               | content standards (e.g., how and the |
|                               | degree to which the skills and       |
|                               | knowledge of the standards are       |
|                               | intended to converge/diverge[^13]).  |
|                               |                                      |
|                               | The State's ELP standards should be  |
|                               | developed with broad input from      |
|                               | within and outside of the            |
|                               | educational community. Such input    |
|                               | might come from committees of        |
|                               | specialists with relevant knowledge  |
|                               | of curriculum (PK-16), instruction,  |
|                               | assessment, measurement,             |
|                               | accountability, policy, content, and |
|                               | the ELL population                   |
|                               | *[and]{.underline}* also from the    |
|                               | State's Technical Advisory           |
|                               | Committee, surveys, public hearings, |
|                               | public comment, or public review.    |
|                               |                                      |
|                               | The composition of groups involved   |
|                               | in standards setting reflects a      |
|                               | broad range of relevant stakeholders |
|                               | and experts represented in the       |
|                               | process, including individuals       |
|                               | knowledgeable and concerned about    |
|                               | the State's ELL population,          |
|                               | including ELLs with disabilities.    |
+-------------------------------+--------------------------------------+
| 1.2                           | The State's process for developing   |
|                               | its ELP standards includes clear     |
| The State's ELP standards are | descriptions of:                     |
| developed on sound            |                                      |
| theoretical and research      | -   Expectations for increasing      |
| bases relevant to English     |     language proficiency levels in   |
| language acquisition and      |     English that build upon skills   |
| English language proficiency. |     in preceding levels;             |
|                               |                                      |
|                               | -   Expectations that include        |
|                               |     language skills needed to        |
|                               |     succeed in the content areas     |
|                               |     (academic language[^14]),        |
|                               |     including language skills        |
|                               |     consistent with higher-order     |
|                               |     thinking skills and              |
|                               |     understanding necessary for      |
|                               |     achievement in the content       |
|                               |     areas;                           |
|                               |                                      |
|                               | -   Expectations related to the      |
|                               |     specific language skills and     |
|                               |     knowledge needed for English     |
|                               |     language proficiency;            |
|                               |                                      |
|                               | -   The relationships among the      |
|                               |     language domains (listening,     |
|                               |     speaking, reading, writing);     |
|                               |                                      |
|                               | > AND                                |
|                               |                                      |
|                               | -   The State has theory- and/or     |
|                               |     research-based rationale for the |
|                               |     ELP and academic language        |
|                               |     expectations---that is, the      |
|                               |     State has theory- and/or         |
|                               |     research-based support for the   |
|                               |     State's decisions regarding its  |
|                               |     definition of ELP and the        |
|                               |     language skills and knowledge    |
|                               |     associated with its grade ranges |
|                               |     and language proficiency levels. |
|                               |                                      |
|                               | State materials regarding the        |
|                               | State's definition of English        |
|                               | language proficiency are complete,   |
|                               | accurate, and clearly written so     |
|                               | that the range of users and the      |
|                               | general public can readily           |
|                               | understand the domain definition     |
|                               | underlying the State ELP standards.  |
|                               |                                      |
|                               | The State's definition of ELP should |
|                               | be developed with broad input from   |
|                               | within and outside of the            |
|                               | educational community. Such input    |
|                               | may come from individuals and        |
|                               | organizations with expertise in      |
|                               | English language acquisition and     |
|                               | development, applied linguistics,    |
|                               | standards, assessment, and           |
|                               | measurement, and with the ELL        |
|                               | population. The process ensures      |
|                               | appropriate accessibility of the     |
|                               | standards; that is, language (words, |
|                               | phrases), symbols, and content that  |
|                               | are generally regarded as biased     |
|                               | against members of the ELL           |
|                               | population have been removed.        |
+-------------------------------+--------------------------------------+
| 1.3                           | The purpose of the State's ELP       |
|                               | standards to guide instruction and   |
| The purpose of the State's    | assessment is clearly stated,        |
| ELP standards (i.e., to guide | including appropriate descriptions   |
| instruction and assessment)   | of the differentiation among         |
| is clearly stated.            | expectations related to              |
|                               | identification, placement, and       |
|                               | progress monitoring, such that all   |
|                               | eligible ELLs have appropriate       |
|                               | access to services and the           |
|                               | opportunity to develop proficiency   |
|                               | in the English language and acquire  |
|                               | language necessary for achievement   |
|                               | in the academic content areas.       |
|                               |                                      |
|                               | State materials related to the ELP   |
|                               | standards are complete, accurate,    |
|                               | and clearly written so that the      |
|                               | range of users and the general       |
|                               | public can readily understand the    |
|                               | purpose of the State ELP standards.  |
|                               |                                      |
|                               | State materials that include or are  |
|                               | based on the ELP standards           |
|                               | explicitly address the purpose and   |
|                               | use of the standards vis-à-vis       |
|                               | relevant local, State, and federal   |
|                               | policy/requirements for ELLs.        |
|                               |                                      |
|                               | The State's articulation of the      |
|                               | purpose of its ELP standards should  |
|                               | include broad input from within and  |
|                               | outside of the educational           |
|                               | community. Such input may come from  |
|                               | the range of relevant stakeholders   |
|                               | and experts familiar with the        |
|                               | State's curriculum, instruction,     |
|                               | assessment, policy, second language  |
|                               | acquisition, and ELL population.     |
|                               |                                      |
|                               | The composition of groups involved   |
|                               | in articulating the purpose of the   |
|                               | State's ELP standards reflects a     |
|                               | broad range of relevant stakeholders |
|                               | and experts represented in the       |
|                               | process, including individuals       |
|                               | knowledgeable and concerned about    |
|                               | the State's ELL population,          |
|                               | including ELLs with disabilities.    |
+-------------------------------+--------------------------------------+
| 1.4                           | The State's process for defining its |
|                               | ELL population involves multiple     |
| The ELL population for which  | sources of information, including    |
| the State's ELP standards are | State data, input from diverse       |
| intended is clearly           | panels of educators, higher          |
| described, taking into        | education representatives, parents,  |
| account the heterogeneity of  | and community members familiar with  |
| the population (i.e., major   | the ELL population and their         |
| relevant sociodemographic,    | educational needs, and experts in    |
| sociocultural, and language   | second language acquisition,         |
| characteristics).[^15]        | language testing, and the ELL        |
|                               | population. The broad range of       |
|                               | relevant stakeholders and experts    |
|                               | represented in the process also      |
|                               | includes individuals knowledgeable   |
|                               | and concerned about the State's ELL  |
|                               | population, including ELLs with      |
|                               | disabilities.[^16]                   |
|                               |                                      |
|                               | State materials defining/describing  |
|                               | its ELL population are complete,     |
|                               | accurate, and clearly written so     |
|                               | that the range of users and the      |
|                               | general public can readily           |
|                               | understand the population            |
|                               | definition.                          |
+-------------------------------+--------------------------------------+
| 1.5                           | The State's process for identifying  |
|                               | and defining academic language---the |
| The State's ELP standards     | language skills students need in     |
| include language skills and   | each of the language domains         |
| knowledge necessary for       | (listening, speaking, reading,       |
| students' achievement of      | writing) to access information and   |
| challenging State academic    | achieve in the academic content      |
| content.                      | areas, including language skills     |
|                               | consistent with higher-order         |
|                               | thinking skills and understanding    |
|                               | necessary for achievement in the     |
|                               | content areas, includes:             |
|                               |                                      |
|                               | -   A clear definition of academic   |
|                               |     language supported by relevant   |
|                               |     theory and/or research; that is, |
|                               |     the State has theory- and/or     |
|                               |     research-based support for the   |
|                               |     State's decisions regarding its  |
|                               |     definition of academic language  |
|                               |     and the language skills and      |
|                               |     knowledge associated with its    |
|                               |     grade ranges and language        |
|                               |     proficiency levels;              |
|                               |                                      |
|                               | -   Criteria and procedures          |
|                               |     appropriate for identifying and  |
|                               |     defining the language skills and |
|                               |     knowledge necessary to achieve   |
|                               |     grade-level content in the       |
|                               |     academic content areas. Criteria |
|                               |     should be theory- and/or         |
|                               |     research-based. Procedures could |
|                               |     include a review of academic     |
|                               |     language in curriculum           |
|                               |     materials, study guides, and     |
|                               |     textbooks related to the         |
|                               |     implementation of the academic   |
|                               |     content standards; and           |
|                               |                                      |
|                               | -   Involvement of individuals with  |
|                               |     relevant expertise (e.g.,        |
|                               |     applied linguistics, language    |
|                               |     testing, content area expertise, |
|                               |     curriculum and instruction, and  |
|                               |     the ELL population).             |
|                               |                                      |
|                               | State materials include a clear      |
|                               | explanation of the relationship      |
|                               | between the ELP standards and        |
|                               | language needed for achievement of   |
|                               | grade-level academic content         |
|                               | standards (e.g., a crosswalk of      |
|                               | academic language skills and the     |
|                               | language needed to achieve in the    |
|                               | content areas) for each grade range  |
|                               | and language proficiency level       |
|                               | defined by the State. The materials  |
|                               | are prepared in a manner that allows |
|                               | for the full range of users to       |
|                               | appropriately access and readily     |
|                               | understand the information           |
|                               | presented.                           |
|                               |                                      |
|                               | (See related Suggested Elements 1.1  |
|                               | and 1.9)                             |
+-------------------------------+--------------------------------------+

+-------------------------------+--------------------------------------+
| **Suggested Element**         | **Possible Considerations for        |
|                               | Addressing the Suggested Element**   |
+===============================+======================================+
| 1.6                           | The State's process for specifying   |
|                               | the structure of its ELP standards   |
| The structure of the State's  | appropriately                        |
| ELP standards (format,        | addresses/accommodates:              |
| organization/hierarchy,       |                                      |
| levels of detail) is          | -   Expectations for increasing      |
| appropriate for the           |     English language proficiency     |
| instructional and assessment  |     levels reflected in the          |
| (including reporting)         |     standards;                       |
| purposes and uses of the ELP  |                                      |
| standards.                    | -   Expectations for the specific    |
|                               |     language skills and knowledge    |
|                               |     needed for English language      |
|                               |     proficiency;                     |
|                               |                                      |
|                               | -   Expectations of language skills  |
|                               |     needed to achieve in the         |
|                               |     academic content areas (academic |
|                               |     language);                       |
|                               |                                      |
|                               | -   The theory and/or research bases |
|                               |     for the ELP standards; and       |
|                               |                                      |
|                               | -   The purpose(s) and use(s) of the |
|                               |     standards in instruction,        |
|                               |     assessment, and reporting.       |
|                               |                                      |
|                               | The State's process includes         |
|                               | consideration of and meaningful      |
|                               | decisions related to the:            |
|                               |                                      |
|                               | -   Number of standards;             |
|                               |                                      |
|                               | -   Level of detail included in the  |
|                               |     standards (e.g., supporting      |
|                               |     skills, indicators, enablers,    |
|                               |     benchmarks)---standards should   |
|                               |     be parsimonious;                 |
|                               |                                      |
|                               | -   Wording of the standards         |
|                               |     (standards statements should     |
|                               |     include wording that is precise  |
|                               |     and enables implementation and   |
|                               |     measurement of the standards;    |
|                               |     wording should be as clear and   |
|                               |     as consistent as possible);      |
|                               |                                      |
|                               | -   Differentiation between          |
|                               |     statements of expected student   |
|                               |     learning outcomes (the *what*)   |
|                               |     and suggestions for              |
|                               |     curriculum/instruction (the      |
|                               |     *how*)---standards should focus  |
|                               |     on the former;                   |
|                               |                                      |
|                               | -   Specific language skills and     |
|                               |     knowledge needed for English     |
|                               |     language proficiency;            |
|                               |                                      |
|                               | -   Differences in the standards     |
|                               |     (e.g., wording, detail,          |
|                               |     structure) that reflect *real*   |
|                               |     differences affecting structure  |
|                               |     (e.g., differences due to the    |
|                               |     language domain, nature of the   |
|                               |     skill/set of skills, language    |
|                               |     proficiency level, grade range); |
|                               |                                      |
|                               | -   Degree of overlap of language    |
|                               |     skills and knowledge versus      |
|                               |     delineation within and across    |
|                               |     language proficiency levels as   |
|                               |     well as within and across        |
|                               |     language domains and grade       |
|                               |     ranges;[^17] and                 |
|                               |                                      |
|                               | -   Use(s) of the standards in       |
|                               |     instruction, assessment, and     |
|                               |     reporting.                       |
|                               |                                      |
|                               | State materials clearly explain the  |
|                               | structure of the standards (format,  |
|                               | organization, levels of detail)      |
|                               | vis-à-vis their purpose(s) and       |
|                               | use(s) in instruction, assessment,   |
|                               | and reporting. The materials are     |
|                               | prepared in a manner that allows for |
|                               | the full range of users to           |
|                               | appropriately access and readily     |
|                               | understand the information           |
|                               | presented.                           |
|                               |                                      |
|                               | The State's process for specifying   |
|                               | the structure of its ELP standards   |
|                               | (overall and within and across the   |
|                               | language domains, language           |
|                               | proficiency levels, and grade        |
|                               | ranges) should include broad input   |
|                               | from within and outside of the       |
|                               | educational community. Such input    |
|                               | might come from committees of        |
|                               | specialists with relevant knowledge  |
|                               | of curriculum (PK-16), instruction,  |
|                               | assessment, measurement,             |
|                               | accountability, policy, content, and |
|                               | the ELL population, and also from    |
|                               | the State's Technical Advisory       |
|                               | Committee.                           |
|                               |                                      |
|                               | The composition of groups involved   |
|                               | in specifying the structure of the   |
|                               | State's ELP standards reflects a     |
|                               | broad range of relevant stakeholders |
|                               | and experts represented in the       |
|                               | process, including individuals       |
|                               | knowledgeable and concerned about    |
|                               | the State's ELL population,          |
|                               | including ELLs with disabilities.    |
+-------------------------------+--------------------------------------+

+-------------------------------+--------------------------------------+
| **Suggested Element**         | **Possible Considerations for        |
|                               | Addressing the Suggested Element**   |
+===============================+======================================+
| 1.7                           | The State's ELP standards include    |
|                               | specific language skills and         |
| The State ELP standards are   | knowledge that can be                |
| stated with sufficient        | operationalized in curriculum and    |
| clarity and definition to     | assessment---the ELP standards and   |
| guide curriculum and          | the student learning outcomes        |
| assessment development for    | reflected in the standards are       |
| the ELL population.           | stated parsimoniously and precisely  |
|                               | in order to enable implementation    |
|                               | (curriculum, instruction) and        |
|                               | measurement (assessment). The        |
|                               | State's process includes the         |
|                               | consideration of:                    |
|                               |                                      |
|                               | -   Level of detail included in the  |
|                               |     standards (e.g., supporting      |
|                               |     skills, indicators, enablers,    |
|                               |     benchmarks)---standards should   |
|                               |     be parsimonious;                 |
|                               |                                      |
|                               | -   Wording of the standards         |
|                               |     (standards statements should     |
|                               |     include wording that is precise  |
|                               |     and enables implementation and   |
|                               |     measurement of the standards;    |
|                               |     wording should be as clear and   |
|                               |     as consistent as possible);      |
|                               |                                      |
|                               | -   Differentiation between          |
|                               |     statements of expected student   |
|                               |     learning outcomes (the *what*)   |
|                               |     and suggestions for              |
|                               |     curriculum/instruction (the      |
|                               |     *how*)---standards should focus  |
|                               |     on the former;                   |
|                               |                                      |
|                               | -   Degree of overlap of language    |
|                               |     skills and knowledge versus      |
|                               |     delineation within and across    |
|                               |     language proficiency levels as   |
|                               |     well as within and across        |
|                               |     language domains and grade       |
|                               |     ranges; and                      |
|                               |                                      |
|                               | -   The heterogeneity of the State's |
|                               |     ELL population.[^18]             |
|                               |                                      |
|                               | State documentation illustrates how  |
|                               | the ELP standards are linked to      |
|                               | grade-level content standards        |
|                               | (academic language) for curriculum   |
|                               | and instructional purposes,          |
|                               | including language skills consistent |
|                               | with higher-order thinking skills    |
|                               | and understanding necessary for      |
|                               | achievement in the content areas.    |
|                               |                                      |
|                               | State documentation includes         |
|                               | descriptions and/or examples of key  |
|                               | language competencies for each       |
|                               | language proficiency level and each  |
|                               | language domain across the State's   |
|                               | grade ranges, including examples of  |
|                               | academic language necessary for      |
|                               | achievement in the content           |
|                               | areas---the expectation of what      |
|                               | English language skill/knowledge is  |
|                               | to be acquired and when it is to be  |
|                               | acquired is clear.                   |
|                               |                                      |
|                               | State documentation clearly          |
|                               | articulates and explains criteria    |
|                               | defining the distinction between     |
|                               | English language skills and          |
|                               | knowledge appropriate for State      |
|                               | versus local assessments.[^19]       |
|                               |                                      |
|                               | The State's process for clearly      |
|                               | defining its ELP standards to guide  |
|                               | curriculum and assessment should     |
|                               | include broad input from within and  |
|                               | outside of the educational           |
|                               | community. Such input might come     |
|                               | from committees of specialists with  |
|                               | relevant knowledge of curriculum     |
|                               | (PK-16), instruction, assessment,    |
|                               | measurement, accountability, policy, |
|                               | content, and the ELL population, and |
|                               | also from the State's Technical      |
|                               | Advisory Committee.                  |
|                               |                                      |
|                               | The composition of groups involved   |
|                               | in clearly defining the ELP          |
|                               | standards to guide curriculum and    |
|                               | assessment (e.g., state curricular   |
|                               | documents/frameworks/guides, state   |
|                               | mandated curriculum, state-mandated  |
|                               | assessment, local assessment)        |
|                               | reflects a broad range of relevant   |
|                               | stakeholders and experts represented |
|                               | in the process, including            |
|                               | individuals knowledgeable and        |
|                               | concerned about the State's ELL      |
|                               | population, including ELLs with      |
|                               | disabilities.                        |
+-------------------------------+--------------------------------------+
| 1.8                           | For *vertical* articulation of its   |
|                               | ELP standards, the State has a       |
| The State ELP standards are   | process for ensuring that, as skills |
| articulated[^20] vertically   | and knowledge move from lowest       |
| and horizontally.             | language proficiency level and grade |
|                               | level or grade range to highest:     |
|                               |                                      |
|                               | -   Levels of language skills and    |
|                               |     knowledge appropriately increase |
|                               |     in complexity, and the           |
|                               |     incremental increase is based on |
|                               |     theory and/or research;          |
|                               |                                      |
|                               | -   Prerequisite language skills and |
|                               |     knowledge appear as appropriate  |
|                               |     in lower language proficiency    |
|                               |     levels and grade ranges;         |
|                               |                                      |
|                               | -   Broader, deeper, and new         |
|                               |     language skills and knowledge    |
|                               |     appear in higher language        |
|                               |     proficiency levels and grade     |
|                               |     ranges (building from skills and |
|                               |     knowledge in lower/prior levels) |
|                               |     and are introduced at the        |
|                               |     appropriate language proficiency |
|                               |     level and grade range;           |
|                               |                                      |
|                               | -   Language skills and knowledge    |
|                               |     that are expected to be acquired |
|                               |     are explicit at the appropriate  |
|                               |     language proficiency level and   |
|                               |     grade range;                     |
|                               |                                      |
|                               | -   The balance of                   |
|                               |     representation---coverage and    |
|                               |     complexity of language skills    |
|                               |     and knowledge---shifts           |
|                               |     appropriately across language    |
|                               |     proficiency levels and grade     |
|                               |     range; and                       |
|                               |                                      |
|                               | -   Repetition of standards          |
|                               |     (language skills and knowledge)  |
|                               |     is purposeful, and it is clear   |
|                               |     what English language            |
|                               |     skill/knowledge is to be         |
|                               |     acquired and when it is to be    |
|                               |     acquired.                        |
|                               |                                      |
|                               | For *horizontal* articulation of its |
|                               | ELP standards, the State has a       |
|                               | process for ensuring that skills and |
|                               | knowledge are appropriate for the    |
|                               | language proficiency levels and      |
|                               | grade ranges for each language       |
|                               | domain:                              |
|                               |                                      |
|                               | -   The level of complexity of       |
|                               |     language skills and knowledge    |
|                               |     across language domains are      |
|                               |     comparable within a grade range, |
|                               |     unless differences are supported |
|                               |     by theory and/or research;       |
|                               |                                      |
|                               | -   Repetition of language skills    |
|                               |     and knowledge across the         |
|                               |     language domains within a        |
|                               |     language proficiency level and   |
|                               |     grade range is purposeful and    |
|                               |     meaningful.                      |
|                               |                                      |
|                               | The State has a clear depiction      |
|                               | (e.g., graphic, written description) |
|                               | of the vertical and horizontal       |
|                               | relationships of the State ELP       |
|                               | standards across language domains,   |
|                               | language proficiency levels, and     |
|                               | grade ranges.                        |
|                               |                                      |
|                               | State documents explicitly address   |
|                               | the vertical and horizontal          |
|                               | articulation of the language skills  |
|                               | and knowledge of the ELP standards   |
|                               | within and across language domains,  |
|                               | language proficiency levels, and     |
|                               | grade ranges as appropriate, as well |
|                               | as the process/methodology for       |
|                               | establishing vertical and horizontal |
|                               | articulation of its ELP standards.   |
|                               |                                      |
|                               | Qualifications of individuals        |
|                               | involved in the determination of the |
|                               | vertical and horizontal articulation |
|                               | of the ELP standards include         |
|                               | expertise in curriculum,             |
|                               | instruction, standards, assessment,  |
|                               | measurement, content, English        |
|                               | language acquisition, applied        |
|                               | linguistics, and the ELL population. |
|                               |                                      |
|                               | The State's process for              |
|                               | conceptualizing and determining the  |
|                               | vertical and horizontal articulation |
|                               | of its ELP standards (overall and    |
|                               | within and across the language       |
|                               | domains, language proficiency        |
|                               | levels, and grade ranges) should     |
|                               | include broad input from within and  |
|                               | outside of the educational           |
|                               | community. Such input might come     |
|                               | from committees of specialists with  |
|                               | relevant knowledge of curriculum     |
|                               | (PK-16), instruction, assessment,    |
|                               | measurement, accountability, policy, |
|                               | content, and the ELL population, and |
|                               | the State's Technical Advisory       |
|                               | Committee.                           |
|                               |                                      |
|                               | The composition of groups involved   |
|                               | in the vertical and horizontal       |
|                               | articulation of the State's ELP      |
|                               | standards reflects a broad range of  |
|                               | relevant stakeholders and experts    |
|                               | represented in the process,          |
|                               | including individuals knowledgeable  |
|                               | and concerned about the State's ELL  |
|                               | population, including ELLs with      |
|                               | disabilities.                        |
|                               |                                      |
|                               | [Note]{.underline}: While this       |
|                               | element is presented in terms of     |
|                               | *articulation,* aspects of           |
|                               | *alignment* and *linkage* also are   |
|                               | relevant. See other relevant         |
|                               | Suggested Elements for more          |
|                               | information (e.g., Suggested Element |
|                               | 1.9).                                |
+-------------------------------+--------------------------------------+
| 1.9                           | The State has a process for ensuring |
|                               | a match (linkage) between ELP        |
| The State's ELP standards,    | language skills in each of the       |
| including those in the four   | language domains (listening,         |
| recognized language domains   | speaking, reading, writing) and the  |
| of listening, speaking,       | language skills students need to     |
| reading, and writing, are     | achieve in the academic content      |
| aligned[^21] with the         | areas:                               |
| achievement of challenging    |                                      |
| State academic content and    | -   All relevant aspects of          |
| student academic achievement  |     grade-level academic content     |
| standards (based on Title     |     language are identified and      |
| III, Section 3113); that is,  |     considered: forms, functions, as |
| the State's ELP standards are |     well as language skills          |
| linked to the State academic  |     consistent with higher-order     |
| content standards.            |     thinking skills and              |
|                               |     understanding necessary for      |
|                               |     achievement in the content       |
|                               |     areas;                           |
|                               |                                      |
|                               | -   The ways in which and degree to  |
|                               |     which language and content       |
|                               |     converge/diverge is clear;[^22]  |
|                               |                                      |
|                               | -   At a minimum, language skills    |
|                               |     and knowledge in the ELP         |
|                               |     standards are evaluated          |
|                               |     according to the degree to which |
|                               |     they:                            |
|                               |                                      |
|                               | ```{=html}                           |
|                               | <!-- -->                             |
|                               | ```                                  |
|                               | -   Cover relevant aspects of        |
|                               |     grade-level academic content     |
|                               |     language (breadth, depth,        |
|                               |     complexity---as described        |
|                               |     previously) in the State's       |
|                               |     academic content standards;      |
|                               |                                      |
|                               | -   Cover the range of language      |
|                               |     complexity necessary for         |
|                               |     achievement in the content       |
|                               |     areas, including levels of       |
|                               |     language complexity consistent   |
|                               |     with higher-order thinking       |
|                               |     skills and understanding         |
|                               |     reflected in the State's         |
|                               |     academic content standards; and  |
|                               |                                      |
|                               | -   Reflect a balance of             |
|                               |     representation---coverage of     |
|                               |     language skills and knowledge    |
|                               |     and complexity of language       |
|                               |     skills and                       |
|                               |     knowledge---consistent with the  |
|                               |     language needed for achievement  |
|                               |     of grade-level academic content  |
|                               |     reflected in the State's         |
|                               |     academic content standards.      |
|                               |                                      |
|                               | The State has documents related to   |
|                               | the match (linkage) between the      |
|                               | State ELP standards and the State    |
|                               | academic content standards that      |
|                               | indicates that these standards were  |
|                               | reviewed specifically to evaluate    |
|                               | the degree of concurrence between    |
|                               | these sets of standards, focusing on |
|                               | language skills and knowledge in     |
|                               | general and particularly on the      |
|                               | language skills and knowledge        |
|                               | necessary to achieve the grade-level |
|                               | content in the academic content      |
|                               | areas. Documentation could include a |
|                               | crosswalk of language skills         |
|                               | expected in the ELP standards and    |
|                               | the academic content standards. The  |
|                               | process/methodology for establishing |
|                               | the linkage between the State ELP    |
|                               | standards and the State academic     |
|                               | content standards is included in     |
|                               | relevant State documents.            |
|                               |                                      |
|                               | Qualifications of individuals        |
|                               | involved in linking the ELP          |
|                               | standards to State academic content  |
|                               | standards include expertise in       |
|                               | curriculum, instruction, standards,  |
|                               | assessment, measurement, content,    |
|                               | English language acquisition,        |
|                               | applied linguistics, and the ELL     |
|                               | population.                          |
|                               |                                      |
|                               | The composition of groups involved   |
|                               | in linking the ELP standards to      |
|                               | State academic content standards     |
|                               | reflects a broad range of relevant   |
|                               | stakeholders and experts represented |
|                               | in the process, including            |
|                               | individuals knowledgeable and        |
|                               | concerned about the State's ELL      |
|                               | population, including ELLs with      |
|                               | disabilities.                        |
|                               |                                      |
|                               | [Note]{.underline}: Other types of   |
|                               | alignment also are recommended to    |
|                               | demonstrate the validity of the ELP  |
|                               | standards and assessment systems     |
|                               | (e.g., alignment of ELP standards to |
|                               | the ELP assessment blueprint). See   |
|                               | other relevant Suggested Elements in |
|                               | both the Standards and the           |
|                               | Assessments sections of this         |
|                               | Framework.                           |
+-------------------------------+--------------------------------------+
| 1.10                          | The State's ELP proficiency level    |
|                               | descriptors fully reflect its ELP    |
| The State has ensured         | standards for each grade range and   |
| alignment between its ELP     | language domain and describe the     |
| standards and language        | language skills and knowledge        |
| proficiency level             | expectations for each language       |
| descriptors.                  | proficiency level from "beginning"   |
|                               | through "advanced," including        |
|                               | appropriate vertical articulation of |
|                               | language skills and knowledge across |
|                               | the language proficiency levels and  |
|                               | horizontal articulation across       |
|                               | language domains.                    |
|                               |                                      |
|                               | The "proficient" designation         |
|                               | represents the attainment of         |
|                               | language skills and knowledge        |
|                               | expectations for English language    |
|                               | proficiency as defined by the State  |
|                               | and based on relevant theory and     |
|                               | research (see Suggested Element      |
|                               | 1.2).                                |
|                               |                                      |
|                               | "Proficiency"---overall as well as   |
|                               | in each of the language domains (as  |
|                               | appropriate)---is defined in a       |
|                               | manner consistent with the ELP       |
|                               | standards.                           |
|                               |                                      |
|                               | Each language proficiency level      |
|                               | descriptor clearly defines the       |
|                               | language skills and knowledge for    |
|                               | the attainment of that level of      |
|                               | proficiency. The language            |
|                               | proficiency level descriptors are    |
|                               | consistent with the ELP standards in |
|                               | terms of wording, structure, and     |
|                               | use. Differences across language     |
|                               | proficiency level descriptors        |
|                               | reflect real differences in language |
|                               | skill and knowledge expectations     |
|                               | (e.g., along the expected continuum  |
|                               | of English language acquisition)     |
|                               | vis-à-vis proficiency and progress   |
|                               | toward proficiency as well as        |
|                               | functional differences (e.g., for    |
|                               | reporting purposes).                 |
|                               |                                      |
|                               | The process used to review the       |
|                               | alignment between the ELP standards  |
|                               | and language proficiency level       |
|                               | descriptors indicates that these     |
|                               | standards were reviewed specifically |
|                               | to evaluate their degree of          |
|                               | consistency, focusing on the         |
|                               | language skills and knowledge        |
|                               | necessary to progress toward and     |
|                               | achieve English language             |
|                               | proficiency.                         |
|                               |                                      |
|                               | The composition of groups involved   |
|                               | in evaluating the degree of          |
|                               | consistency between the ELP          |
|                               | standards and language proficiency   |
|                               | level descriptors reflects a broad   |
|                               | range of relevant stakeholders and   |
|                               | experts, including individuals       |
|                               | knowledgeable and concerned about    |
|                               | the State's ELL population,          |
|                               | including ELLs with disabilities     |
|                               |                                      |
|                               | [Note]{.underline}: Other types of   |
|                               | alignment also are recommended. See  |
|                               | other relevant Suggested Elements    |
|                               | for more information.                |
+-------------------------------+--------------------------------------+
| 1.11                          | See relevant Suggested Element in    |
|                               | Section 2: English Language          |
| The State's system of ELP     | Proficiency Assessments.             |
| standards and assessments is  |                                      |
| aligned.                      | [Note]{.underline}: Other types of   |
|                               | alignment also are recommended. See  |
|                               | other relevant Suggested Elements    |
|                               | for more information.                |
+-------------------------------+--------------------------------------+
| 1.12                          | The State has a process and schedule |
|                               | for communicating the purpose and    |
| The State provides guidance   | use of the State's ELP standards.    |
| and training to LEAs---both   | The process reflects broad and       |
| ESL teachers and content area | purposeful dissemination of relevant |
| teachers---on the ELP         | information/guidance and coordinated |
| standards and their purpose   | training efforts. The guidance and   |
| and use.                      | training provided by the State       |
|                               | builds upon/makes use of existing    |
|                               | structures and systems (at State and |
|                               | LEA levels), to the degree possible  |
|                               | and appropriate.                     |
|                               |                                      |
|                               | The process includes the full range  |
|                               | of relevant stakeholders familiar    |
|                               | with the State's infrastructure,     |
|                               | resources, and goals as well as with |
|                               | the State's professional             |
|                               | development, curriculum,             |
|                               | instruction, assessment, policy,     |
|                               | second language acquisition, and the |
|                               | ELL population.[^23]                 |
|                               |                                      |
|                               | Appropriate differentiation among    |
|                               | expectations related to              |
|                               | identification, placement, and       |
|                               | progress monitoring are clear in     |
|                               | order to ensure consistent           |
|                               | understanding of the implementation  |
|                               | of the standards and that all        |
|                               | eligible students have appropriate   |
|                               | access to services and opportunity   |
|                               | to develop proficiency in the        |
|                               | English language and acquire         |
|                               | language necessary for achievement   |
|                               | in the content areas.                |
|                               |                                      |
|                               | Documents related to the ELP         |
|                               | standards are complete, accurate,    |
|                               | and clearly written so that the full |
|                               | range of users and the general       |
|                               | public can readily understand the    |
|                               | content.                             |
|                               |                                      |
|                               | State materials that include or are  |
|                               | based on the ELP standards           |
|                               | explicitly address the purpose and   |
|                               | use of the standards vis-à-vis       |
|                               | relevant local, State, and federal   |
|                               | policy/requirements for ELLs.        |
|                               |                                      |
|                               | State materials illustrate/describe  |
|                               | how the ELP standards are linked to  |
|                               | grade-level content standards        |
|                               | (language skills, academic language) |
|                               | for curriculum and instructional     |
|                               | purposes, including language skills  |
|                               | consistent with higher-order         |
|                               | thinking skills and understanding    |
|                               | necessary for achievement in the     |
|                               | content areas.                       |
|                               |                                      |
|                               | State documentation includes         |
|                               | descriptions and/or examples of key  |
|                               | language competencies for each       |
|                               | language proficiency level and each  |
|                               | language domain across the State's   |
|                               | grade ranges, including examples of  |
|                               | academic language necessary for      |
|                               | achievement in the content           |
|                               | areas---the expectation of what      |
|                               | English language skill/knowledge is  |
|                               | to be acquired and when it is to be  |
|                               | acquired is clear.                   |
|                               |                                      |
|                               | Factors affecting the distinction    |
|                               | between language skills and          |
|                               | knowledge appropriate for State      |
|                               | versus local assessments are clear   |
|                               | (e.g., some standards may be         |
|                               | difficult to assess in a technically |
|                               | defensible or practical manner on a  |
|                               | Statewide assessment, such as        |
|                               | standards that require student       |
|                               | production of complex language       |
|                               | samples that necessitate a           |
|                               | combination of skills across         |
|                               | language domains).                   |
|                               |                                      |
|                               | Individuals involved in the          |
|                               | specification and development of the |
|                               | guidance materials reflect a broad   |
|                               | range of relevant stakeholders and   |
|                               | experts, including individuals       |
|                               | knowledgeable and concerned the      |
|                               | State's ELL population, including    |
|                               | ELLS with disabilities.              |
+-------------------------------+--------------------------------------+
| 1.13                          | The State has a process and schedule |
|                               | for monitoring the implementation of |
| The State monitors and        | the State's ELP standards, including |
| evaluates the implementation  | a plan for evaluating the guidance   |
| of ELP standards in LEAs,     | and training it provides to LEAs and |
| schools, and classrooms.      | teachers. To the degree appropriate  |
|                               | and feasible, this process and       |
|                               | schedule should be a part            |
|                               | of/combined with the State's         |
|                               | existing monitoring and evaluation   |
|                               | systems and structures.              |
|                               |                                      |
|                               | The State relies on multiple sources |
|                               | of data/information for evaluating   |
|                               | the fidelity and effectiveness of    |
|                               | the implementation of its ELP        |
|                               | standards (e.g., internal and        |
|                               | external monitoring, qualitative     |
|                               | data/analyses, quantitative          |
|                               | data/analyses).                      |
|                               |                                      |
|                               | State materials relevant to these    |
|                               | monitoring and evaluation processes  |
|                               | explicitly address the purpose and   |
|                               | use of the standards vis-à-vis       |
|                               | relevant local, State, and federal   |
|                               | policy/requirements for ELLs.        |
|                               |                                      |
|                               | The process includes the full range  |
|                               | of relevant stakeholders familiar    |
|                               | with the State's infrastructure,     |
|                               | resources, and goals as well as with |
|                               | the State's professional             |
|                               | development, curriculum,             |
|                               | instruction, assessment, policy,     |
|                               | evaluation, and ELL population.      |
|                               |                                      |
|                               | The process should be developed with |
|                               | broad input from within and outside  |
|                               | of the educational community. Such   |
|                               | input might come from committees of  |
|                               | specialists with relevant knowledge  |
|                               | of professional development,         |
|                               | curriculum, instruction, assessment, |
|                               | measurement, accountability, policy, |
|                               | content, evaluation, State and local |
|                               | systems, and the ELL population, and |
|                               | the State's Technical Advisory       |
|                               | Committee.                           |
+-------------------------------+--------------------------------------+

# **SECTION 2: ENGLISH LANGUAGE PROFICIENCY ASSESSMENTS**

# 

# **Section 2 Overview**

This section presents suggested elements necessary for the
implementation and evaluation of high-quality English language
proficiency (ELP) assessments. The suggested elements and possible
considerations (i.e., guidance, criteria) are based on
research-supported theory (including second language acquisition,
English language development/acquisition, socio-cultural, measurement)
and applied research relevant to the development of assessments that
will measure student progress toward and attainment of proficiency in
English, including English language skills and knowledge necessary for
achievement in the academic content areas. Although there is overlap
among the suggested elements, they are listed separately in this
Framework to facilitate State processes for addressing the elements and
to reflect their individual contribution to a valid and reliable ELP
assessment system. Although these elements are listed separately, States
should keep in mind the inter-relationships among these suggested
elements for establishing a comprehensive and cohesive plan for
addressing the needs of their English language learners (ELLs).[^24]

Generally, as States evaluate and implement their ELP assessments, they
should consider the following:

-   The characteristics of the State's ELL population;

-   The process and people involved in developing, implementing, and
    evaluating the ELP assessments;

-   The degree to which the ELP assessments fit into the State's larger
    plan for educating all of its students; and

-   The degree to which the ELP assessments align with the State's ELP
    standards.

# 

# **Issues Relevant to the Development of High Quality English Language Proficiency Assessments** 

Criteria relevant to the development and evaluation of academic content
standards and assessments have been written about extensively (AERA,
APA, NCME, 1999; Green, 1998; Messick, 1989; Webb, Horton, & O'Neal,
2002). Such criteria have been used to ensure that both standards and
assessments are of high quality and are technically sound. Some
researchers had been concerned whether the methods and procedures used
to develop assessment for general population are directly transferable
for assessment aimed at diverse ELL populations (Rabinowitz & Sato,
2005).

The applicability of such criteria to assessments for ELLs is affected
by a number of inter-related factors that include:

-   The nature of the domain (i.e., English language proficiency);

-   The heterogeneity of the ELL population;

-   Similarities and differences in the applicability of technical
    criteria;

-   Access and accommodations; and

-   State context factors (i.e., history, policies, resources).

These factors are described briefly below and are addressed where
relevant in the Suggested Elements and Possible Considerations presented
in Section 2 of this Framework. States should keep these factors in mind
as they evaluate and improve their English language proficiency
assessments for their ELL population. Doing so will ensure technically
adequate assessments that facilitate the valid measurement of English
language proficiency and achievement of these students, meet the
rigorous requirements of Title III, and yield information that will help
address the persistent achievement gap.

# The Nature of the Domain: English Language Proficiency

Clear models or frameworks of what acquiring a language involves (i.e.,
English for students who are speakers of other languages) and clear
definitions of what constitutes proficiency in English are necessary for
establishing high-quality English language proficiency (ELP) standards
and the assessments based on these standards. Researchers have a
long-standing debate on the very nature of language acquisition and
proficiency ( Del Vecchio & Guerrero, 1995; Menken, 2000; Walqui, 2000).

In this Framework, States are encouraged to re-examine the theoretical
(e.g., theories of second language acquisition, English language
development/acquisition, sociocultural development,
sociolinguistics)[^25] and research bases for the State's definition of
English language proficiency, as well as evaluate the breadth, depth,
complexity, and articulation of the associated skills and knowledge
expected of students as they progress toward and achieve "proficiency"
(See relevant Suggested Elements and Possible Considerations.). Clarity
of the theoretical and research-based underpinnings of the State's
definition of English language proficiency and the manner and
purposefulness with which the State uses such a foundation to drive its
assessment development and implementation process are integral to
ensuring high-quality assessments that fairly and accurately meet all
intended purposes of the State's assessment system.

# The Heterogeneity of the English Language Learner Population

English language learners are diverse in their educational history,
background experiences, sociocultural practices, and socioeconomic
status. This diversity[^26] may affect how different groups of ELLs
interpret academic content in curriculum, instruction, and assessment
(Abedi, 2004; Abedi & Dietel, 2004; Kopriva, 2000; Solano-Flores &
Trumbull, 2003). Some also will be eligible for a range of special
education services, if they have been identified as having learning or
other disabilities.

In this Framework, States are encouraged to involve experts who have
familiarity with the range of diversity represented in their ELL
population as they design and implement their assessments for these
students (e.g., item content and bias review, field test design and
statistical analyses of field test results, test administration
training). These experts should be included among other experts (e.g.,
applied linguists, experts in second language acquisition, English
language development, and language testing) at appropriate points in the
design, development, review, and implementation of the State's ELP
assessments (See relevant Suggested Elements and Possible
Considerations.).

Consideration of the heterogeneity of the State's ELL population should
*not be interpreted to* suggest different assessments or achievement
expectations for different ELL subgroups. Rather, the State's ELP
assessment should be accessible to all students. That is, the State's
ELP assessment and its tasks (e.g., graphics, cultural/situational
references in a test item) should reflect sensitivity to the backgrounds
and experiences (e.g., cultural, geographic) of ELLs that may affect
their performance on the assessment.

[Similarities and Differences in the Applicability of Technical
Criteria]{.underline}

Some researchers had been concerned whether the methods and procedures
used to develop assessment for general population are directly
transferable for assessment aimed at diverse ELL populations (Rabinowitz
& Sato, 2005. Some criteria may not transfer directly or may be less
critical for establishing the technical adequacy of ELP assessments.
Thus, procedures and criteria widely used for technical reviews of
assessments for non-ELL populations and the establishment of the
validity of these assessments may need to be modified before they are
applied to the assessment of more specialized populations such as ELLs.

This *does not* suggest that current psychometric processes are
insufficient. Rather, the suggestion is that some standard practices may
need to be tailored because of the particular characteristics of this
population and types of tests used to assess them for Title III
purposes. Just as there are important differences in developing and
validating assessments for general education students without
disabilities versus students with disabilities, there may be important
differences (some of which have been described previously) in developing
and validating assessments for ELLs. This has also been found true for
multiple-choice tests versus performance assessments; ELP tests
typically have performance-based sections that can suffer from small
sample sizes and affect statistical analyses due to bias or reliability
estimates (Kopriva, 2000).

In this Framework, States are encouraged to consider the specific
characteristics of their ELL population and the nature of the ELP domain
as they:

-   Articulate the purpose(s) and role of the ELP assessment in the
    State's assessment system;

-   Develop item and test design specifications and test blueprints;

-   Define the assessment's sampling/norming group and groups with whom
    they may want to pilot items/tasks;

-   Plan the State assessment's field test design and statistical
    analyses (including differential item functioning and other bias
    identification methodologies);

-   Establish criteria and protocols for bias and sensitivity reviews
    (linguistic, sociocultural, etc.) and specify the full range of
    experts and committee members within and outside of the educational
    community necessary for such reviews;

-   Develop administration and scoring protocols and specify the
    qualifications of those needed for test administration and scoring;

-   Develop a standard setting protocol and articulate language
    proficiency level descriptors consistent with the language skills
    and knowledge specified in the State ELP standards; and

-   Monitor and evaluate the implementation of the State ELP assessment,
    including examining the burden on teachers and LEAs of the
    administration, scoring, reporting, and security protocols.

Many of the procedures and criteria found appropriate and essential for
establishing the technical adequacy of assessments for ELLs are similar
to those relevant to the design, development, implementation, and
evaluation of State academic content assessments. Therefore, this
Framework encourages States to build on or extend wherever possible
their current systems (e.g., for Title I), in order to address
requirements for their ELP assessments. This will help support a more
technically sound, coherent State assessment system. As States routinely
review and revise both assessments (Title I and Title III),
opportunities to create greater coherence across the two may occur.

### [Access and Accommodations]{.underline} 

Appropriate access[^27] and accommodations[^28] for ELLs on ELP
assessments should not result in the simplification or alteration of the
test's targeted constructs or the misrepresentation of students'
achievement of English language proficiency. Rather, methods for
providing access and selecting accommodations for ELLs should be based
on sound theory (e.g., second language acquisition, English language
development/acquisition, measurement) and practice as well as
research-based evidence. These methods should be sensitive to the
specific characteristics and needs of this population of students and
the nature of the assessed domain (English language proficiency).

As discussed previously, the ELL population is unique in a number of key
ways (Liu, Anderson, Swierzbin, & Thurlow, 1999; Solano-Flores & Li,
2006) that have an impact on how to assess this population and validate
the results. Additionally, the assessed domain of English language
proficiency requires careful consideration in terms of the nature of
this domain and how it is both similar to and different from academic
content domains (e.g., English language arts, mathematics, science,
social studies). Furthermore, as ELLs develop English language
proficiency, their language abilities often develop at different rates
across language domains (i.e., listening, speaking, reading, writing),
providing another layer of complexity to ensuring appropriate access to
test content and valid measurement of their English language proficiency
skills and knowledge.

In this Framework, States are encouraged to consider factors that may
interfere with their ELLs' ability to access the tested content (e.g.,
familiarity with assessment item/task format or presentation,
familiarity with the sociocultural context and/or graphics associated
with the assessment items/tasks). Appropriate access for ELLs (both with
and without disabilities) maximizes their understanding of what is being
asked of them and enables them to fully demonstrate their English
language proficiency skills and knowledge across the language domains.
Additionally, States should consider and clearly specify strategies that
appropriately facilitate access in each language domain without
compromising the integrity of the assessment and its targeted
content/constructs in each of the language domains (e.g., oral prompting
requires listening comprehension).

With regard to accommodations on ELP assessments, some ELLs will be
eligible for special education services if they have been identified as
having learning or other disabilities, and they will require assessment
accommodations per their Individualized Education Programs (IEPs).
Therefore, in this Framework, States are encouraged to carefully
articulate their protocols guiding the selection and implementation of
accommodations appropriate for their ELP assessments. It is critical
that States allow only accommodations that will not alter the assessed
constructs, as with other student subgroups. Additionally, States should
consider and clearly specify modality-appropriate accommodations that
facilitate access for ELLs without compromising the integrity of the
assessment and its targeted content/constructs across the language
domains.

# State Context Factors: History, Policies, Resources

As discussed in Section 1 of this Framework, there are a number of
political and resource factors that affect the development of ELP
assessments and their implementation. These contextual factors include
readiness, familiarity with the population, and structures and resources
to support the instruction and assessment of the population. For
example, States that have a history of large numbers of ELLs (e.g.,
California, Texas, Florida) will have different policies,
practices/structures, and resources available to support the achievement
of their ELLs compared to states that have recently experienced
significant increases in ELLs ("rapid growth" States such as South
Carolina, Kentucky, and Indiana have experienced an increase in their
PreK-12 public school ELL population by more than 400% in the 2004-2005
school year from the 1994-1995 school year \[National Clearinghouse for
English Language Acquisition, 2006\]).

**Overview of Suggested Elements for High-Quality English Language
Proficiency Assessments**

Below is an overview of the suggested elements related to high-quality
ELP assessments. As mentioned previously, suggested elements are listed
separately in this Framework to facilitate State processes for
addressing the elements and to reflect their individual contribution to
a valid and reliable ELP assessment system. Although these elements are
listed separately, States should keep in mind the inter-relationships
among these suggested elements for establishing a comprehensive and
cohesive plan for addressing the needs of their English language
learners (ELLs). Each suggested element is described in greater detail
in the following section.

2.1: The State has a comprehensive and cohesive plan to ensure that its
English language proficiency (ELP) assessment system meets technical
requirements (validity, reliability, freedom from bias). This plan
includes ensuring that local assessments meet technical requirements
when the State's ELP assessments for Title III purposes include
assessments developed or adopted at both the local and State levels.

2.2: The State's ELP assessment system is comprised of measures of
language skills and knowledge necessary for attaining English language
proficiency and for the achievement of challenging State academic
content, and these measures are appropriate for each language domain
(listening, speaking, reading, writing) and also for comprehension.

2.3: The State's ELP assessments are valid for each of their intended
uses/purposes (i.e., initial screening, comprehensive evaluation,
progress monitoring).

2.4: The State's ELP assessment system yields coherent and valid
information for its ELL population, grade ranges, and language
proficiency levels, and yields information necessary for Title III
instructional and accountability purposes.

2.5: The State's ELP assessments are reliable for each of their intended
uses/purposes (i.e., initial screening, comprehensive evaluation,
progress monitoring).

2.6: The State's ELP assessments are fair and accessible to the State's
ELL population for which they are intended, including ELLs with
disabilities.

2.7: The State has a coherent approach to ensuring alignment between
each of its ELP assessments, or combination of assessments, and the ELP
standards and language skills and knowledge of the academic content
standards the assessments are designed to measure.

2.8: The State has a system for monitoring and improving the quality of
its assessment system, including a plan for ongoing procedures to
maintain and improve alignment between the State's ELP assessment and
ELP standards over time.

2.9: The State's reporting system facilitates appropriate, credible, and
defensible interpretation and use of its ELP assessment data.

2.10: The State reports participation and assessment results for all
ELLs in its reports at the school, LEA, and State levels, and the
reports for any group or subgroup do not reveal personally identifiable
information about an individual student.

2.11: The State produces interpretive guidance following each
administration of its ELP assessments.

2.12: The State has a validated standard-setting process, and its
associated language proficiency levels, descriptions of language
competencies for each language proficiency level, and cut scores
appropriately differentiate among English language skills and knowledge
in each language domain and each grade range that are necessary for
demonstrating progress toward and attainment of English language
proficiency, per the State's ELP standards and the theory and research
on which the standards are based.

**SECTION 2: ENGLISH LANGUAGE PROFICIENCY ASSESSMENTS**

**Suggested Elements for High-Quality English Language Proficiency
Assessments**

+-------------------------------+--------------------------------------+
| **Suggested Element**[^29]    | **Possible Considerations for        |
|                               | Addressing the Suggested             |
|                               | Element**[^30]                       |
+===============================+======================================+
| 2.1                           | The State's plan for developing,     |
|                               | implementing, and validating its ELP |
| The State has a comprehensive | assessments (initial screening,      |
| and coherent plan to ensure   | comprehensive evaluation, progress   |
| that its English language     | monitoring) for grades K-12 in the   |
| proficiency (ELP) assessment  | four language domains (listening,    |
| system meets technical        | speaking, reading, writing), and as  |
| requirements (validity,       | appropriate, for providing guidance  |
| reliability, freedom from     | to local educational agencies (LEAs) |
| bias)[^31].                   | for assessments for Title III        |
|                               | purposes ensure:                     |
| This plan includes ensuring   |                                      |
| that local assessments[^32]   | -   The assessments are accessible   |
| meet technical requirements   |     to the State's English language  |
| when the State's ELP          |     learner (ELL) population;        |
| assessments for Title III     |                                      |
| purposes include assessments  | -   The assessments are unbiased;    |
| developed or adopted at both  |                                      |
| the local and State levels.   | -   The assessments yield reliable   |
|                               |     and comparable results;          |
| This plan also includes State |                                      |
| ELP assessments employing a   | -   The assessments yield valid      |
| matrix design.[^33]           |     results and judgments based on   |
|                               |     these results (e.g., level of    |
| Assessments may include:[^34] |     English language proficiency);   |
|                               |                                      |
| -   Initial screening (for    | -   The assessments are aligned with |
|     identification purposes)  |     the State's ELP standards and    |
|                               |     will yield information           |
| -   Comprehensive evaluation  |     consistent with language         |
|     (for placement purposes)  |     proficiency levels defined by    |
|                               |     the State across the language    |
| -   Progress monitoring (for  |     domains (listening, speaking,    |
|     measuring progress toward |     reading, writing) and grade      |
|     English language          |     ranges;                          |
|     proficiency)              |                                      |
|                               | -   The content covered by the       |
|                               |     assessment (i.e., in terms of    |
|                               |     language skills and knowledge,   |
|                               |     breadth, depth, and range of     |
|                               |     complexity) reflects the ELP     |
|                               |     standards and supports the       |
|                               |     assessment's purpose;            |
|                               |                                      |
|                               | -   Local assessments for            |
|                               |     identification and/or placement) |
|                               |     yield results that can be        |
|                               |     compared and aggregated as       |
|                               |     necessary and appropriate to     |
|                               |     serve the information,           |
|                               |     accountability, and reporting    |
|                               |     needs of the State; and          |
|                               |                                      |
|                               | -   There is a clear process for     |
|                               |     refining/updating the State ELP  |
|                               |     assessment system as necessary.  |
|                               |                                      |
|                               | The State's processes for developing |
|                               | and implementing assessments         |
|                               | employing a matrix design ensure:    |
|                               |                                      |
|                               | -   All forms of an assessment are   |
|                               |     appropriately aligned with the   |
|                               |     State's ELP standards and        |
|                               |     language proficiency levels      |
|                               |     across the grade ranges and      |
|                               |     language domains;                |
|                               |                                      |
|                               | -   All forms of the assessment meet |
|                               |     the State's assessment           |
|                               |     blueprint;                       |
|                               |                                      |
|                               | -   All forms of an assessment are   |
|                               |     equivalent to one another in     |
|                               |     terms of coverage of English     |
|                               |     language skills and knowledge,   |
|                               |     range of difficulty, and         |
|                               |     quality; and                     |
|                               |                                      |
|                               | -   All assessments yield comparable |
|                               |     results across the State's ELL   |
|                               |     population.                      |
|                               |                                      |
|                               | To the degree possible and           |
|                               | appropriate, the State               |
|                               | plan/processes build on or extend    |
|                               | its existing systems/procedures      |
|                               | (e.g., for Title I) for ensuring     |
|                               | that its ELP assessments meet        |
|                               | technical requirements.              |
|                               |                                      |
|                               | Evaluations related to the technical |
|                               | quality of the local assessments     |
|                               | (i.e., comparability, reliability,   |
|                               | validity, alignment/linkage,         |
|                               | accessibility, freedom from bias,    |
|                               | usability of results/data for local  |
|                               | and State purposes) are conducted by |
|                               | the State and/or by independent      |
|                               | evaluators with relevant expertise   |
|                               | (e.g., consultants, organizations,   |
|                               | State Technical Advisory Committee), |
|                               | as appropriate, for the type of      |
|                               | assessment evaluated (i.e., initial  |
|                               | screening, comprehensive             |
|                               | evaluation).                         |
|                               |                                      |
|                               | State and/or LEA documents (e.g.,    |
|                               | technical manuals, administration    |
|                               | guides) appropriately reflect the    |
|                               | development, implementation, and     |
|                               | standardization of the assessment    |
|                               | processes/protocols (training,       |
|                               | administration, scoring, reporting,  |
|                               | security), and as appropriate,       |
|                               | address the technical aspects of ELP |
|                               | assessments employing a matrix       |
|                               | design.                              |
|                               |                                      |
|                               | The State's plan should be developed |
|                               | with broad input from within and     |
|                               | outside of the educational           |
|                               | community. Such input might come     |
|                               | from committees of specialists with  |
|                               | relevant knowledge of curriculum     |
|                               | (PK-16), instruction, assessment,    |
|                               | measurement, psychometrics,          |
|                               | accountability, policy, content, the |
|                               | State's systems and infrastructure,  |
|                               | and the State's ELL population       |
|                               | *[and]{.underline}* also from the    |
|                               | State's Technical Advisory           |
|                               | Committee.                           |
|                               |                                      |
|                               | The composition of groups involved   |
|                               | in the State's plan reflects a broad |
|                               | range of relevant stakeholders and   |
|                               | experts, including individuals       |
|                               | knowledgeable and concerned about    |
|                               | the State's ELL population including |
|                               | ELLs with disabilities.              |
+-------------------------------+--------------------------------------+
| 2.2                           | The State's procedures for           |
|                               | developing measures for each ELP     |
| The State's ELP assessment    | language domain (listening,          |
| system is comprised of        | speaking, reading, writing) as well  |
| measures of language skills   | as for comprehension reflect         |
| and knowledge necessary for   | purposeful inclusion of the breadth, |
| attaining English language    | depth, and range of complexity of    |
| proficiency and for the       | language associated with the         |
| achievement of challenging    | attainment of English language       |
| State academic content, and   | proficiency and includes language    |
| these measures are            | skills and knowledge consistent with |
| appropriate for each language | higher-order thinking skills and     |
| domain (listening, speaking,  | understanding[^35] necessary for the |
| reading, writing) and also    | achievement of challenging State     |
| for comprehension.            | academic content. The State's        |
|                               | process includes test blueprints,    |
|                               | item specifications, and criteria    |
|                               | for determining the inclusion of     |
|                               | relevant English language            |
|                               | proficiency skills and knowledge.    |
|                               |                                      |
|                               | The State has documented evidence    |
|                               | (e.g., percentages of items in       |
|                               | blueprint, results of factor         |
|                               | analyses) that supports actual (as   |
|                               | opposed to intended) inclusion of    |
|                               | appropriate language skills and      |
|                               | knowledge necessary for attaining    |
|                               | English language proficiency and for |
|                               | the achievement of challenging State |
|                               | academic content, as well as         |
|                               | evidence of adequate coverage of the |
|                               | breadth, depth, and range of         |
|                               | complexity of these language skills  |
|                               | and knowledge.                       |
|                               |                                      |
|                               | The State's procedures should be     |
|                               | developed and implemented with broad |
|                               | input from within and outside of the |
|                               | educational community. Such input    |
|                               | might come from committees of        |
|                               | specialists with relevant knowledge  |
|                               | of curriculum (PK-16), instruction,  |
|                               | assessment, measurement,             |
|                               | psychometrics, accountability,       |
|                               | policy, content, applied             |
|                               | linguistics, second language         |
|                               | acquisition, language testing, and   |
|                               | the State's ELL population           |
|                               | *[and]{.underline}* also from the    |
|                               | State's Technical Advisory           |
|                               | Committee.                           |
|                               |                                      |
|                               | The composition of groups involved   |
|                               | in the State's procedures reflects a |
|                               | broad range of relevant stakeholders |
|                               | and experts, including individuals   |
|                               | knowledgeable and concerned about    |
|                               | the State's ELL population including |
|                               | ELLs with disabilities.              |
+-------------------------------+--------------------------------------+
| 2.3                           | The State has established clear      |
|                               | criteria for the administration,     |
| The State's ELP assessments   | scoring, analysis, and reporting     |
| are **valid** for each of     | components of the State ELP          |
| their intended                | assessment system. The State         |
| uses/purposes[^36] (i.e.,     | developed a set of management        |
| initial screening,            | controls or standards for each of    |
| comprehensive evaluation,     | these components and has             |
| progress monitoring) with     | communicated these criteria to its   |
| respect to *all* of the       | contractor(s), LEAs, and schools.    |
| following categories (from    | The State requires its contractor(s) |
| AERA, APA, NCME, 1999):       | to provide specific information on   |
|                               | the degree to which each criterion   |
| a.  The purpose(s) of the     | is met.                              |
|     assessments is specified, |                                      |
|     including delineation of  | The State uses an extensive system   |
|     the types of uses and     | of training and monitoring to ensure |
|     decisions most            | that each person who is responsible  |
|     appropriate for each      | for handling or administering any    |
|     assessment;               | portion of its assessments does so   |
|     *[and]{.underline}*       | in a way that protects the security  |
|                               | of the assessments and maintains     |
| b.  The assessments are       | equivalence of administration        |
|     measuring the language    | conditions across students and       |
|     skills and knowledge      | schools. The State's system          |
|     described in the State's  | includes:                            |
|     ELP standards and *not*   |                                      |
|     knowledge, skills, or     | -   Criteria for administration,     |
|     other characteristics     |     scoring, analysis, and reporting |
|     that are not specified in |     that are communicated to State   |
|     the ELP standards for all |     contractor(s), as appropriate.   |
|     grade ranges, language    |                                      |
|     proficiency levels, and   | -   Test security policy and         |
|     language domain           |     consequences for violation that  |
|     expectations;             |     are communicated to the public   |
|     *[and]{.underline}*       |     and to local educators.          |
|                               |                                      |
| c.  The assessment items and  | -   A written plan for training and  |
|     tasks are assessing the   |     monitoring assessment            |
|     breadth, depth, and range |     administration conditions across |
|     of complexity of the      |     the State, even parts of the     |
|     intended ELP language     |     State's ELP assessment system    |
|     skills and knowledge and  |     that are local assessments.      |
|     the items and tasks are   |                                      |
|     appropriate for the grade | -   Monitoring of assessment         |
|     ranges, language          |     administration conditions across |
|     proficiency levels, and   |     the State, based on the          |
|     language domains;         |     particulars of its written plan. |
|     [a*nd*]{.underline}       |                                      |
|                               | The State has developed a validity   |
| d.  The scoring and reporting | plan that includes all appropriate   |
|     structures are consistent | categories of validity (e.g.,        |
|     with the State's          | content, predictive, construct,      |
|     definition of English     | consequential) for each intended     |
|     language proficiency as   | purpose (i.e., initial screening,    |
|     well as with the          | comprehensive evaluation, progress   |
|     definition of each        | monitoring).                         |
|     language domain           |                                      |
|     (listening, speaking,     | The State has a process and schedule |
|     reading, writing) and     | for addressing and documenting       |
|     comprehension, and their  | validity evidence in each of the     |
|     structures (i.e., item    | categories (a-g) as well as a        |
|     interrelationships are    | process and schedule for addressing  |
|     consistent with the       | any deficiencies either in the       |
|     framework from which the  | validity of the State's ELP          |
|     test arises);             | assessments or in its approach to    |
|     [a*nd*]{.underline}       | gathering/documenting validity       |
|                               | evidence.                            |
| e.  The test scores are       |                                      |
|     related to external       | Processes/evidence supporting the    |
|     variables as intended     | State's ELP assessment validity      |
|     (e.g., scores are         | include:                             |
|     correlated strongly with  |                                      |
|     relevant measures of      | -   Minutes or policies of the State |
|     English language          |     Board of Education or State      |
|     proficiency and are       |     legislative code, technical      |
|     weakly correlated, if at  |     reports, etc. that clearly       |
|     all, with irrelevant      |     define the purpose(s) and role   |
|                               |     of the State's ELP assessment    |
|  characteristics/constructs); |     system (category a)              |
|     *and*                     |                                      |
|                               | -   Studies that provide evidence    |
| f.  Decisions based on the    |     (qualitative and/or              |
|     results of the State ELP  |     quantitative) that supports the  |
|     assessments are           |     validity of interpretations/uses |
|     consistent with the       |     of results from the State's ELP  |
|     purposes for which the    |     assessments vis-à-vis their      |
|     assessments were          |     stated purpose(s) (categories    |
|     designed;                 |     b-g)                             |
|     *[and]{.underline}*       |                                      |
|                               | -   Studies that provide evidence of |
| g.  The State ELP assessments |     comparability of assessments     |
|     produce intended          |     (e.g., computer-based and        |
|     consequences, and         |     paper-based) and assessment      |
|     unintended consequences   |     results across groups and time,  |
|     have been considered and  |     including equating or linking    |
|     proactively and           |     studies that support assessment  |
|     appropriately addressed.  |     comparability and a process for  |
|                               |     addressing any deficiencies      |
|                               |     identified.                      |
|                               |                                      |
|                               | -   Studies address:                 |
|                               |                                      |
|                               | ```{=html}                           |
|                               | <!-- -->                             |
|                               | ```                                  |
|                               | -   Internal validity                |
|                               |                                      |
|                               | -   Content validity (i.e., related  |
|                               |     to the State's definition and    |
|                               |     operationalization of English    |
|                               |     language proficiency overall and |
|                               |     by language domain)              |
|                               |                                      |
|                               | -   Alignment/Linkage (see Suggested |
|                               |     Element 2.8)                     |
|                               |                                      |
|                               | -   Criterion/Predictive validity    |
|                               |                                      |
|                               | -   Construct validity               |
|                               |                                      |
|                               | -   Consequential validity           |
|                               |                                      |
|                               | -   Comparability                    |
|                               |                                      |
|                               | The State has a plan for addressing  |
|                               | issues that threaten the validity of |
|                               | its ELP assessment. The plan is      |
|                               | based on theory, research, and best  |
|                               | practices, as appropriate. And, the  |
|                               | plan involves the input of relevant  |
|                               | experts (e.g., the State's Technical |
|                               | Advisory Committee).                 |
|                               |                                      |
|                               | The State's validity plan and        |
|                               | procedures should be developed and   |
|                               | implemented with broad input from    |
|                               | within and outside of the            |
|                               | educational community. Such input    |
|                               | might come from committees of        |
|                               | specialists with relevant knowledge  |
|                               | of curriculum (PK-16), instruction,  |
|                               | assessment, measurement,             |
|                               | psychometrics, accountability,       |
|                               | policy, content, applied             |
|                               | linguistics, second language         |
|                               | acquisition, language testing, and   |
|                               | the State's ELL population           |
|                               | *[and]{.underline}* also from the    |
|                               | State's Technical Advisory           |
|                               | Committee.                           |
|                               |                                      |
|                               | The composition of groups involved   |
|                               | in the State's plan and procedures   |
|                               | reflects a broad range of relevant   |
|                               | stakeholders and experts, including  |
|                               | individuals knowledgeable and        |
|                               | concerned about the State's ELL      |
|                               | population including ELLs with       |
|                               | disabilities.                        |
+-------------------------------+--------------------------------------+
| 2.4                           | The State has a plan for ensuring    |
|                               | that for each grade range and        |
| The State's ELP assessment    | language domain (listening,          |
| system yields coherent and    | speaking, reading, writing), the ELP |
| valid information for its ELL | assessment system provides coherent  |
| population, grade ranges, and | and valid information by ensuring    |
| language proficiency levels,  | alignment/linkage of the ELP         |
| and yields information        | assessments (i.e., vertical and      |
| necessary for Title III       | horizontal alignment; alignment to   |
| instructional and             | language proficiency level           |
| accountability purposes.[^37] | descriptors; linkage to content area |
|                               | standards) as well as alignment of   |
|                               | the ELP assessments to the ELP       |
|                               | standards (see other relevant        |
|                               | Suggested Elements in this section   |
|                               | for more detail).                    |
|                               |                                      |
|                               | The State's plan ensures that for    |
|                               | each grade range and language        |
|                               | domain, the ELP assessment system    |
|                               | provides coherent and valid          |
|                               | information by ensuring that         |
|                               | assessment results appropriately     |
|                               | inform determination of Annual       |
|                               | Measurable Achievement Objectives    |
|                               | (AMAOs) and includes specification   |
|                               | of the relative contribution and     |
|                               | plan for how the assessment results  |
|                               | contribute to determining            |
|                               | AMAOs.[^38]                          |
|                               |                                      |
|                               | State materials clearly indicate the |
|                               | purpose of each assessment           |
|                               | (placement, progress, etc.) and      |
|                               | intended use of the assessment       |
|                               | results vis-à-vis Title III          |
|                               | requirements.                        |
|                               |                                      |
|                               | The State's plan and procedures      |
|                               | should be developed and implemented  |
|                               | with broad input from within and     |
|                               | outside of the educational           |
|                               | community. Such input might come     |
|                               | from committees of specialists with  |
|                               | relevant knowledge of curriculum     |
|                               | (PK-16), instruction, assessment,    |
|                               | measurement, psychometrics,          |
|                               | accountability, policy, content,     |
|                               | applied linguistics, second language |
|                               | acquisition, language testing, and   |
|                               | the State's ELL population           |
|                               | *[and]{.underline}* also from the    |
|                               | State's Technical Advisory           |
|                               | Committee.                           |
|                               |                                      |
|                               | The composition of groups involved   |
|                               | in the State's plan and procedures   |
|                               | reflects a broad range of relevant   |
|                               | stakeholders and experts, including  |
|                               | individuals knowledgeable and        |
|                               | concerned about the State's ELL      |
|                               | population including ELLs with       |
|                               | disabilities.                        |
+-------------------------------+--------------------------------------+
| 2.5                           | The State has a process for          |
|                               | addressing and documenting evidence  |
| The State's ELP assessments   | of reliability in each of the        |
| are **reliable** for each of  | categories (a-c) as well as a        |
| their intended                | process for addressing any           |
| uses/purposes[^39] (i.e.,     | deficiencies either in the           |
| initial screening,            | reliability of the State's ELP       |
| comprehensive evaluation,     | assessments or in its approach to    |
| progress monitoring) with     | gathering/documenting evidence of    |
| respect to *all* of the       | reliability.                         |
| following categories (based   |                                      |
| on AERA, APA, NCME, 1999):    | > Processes/evidence supporting the  |
|                               | > reliability of State's ELP         |
| > a\. The scores are reliable | > assessment include:                |
| > for the State's ELL         |                                      |
| > population;                 | -   Studies that provide evidence of |
| > *[and]{.underline}*         |     the precision of the assessments |
| >                             |     at cut scores and evidence of a  |
| > b\. There is evidence of    |     systematic process for           |
| > consistency:                |     addressing any deficiencies      |
| >                             |     identified;                      |
| > \-- of the conditional      |                                      |
| > standard error of           | -   Studies that provide evidence of |
| > measurement and student     |     consistency of student level     |
| > classification is           |     classification and evidence of a |
| > consistent at each cut      |     systematic process for           |
| > score across ELL subgroups  |     addressing any deficiencies      |
| > (e.g., subgroups based on   |     identified; and                  |
| > level of English language   |                                      |
| > proficiency);               | -   Studies that address:            |
| >                             |                                      |
| > \-- of item responses       | ```{=html}                           |
| > (internal consistency);\    | <!-- -->                             |
| > across forms of the test,   | ```                                  |
| > if applicable;              | -   Internal consistency and other   |
| >                             |     traditional measures of          |
| > \-- of results over time;   |     reliability;                     |
| > and                         |                                      |
| >                             | -   Inter- and intra-rater           |
| > \-- within and across       |     reliability for                  |
| > raters in scoring (intra-   |     constructed-response             |
| > and inter-rater             |     items/tasks, as appropriate; and |
| > reliability);               |                                      |
| > *[and]{.underline}*         | -   Generalizability (e.g., g        |
| >                             |     studies). []{.mark}              |
| > c\. There is                |                                      |
| > generalizability of results | The State's plan and procedures      |
| > across the State's ELL      | should be developed and implemented  |
| > population and LEAs (i.e.,  | with broad input from within and     |
| > generalizability for all    | outside of the educational           |
| > relevant sources of         | community. Such input might come     |
| > variance across groups,     | from committees of specialists with  |
| > schools, and sampled        | relevant knowledge of curriculum     |
| > content).                   | (PK-16), instruction, assessment,    |
|                               | measurement, psychometrics,          |
|                               | accountability, policy, content,     |
|                               | applied linguistics, second language |
|                               | acquisition, language testing, and   |
|                               | the State's ELL population           |
|                               | *[and]{.underline}* also from the    |
|                               | State's Technical Advisory           |
|                               | Committee.                           |
|                               |                                      |
|                               | The composition of groups involved   |
|                               | in the State's plan and procedures   |
|                               | reflects a broad range of relevant   |
|                               | stakeholders and experts, including  |
|                               | individuals knowledgeable and        |
|                               | concerned about the State's ELL      |
|                               | population including ELLs with       |
|                               | disabilities.                        |
+-------------------------------+--------------------------------------+
| 2.6                           | The State has appropriate judgmental |
|                               | (e.g., committee reviews) and        |
| The State's ELP assessments   | data-based (e.g., DIF studies)       |
| are **fair and accessible**   | procedures/processes at key points   |
| to the State's ELL population | throughout the assessment's          |
| for which they are intended,  | development and implementation to    |
| including ELLs with           | ensure that its ELP assessments are  |
| disabilities, with respect to | fair and accessible to all ELL       |
| each of the following issues: | subgroups for which they are         |
|                               | intended, including ELLs with        |
| a.  The assessments provide   | disabilities.                        |
|     an appropriate variety of |                                      |
|     accommodations for ELLs   | State materials (e.g., training      |
|     with disabilities         | materials, review guidelines)        |
|     [*and*]{.underline}       | clearly define criteria, which are   |
|                               | research based, for judging fairness |
| b.  Steps have been taken to  | and accessibility for the State's    |
|     ensure fairness and       | ELL population.                      |
|     accessibility in the      |                                      |
|     development of the        | State materials (e.g., training      |
|     assessments               | materials, administration and        |
|     *[and]{.underline}*       | technical manuals) clearly define    |
|                               | the policies and procedures for      |
| c.  The use of accommodations | selecting accommodations for use     |
|     yields meaningful scores. | with the State's ELP assessments,    |
|                               | including specifications that        |
|                               | clearly delineate which              |
|                               | accommodations may be used for       |
|                               | specific sections of the test (e.g., |
|                               | accommodations appropriate for each  |
|                               | language domain tested).             |
|                               |                                      |
|                               | There is research-based rationale    |
|                               | for the appropriateness of           |
|                               | accommodations and the impact of the |
|                               | accommodations on the test scores of |
|                               | ELLs with disabilities.              |
|                               |                                      |
|                               | The State has a process for          |
|                               | examining its accommodations in      |
|                               | terms of:                            |
|                               |                                      |
|                               | -   Their appropriateness vis-à-vis  |
|                               |     the ELLs with disabilities       |
|                               |     eligible for the State ELP       |
|                               |     assessment, and the degree to    |
|                               |     which the use of the             |
|                               |     accommodations is consistent     |
|                               |     with instructional approaches    |
|                               |     for each student, as determined  |
|                               |     by a student's IEP or 504 plan;  |
|                               |                                      |
|                               | -   Their impact on the assessed     |
|                               |     constructs (i.e., English        |
|                               |     language proficiency skills and  |
|                               |     knowledge) and the inferences    |
|                               |     based on student performance on  |
|                               |     accommodated ELP assessments;    |
|                               |                                      |
|                               | -   The degree to which scores for   |
|                               |     ELLs with disabilities that are  |
|                               |     based on accommodated            |
|                               |     administration conditions allow  |
|                               |     for valid inferences about these |
|                               |     students' English language       |
|                               |     proficiency skills and knowledge |
|                               |     and can be combined meaningfully |
|                               |     with scores from                 |
|                               |     non-accommodated administration  |
|                               |     conditions; and                  |
|                               |                                      |
|                               | -   The degree of effectiveness of   |
|                               |     specific accommodations for      |
|                               |     different groups of ELLs with    |
|                               |     disabilities.                    |
|                               |                                      |
|                               | The State has a plan and schedule    |
|                               | for routinely monitoring the extent  |
|                               | to which accommodations used during  |
|                               | ELP assessment administration are    |
|                               | consistent with those specified in   |
|                               | the IEP or 504 plans for eligible    |
|                               | ELLs with disabilities.              |
|                               |                                      |
|                               | The State's plan and procedures      |
|                               | should be developed and implemented  |
|                               | with broad input from within and     |
|                               | outside of the educational           |
|                               | community. Such input might come     |
|                               | from committees of specialists with  |
|                               | relevant knowledge of curriculum     |
|                               | (PK-16), instruction, assessment,    |
|                               | measurement, psychometrics,          |
|                               | accountability, policy, content,     |
|                               | applied linguistics, second language |
|                               | acquisition, language testing, and   |
|                               | the State's ELL population           |
|                               | *[and]{.underline}* also from the    |
|                               | State's Technical Advisory           |
|                               | Committee.                           |
|                               |                                      |
|                               | The composition of groups involved   |
|                               | in the State's plan and procedures   |
|                               | reflects a broad range of relevant   |
|                               | stakeholders and experts, including  |
|                               | individuals knowledgeable and        |
|                               | concerned about the State's ELL      |
|                               | population including ELLs with       |
|                               | disabilities.                        |
+-------------------------------+--------------------------------------+
| 2.7                           | The State has developed an           |
|                               | assessment system consistent with    |
| The State has a coherent      | its ELP standards, including English |
| approach to ensuring          | language skills and knowledge        |
| alignment between each of its | necessary for achievement in the     |
| ELP assessments, or           | academic content areas (academic     |
| combination of assessments,   | language[^40]).                      |
| and the ELP standards and     |                                      |
| language skills and knowledge | The State has a process to ensure    |
| of the academic content       | that full alignment is achieved      |
| standards the assessments are | between its ELP assessment and ELP   |
| designed to measure.          | standards initially and maintained   |
|                               | over time through quality control    |
| More specifically:            | reviews. This process includes:      |
|                               |                                      |
| -   The State's ELP           | -   Necessary                        |
|     assessments and ELP       |     qualifications/expertise of      |
|     standards are aligned     |     individuals (internal, external) |
|     **comprehensively**,      |     involved in the State's          |
|     meaning that the          |     assessment development process;  |
|     assessments reflect the   |                                      |
|     full **range** (breadth,  | -   Necessary                        |
|     depth, complexity) of the |     qualifications/expertise of      |
|     ELP standards, including  |     individuals (internal, external) |
|     the range of language     |     involved in the State's          |
|     skills and knowledge      |     alignment process;               |
|     needed for achievement in |                                      |
|     the content areas.        | -   Criteria for identifying and     |
|                               |     defining the academic language   |
| -   The State's ELP           |     in the State's academic content  |
|     assessments reflect the   |     standards, including the         |
|     same **degree and pattern |     language knowledge and skills    |
|     of emphasis** that are    |     associated with higher-order     |
|     reflected in the State's  |     thinking and understanding       |
|     ELP standards.            |     challenging academic content     |
|                               |     (see related Suggested Elements  |
|                               |     1.5 and 1.9 in Section 1:        |
|                               |     Standards);                      |
|                               |                                      |
|                               | -   Criteria and methodology         |
|                               |     appropriate for evaluating the   |
|                               |     degree of alignment between the  |
|                               |     State's ELP standards and        |
|                               |     assessments, given their         |
|                               |     targeted population, specific    |
|                               |     purpose(s), and "content" (i.e., |
|                               |     English language proficiency     |
|                               |     skills and knowledge and         |
|                               |     academic language); and          |
|                               |                                      |
|                               | -   Independent alignment studies    |
|                               |     (i.e., conducted by an entity    |
|                               |     other than the State or its      |
|                               |     assessment contractor) that      |
|                               |     yield evidence of degree of      |
|                               |     alignment and an associated      |
|                               |     systematic process for           |
|                               |     addressing any gaps or           |
|                               |     weaknesses identified in these   |
|                               |     studies.                         |
|                               |                                      |
|                               | If the State has multiple            |
|                               | assessments within a grade range or  |
|                               | language domain, the State has a     |
|                               | process for ensuring that this       |
|                               | combination of assessments is        |
|                               | aligned to the full scope (breadth,  |
|                               | depth, range of complexity) of the   |
|                               | ELP standards.                       |
|                               |                                      |
|                               | The State's assessment plan,         |
|                               | assessment blueprints, and/or        |
|                               | item/task specifications describe    |
|                               | how all ELP standards are assessed   |
|                               | and how the language domains are     |
|                               | sampled to lead to valid inferences  |
|                               | about student performance on the     |
|                               | standards, individually and in the   |
|                               | aggregate.                           |
|                               |                                      |
|                               | The State has a written plan to      |
|                               | ensure that its assessment system    |
|                               | reflects the full range of language  |
|                               | skills and knowledge (breadth,       |
|                               | depth, complexity) specified in its  |
|                               | ELP standards, as well as the range  |
|                               | of language skills and knowledge     |
|                               | needed for achievement in the        |
|                               | academic content areas. This plan    |
|                               | yields evidence that:                |
|                               |                                      |
|                               | -   The full scope of the standards  |
|                               |     and their differential emphases  |
|                               |     are reflected in the assessment  |
|                               |     plan/blueprints/specifications;  |
|                               |     and                              |
|                               |                                      |
|                               | -   The assessments match the        |
|                               |     plan/blueprints/specifications.  |
|                               |                                      |
|                               | The number of score points on the    |
|                               | State's ELP assessment is consistent |
|                               | with the representation of language  |
|                               | skills and knowledge within and      |
|                               | across each language domain. The     |
|                               | State has detailed assessment        |
|                               | specifications and a description of  |
|                               | the process used to ensure that its  |
|                               | ELP assessment system reflects the   |
|                               | degree and patterns of emphasis that |
|                               | are specified in its ELP standards.  |
|                               |                                      |
|                               | The State is implementing on-going   |
|                               | quality control reviews to ensure    |
|                               | that the system remains fully        |
|                               | aligned over time.                   |
|                               |                                      |
|                               | The State's plan and procedures      |
|                               | should be developed and implemented  |
|                               | with broad input from within and     |
|                               | outside of the educational           |
|                               | community. Such input might come     |
|                               | from committees of specialists with  |
|                               | relevant knowledge of curriculum     |
|                               | (PK-16), instruction, assessment,    |
|                               | measurement, psychometrics,          |
|                               | accountability, policy, content,     |
|                               | alignment/linkage, applied           |
|                               | linguistics, second language         |
|                               | acquisition, language testing, and   |
|                               | the State's ELL population           |
|                               | *[and]{.underline}* also from the    |
|                               | State's Technical Advisory           |
|                               | Committee.                           |
|                               |                                      |
|                               | The composition of groups involved   |
|                               | in the State's plan and procedures   |
|                               | reflects a broad range of relevant   |
|                               | stakeholders and experts, including  |
|                               | individuals knowledgeable and        |
|                               | concerned about the State's ELL      |
|                               | population including ELLs with       |
|                               | disabilities.                        |
+-------------------------------+--------------------------------------+
| 2.8                           | The State has a process for using    |
|                               | the information gained through its   |
| The State has a system for    | series of studies related to         |
| monitoring and improving the  | validity, reliability,               |
| quality of its assessment     | fairness/accessibility, and          |
| system.                       | alignment/linkage to eliminate gaps  |
|                               | and address weaknesses, and the      |
|                               | State has a plan for regular quality |
|                               | review.                              |
|                               |                                      |
|                               | The State has a process and schedule |
|                               | for monitoring the implementation of |
|                               | the State's ELP assessments and      |
|                               | related consequences.                |
|                               |                                      |
|                               | The State relies on multiple sources |
|                               | of data/information for evaluating   |
|                               | the quality and effectiveness of its |
|                               | ELP assessments (e.g., internal and  |
|                               | external monitoring, qualitative     |
|                               | data/analyses, quantitative          |
|                               | data/analyses).                      |
|                               |                                      |
|                               | Documents relevant to these          |
|                               | monitoring and evaluation processes  |
|                               | explicitly address the purpose(s)    |
|                               | and use(s) of the findings vis-à-vis |
|                               | relevant local, State, and federal   |
|                               | policy/requirements for ELLs.        |
|                               |                                      |
|                               | The State's procedures should be     |
|                               | developed and implemented with broad |
|                               | input from within and outside of the |
|                               | educational community. Such input    |
|                               | might come from committees of        |
|                               | specialists with relevant knowledge  |
|                               | of curriculum (PK-16), instruction,  |
|                               | assessment, measurement,             |
|                               | psychometrics, accountability,       |
|                               | policy, content, applied             |
|                               | linguistics, second language         |
|                               | acquisition, language testing, and   |
|                               | the ELL population, and familiarity  |
|                               | with the State's infrastructure,     |
|                               | resources, and goals                 |
|                               | *[and]{.underline}* also from the    |
|                               | State's Technical Advisory           |
|                               | Committee.                           |
|                               |                                      |
|                               | The composition of groups involved   |
|                               | in the State's procedures reflects a |
|                               | broad range of relevant stakeholders |
|                               | and experts, including individuals   |
|                               | knowledgeable and concerned about    |
|                               | the State's ELL population including |
|                               | ELLs with disabilities.              |
+-------------------------------+--------------------------------------+
| 2.9                           | The State's reporting system         |
|                               | includes supporting information to   |
| The State's reporting system  | facilitate accurate and appropriate  |
| facilitates appropriate,      | interpretation of data for those who |
| credible, and defensible      | will receive and use its reports,    |
| interpretation and use of its | such as information about the        |
| ELP assessment data.[^41]     | content and structure of its         |
|                               | assessments, and how the ELP         |
| The State has a process for   | assessments are related to its ELP   |
| producing itemized score      | and academic content standards.      |
| analyses (e.g., by language   |                                      |
| domain, by language skill     | In its reports at the student,       |
| clusters) so that parents,    | classroom, school, and LEA levels,   |
| teachers, and principals can  | the State includes results for each  |
| interpret and address the     | of its ELP standards and also each   |
| specific English language     | of the language domains within these |
| needs of students.            | standards, to the extent that these  |
|                               | subscores are based on enough items  |
|                               | or score points to be meaningful.    |
|                               |                                      |
|                               | The State's reporting system         |
|                               | supports longitudinal reporting for  |
|                               | the purposes of tracking student     |
|                               | improvement/achievement.             |
|                               |                                      |
|                               | The State's reporting system for its |
|                               | ELP assessment is similar to its     |
|                               | other assessment reporting systems,  |
|                               | as appropriate and feasible.         |
|                               |                                      |
|                               | The State has designed reports and   |
|                               | communicated assessment results in   |
|                               | terms of its language proficiency    |
|                               | levels and the meaning of these      |
|                               | levels.                              |
|                               |                                      |
|                               | The State uses a variety of ways to  |
|                               | disseminate information related to   |
|                               | assessment results, such as manuals, |
|                               | bulletins, reports of results, and   |
|                               | Websites to reach the full range of  |
|                               | users of the assessment results.     |
|                               | Effective dissemination of           |
|                               | information may necessitate the      |
|                               | translation of materials into other  |
|                               | languages for users who do not read  |
|                               | English.                             |
|                               |                                      |
|                               | The State has a system for training  |
|                               | educators on the appropriate         |
|                               | interpretation and use of assessment |
|                               | results. State training materials    |
|                               | include: the purpose and content of  |
|                               | the assessments, the reliability of  |
|                               | the assessment scores, sufficient    |
|                               | information to allow use of the      |
|                               | assessment results in making sound   |
|                               | educational decisions or for         |
|                               | conducting scientifically based      |
|                               | research to improve educational      |
|                               | outcomes, and ways in which the      |
|                               | assessment results should *not* be   |
|                               | used.                                |
|                               |                                      |
|                               | The State's reporting system (e.g.,  |
|                               | score reports, interpretive guides)  |
|                               | should be developed and implemented  |
|                               | with broad input from within and     |
|                               | outside of the educational           |
|                               | community. Such input might come     |
|                               | from committees of specialists with  |
|                               | relevant knowledge of curriculum     |
|                               | (PK-16), instruction, assessment,    |
|                               | measurement, psychometrics,          |
|                               | accountability, policy, content,     |
|                               | applied linguistics, second language |
|                               | acquisition, language testing, and   |
|                               | the ELL population, and familiarity  |
|                               | with the State's infrastructure,     |
|                               | resources, and goals                 |
|                               | *[and]{.underline}* also from the    |
|                               | State's Technical Advisory           |
|                               | Committee.                           |
|                               |                                      |
|                               | The composition of groups involved   |
|                               | in the State's development and       |
|                               | implementation of its reporting      |
|                               | system reflects a broad range of     |
|                               | relevant stakeholders and experts,   |
|                               | including individuals knowledgeable  |
|                               | and concerned about the State's ELL  |
|                               | population including ELLs with       |
|                               | disabilities.                        |
+-------------------------------+--------------------------------------+
| 2.10                          | The State reports participation and  |
|                               | performance results for all ELLs and |
| The State reports             | for each required subgroup at the    |
| participation and assessment  | school, LEA, and State levels        |
| results for all ELLs in its   |                                      |
| reports at the school, LEA,   | The State has established and        |
| and State levels, and the     | justified the minimum number and     |
| reports for any group or      | minimum and maximum proportions of   |
| subgroup do not reveal        | students necessary to allow          |
| personally identifiable       | reporting of scores for any group or |
| information about an          | subgroup to ensure that personally   |
| individual student.           | identifiable information about any   |
|                               | individual student is not reported   |
|                               | publicly. The State has clear        |
|                               | descriptions of rules for            |
|                               | determining whether data are         |
|                               | reported for a group or subgroup as  |
|                               | well as a description of how these   |
|                               | rules are implemented and monitored. |
|                               |                                      |
|                               | The State has a system for securely  |
|                               | maintaining student-level assessment |
|                               | data to protect student              |
|                               | confidentiality.                     |
|                               |                                      |
|                               | The State has a clear policy and     |
|                               | detailed procedures for allowing     |
|                               | access to its student-level          |
|                               | assessment data as well as a process |
|                               | for ensuring that electronic student |
|                               | files that are needed by both the    |
|                               | LEA and SEA are available and can be |
|                               | shared across systems in a manner    |
|                               | that is secure both physically and   |
|                               | electronically.                      |
+-------------------------------+--------------------------------------+
| 2.11                          | The State has a plan and schedule    |
|                               | for disseminating as soon as         |
| The State produces            | possible after each assessment       |
| interpretive guidance         | administration the assessment scores |
| following each administration | and information regarding student    |
| of its ELP assessments.       | performance across the language      |
|                               | domains defined in the State's ELP   |
| a.  The reported information  | standards. These reports indicate    |
|     is valid and reliable     | which of the language proficiency    |
|     information regarding     | levels the student's scores          |
|     language proficiency in   | correspond to.                       |
|     relation to the State's   |                                      |
|     ELP standards;            | The State includes interpretive      |
|                               | guidance with each of the reports    |
| b.  The reports provide       | and supports local efforts to        |
|     information for parents,  | translate this guidance as needed to |
|     teachers, and principals  | make it accessible to parents who do |
|     to help them understand   | not read English. The guidance       |
|     and address students'     | includes information about the       |
|     English language needs    | reliability of the scores that are   |
|     and is displayed in a     | reported.                            |
|     format and language that  |                                      |
|     is understandable to      | The State includes interpretive      |
|     parents, teachers, and    | guidance with each of the reports    |
|     principals and are the    | and supports local efforts to        |
|     reports accompanied by    | appropriately use the information to |
|     interpretive guidance for | guide curricular and instructional   |
|     these audiences.          | strategies/programs for the ELLs.    |
|                               |                                      |
| c.  The State reports are     | Documentation clearly depicts the    |
|     delivered to parents,     | scoring and reporting timeline for   |
|     teachers, and principals  | the State ELP assessment.            |
|     as soon as possible after |                                      |
|     the assessment is         | All total scores, language domain    |
|     administered.             | scores, and subdomain scores have    |
|                               | sufficient reliability at the level  |
|                               | each is reported.                    |
|                               |                                      |
|                               | The State's procedures should be     |
|                               | developed and implemented with broad |
|                               | input from within and outside of the |
|                               | educational community. Such input    |
|                               | might come from committees of        |
|                               | specialists with relevant knowledge  |
|                               | of curriculum (PK-16), instruction,  |
|                               | assessment, measurement,             |
|                               | psychometrics, accountability,       |
|                               | policy, content, applied             |
|                               | linguistics, second language         |
|                               | acquisition, language testing, and   |
|                               | the ELL population, and familiarity  |
|                               | with the State's infrastructure,     |
|                               | resources, and goals                 |
|                               | *[and]{.underline}* also from the    |
|                               | State's Technical Advisory           |
|                               | Committee.                           |
|                               |                                      |
|                               | The composition of groups involved   |
|                               | in the State's development and       |
|                               | dissemination of its guidance        |
|                               | reflects a broad range of relevant   |
|                               | stakeholders and experts, including  |
|                               | individuals knowledgeable and        |
|                               | concerned about the State's ELL      |
|                               | population including ELLs with       |
|                               | disabilities.                        |
+-------------------------------+--------------------------------------+
| 2.12                          | The standard-setting approach is     |
|                               | appropriate for the State's          |
| The State has a validated     | reporting and use of the ELP         |
| standard-setting process, and | assessment results. For example, if  |
| its associated language       | separate language domain scores are  |
| proficiency levels,           | reported with proficiency levels     |
| descriptions of language      | (performance categories), the State  |
| competencies for each         | has used an appropriate standard     |
| language proficiency level,   | setting approach.                    |
| and cut scores appropriately  |                                      |
| differentiate among English   | The State's ELP assessments yield    |
| language skills and knowledge | scores that reflect the full range   |
| in each language domain and   | of English language proficiency      |
| each grade range that are     | reflected by the State's ELP         |
| necessary for demonstrating   | standards, and assessment results    |
| progress toward and           | are expressed in terms of the        |
| attainment of English         | language proficiency levels, not     |
| language proficiency, per the | just scale scores or percentiles.    |
| State ELP standards and the   |                                      |
| theory and research on which  | The State's ELP assessments have     |
| the standards are based.      | sufficient items/tasks (score        |
|                               | points) at each language proficiency |
|                               | level to allow students to           |
|                               | demonstrate the full range of        |
|                               | language skills and knowledge        |
|                               | specified in the State's ELP         |
|                               | standards.                           |
|                               |                                      |
|                               | The State's ELP assessments yield    |
|                               | scores that are clearly aligned with |
|                               | the State's ELP language proficiency |
|                               | levels overall and across each       |
|                               | language domain and grade range.     |
|                               |                                      |
|                               | State documentation and interpretive |
|                               | materials related to the ELP         |
|                               | language proficiency levels are      |
|                               | complete, accurate, and clearly      |
|                               | written so that the range of users   |
|                               | and the general public can readily   |
|                               | understand the information. The      |
|                               | documentation may address:           |
|                               |                                      |
|                               | -   Number of language proficiency   |
|                               |     levels and the labels for each   |
|                               |     level;                           |
|                               |                                      |
|                               | -   The relative contribution of     |
|                               |     each language domain and overall |
|                               |     performance (total score) to     |
|                               |     judgments of proficiency;        |
|                               |                                      |
|                               | -   The standard-setting             |
|                               |     methodology, including the       |
|                               |     method(s) used to determine cut  |
|                               |     scores;                          |
|                               |                                      |
|                               | -   Descriptions of the individuals  |
|                               |     involved in the standard-setting |
|                               |     process, including the           |
|                               |     articulation of the proficiency  |
|                               |     level descriptors; and           |
|                               |                                      |
|                               | -   The validation of the State      |
|                               |     language proficiency levels      |
|                               |     against external measures (e.g., |
|                               |     performance on content           |
|                               |     assessments).                    |
|                               |                                      |
|                               | The State's procedures should be     |
|                               | developed and implemented with broad |
|                               | input from within and outside of the |
|                               | educational community. Such input    |
|                               | might come from committees of        |
|                               | specialists with relevant knowledge  |
|                               | of curriculum (PK-16), instruction,  |
|                               | assessment, measurement,             |
|                               | psychometrics, accountability,       |
|                               | policy, content, applied             |
|                               | linguistics, second language         |
|                               | acquisition, language testing, and   |
|                               | the ELL population, and familiarity  |
|                               | with the State's infrastructure,     |
|                               | resources, and goals                 |
|                               | *[and]{.underline}* also from the    |
|                               | State's Technical Advisory           |
|                               | Committee.                           |
|                               |                                      |
|                               | The composition of groups involved   |
|                               | in the State's standard setting      |
|                               | procedures reflects a broad range of |
|                               | relevant stakeholders and experts,   |
|                               | including individuals knowledgeable  |
|                               | and concerned about the State's ELL  |
|                               | population including ELLs with       |
|                               | disabilities.                        |
+-------------------------------+--------------------------------------+

**Recommendations for Application of the Framework**

The purpose of this Framework is to provide a clear statement of the
criteria that States (including consortia[^42]) and their development
partners should address as they develop and implement their ELP
standards and assessments. The ultimate goals for the LEP Partnership in
developing this Framework are: (1) the development of high-quality ELP
standards and ELP assessments and (2) the effective implementation of
these standards and assessments in ways that support students'
attainment of English language proficiency and language skills and
knowledge necessary for achievement in the academic content areas.

Therefore, the purpose of this Framework is to provide States with
information on:

-   **Suggested elements** related to evaluating and improving the
    quality of English language proficiency (ELP) standards and
    assessments;

-   **Possible Considerations** to assist States in the evaluation and
    improvement of the quality of their ELP standards and
    assessments---this includes specific and practical guidance on how
    to approach each suggested element; and

-   **Resources** States can use to inform their evaluation and
    improvement of the quality of their ELP standards and assessments.

This Framework is intended for use by States to **evaluate** and
**improve** their current ELP standards and assessments and to
**identify technical assistance needs** related to their ELP standards
and assessments and the implementation of Title III requirements for
these standards and assessments, per the Elementary and Secondary
Education Act of 1965 (ESEA), as amended by the No Child Left Behind Act
of 2001 (NCLB).

To the extent appropriate and practicable, this draft Framework for
High-Quality English Language Proficiency Standards and Assessments:

-   Uses/adapts language from the Title I *Standards and Assessments
    Peer Review Guidance* (April 2004). This is done to (a) encourage
    efficiencies between State NCLB Title I and Title III implementation
    efforts, (b) build from knowledge and experience States have
    developed to meet standards and assessment requirements for Title I,
    and (c) apply "lessons learned" from standards and assessment
    implementation and guidance for Title I.

-   Addresses challenges that States are facing as they work to meet the
    requirements of Title III, including those issues raised during the
    public comment period of the three public meetings organized by the
    U.S. Department of Education (June 20, July 18, and July 26, 2007).

-   Is based on research in relevant areas including standards,
    assessment, second language acquisition, academic language,
    alignment and linkage, and the English language learner population.

-   Includes input from experts with knowledge and experience in
    relevant areas including standards, curriculum, assessment,
    professional development, psychometrics, second language
    acquisition, academic language, alignment and linkage, and the
    English language learner population.

-   Seeks to balance specific and practical guidance with flexibility
    needed by States, given their varying population characteristics,
    resources, experiences, and stages of implementation.

-   Provides States with structure/detail whether they are in earlier
    stages of implementation or already have full systems in place and
    need theory- and research-based criteria/parameters for evaluation
    and improvement.

This Framework presents information for a model ELP standards and
assessment program that is based on research-based theory (including
second language acquisition, English language development/acquisition,
sociocultural, measurement), applied research, and best/promising
practices. It is intended to guide the establishment of valid and
accessible State ELP standards and assessments that support effective
instruction. The Department believes that high-quality ELP standards and
assessments are more likely when the Framework is applied using a
transparent evaluation and support process that involves experts with
knowledge and experience in relevant areas of the field, including
English language acquisition, academic language, standards, curriculum,
assessment, psychometrics, alignment and linkage, and the English
language learner population, in an on-going conversation with States.
This Framework does not address the validity and reliability of State
accountability systems (e.g., for State accountability workbooks) or how
the results/uses of ELP assessments ought to contribute to State
accountability; rather this Framework addresses the validity and
reliability of ELP standards and assessments for placement and
proficiency determinations.

The Department invites comments on this first draft document from all
States. Reviewers are invited to comment on/make recommendations related
to:

-   The appropriateness and clarity of each suggested element;

-   The comprehensiveness, coherence, and relationships among the
    suggested elements as a set; and

-   The clarity and utility of the possible considerations related to
    each suggested element.

Additionally, reviewers are invited to:

-   Nominate resources for inclusion in "Recommended Resources" and

```{=html}
<!-- -->
```
-   Recommend terminology and definitions for the Glossary.

It is anticipated that the refinement of this Framework will be further
enhanced by a self-review and feedback from States (see the Introduction
of the draft Framework \[pp. 4-5\] for a description of the pilot
review). The Department will work with States or groups of States
willing to engage in a self-review of the Framework document by
conducting a self-review of State ELP standards and assessments.
Information gathered from the pilot will focus on informing and
improving another draft of this Framework document as well as
identifying States' technical assistance needs. For States involved in
the review, no report will be made to the Department by expert peers and
technical assistance providers about State ELP standards or assessment
systems.

**Instructions for submitting comments/recommendations**:

Email comments and recommendations to either the LEP Partnership or to
the Assessment and Accountability Comprehensive Center.

# LEP Partnership

> **LEP.Partnership@ed.gov**
>
> []{.mark}

# Assessment and Accountability Comprehensive Center

> **lepframework@aacompcenter.org**
>
> or
>
> Edynn Sato, Ph.D., Director of Special Populations at
> **esato@wested.org**

Comments and recommendations should be submitted no later than
**December 15, 2007**.

**Resources**[^43]

The following are resources relevant to English language proficiency,
the English language learner population, and standards and assessments.
This list of resources is representative, not inclusive.

Abedi, J. (2004). The No Child Left Behind Act and English language
learners: Assessment and accountability issues. *Educational
Researcher*, *33*(1), 4--14.

Abedi, J., & Dietel, R. (2004). Challenges in the No Child Left Behind
Act for English language learners. *Phi Delta Kappan, 85*, 782-785.

Abedi, J., & Nambiar, V. V. (Eds.). (in press). *English language
proficiency assessment in the United States: Current status and future
practice*. Davis: University of California, Davis, School of Education,
and University of California, Los Angeles, National Center for Research
on Evaluation, Standards, and Student Testing.

Albus, D., Klein, J. A., Liu, K., & Thurlow, M. (2004). *Connecting
English language proficiency, statewide assessments, and classroom
proficiency* (LEP Projects Report No. 5). Minneapolis: University of
Minnesota, National Center on Educational Outcomes. Retrieved September
27, 2007, from http://education.umn.edu/NCEO/OnlinePubs/LEP5.html

American Educational Research Association, American Psychological
Association, & National Council on Measurement in Education. (1999).
*Standards for educational and psychological testing*. Washington, DC:
AERA.

Anderson, M. E., Minnema, J. E., Thurlow, M. L., & Hall-Lande, J.
(2005). *Confronting the unique challenges of including English language
learners with disabilities in statewide assessments*. Minneapolis:
University of Minnesota, National Center on Educational Outcomes.
Retrieved on September 18, 2007, from
http://education.umn.edu/NCEO/OnlinePubs/ELLsDisReport9.html

Apthorp, H. S., D\'Amato, E. D., & Richardson, A. (2002). *Effective
standards-based practices for Native American students: A review of
research literature.* Aurora, CO: Mid-Continent Research for Education
and Learning.

Bachman, L. F., & Palmer, A. S. (1982). The construct validation of some
components of communicative proficiency. *TESOL Quarterly, 16*, 449-465.

Bailey, A. L. (2007). *The language demands of school: Putting academic
English to the test*. New Haven, CT: Yale University Press.

Bailey, A. L., Butler, F. A., & Sato, E. (2007). Standards-to-standards
linkage under Title III: Exploring common language demands in ELD and
science standards. *Applied Measurement in Education, 20*(1), 53-78.

Baker, E. L. (2005). *Aligning curriculum, standards, and assessments:
Fulfilling the promise of school reform* (CSE Report No. 645). Los
Angeles: University of California, National Center for Research on
Evaluation, Standards, and Student Testing.

Berk, R. A. (1986). A consumer's guide to setting performance standards
on criterion-referenced tests. *Review of Educational Research, 56*(1),
137--172.

Butler, F. A., & Stevens, R. (1998). *Initial steps in the validation of
the second language proficiency descriptors for public high schools,
colleges, and universities in California: Writing* (CSE Technical Report
No. 497). Los Angeles: University of California, National Center for
Research on Evaluation, Standards, and Student Testing.

Carr, J., & Lagunoff, R. (2006). *The map of standards for English
learners, set grades K-5 and 6-12*. San Francisco: WestEd.

Center for Research on Education, Diversity & Excellence. (2003). *A
national study of school effectiveness for language minority students'
long-term academic achievement* (Research Brief No. 10). Berkeley, CA:
Author. Available:
http://crede.berkeley.edu/research/llaa/1.1_final.html

Cizek, G. (2001). *Setting performance standards: Concepts, methods, and
perspectives.* Mahwah, NJ: Lawrence Erlbaum.

Cook, H. G. (2006). Aligning English language proficiency tests to
English language learning standards. In Council of Chief State School
Officers, *Aligning assessment to guide the learning of all students:
Six reports* (pp. 131-154). Washington, DC: Council of Chief State
School Officers.

Council of Chief State School Officers. (2002). *Alignment study in
language arts, mathematics, science, and social studies of state
standards and assessments in four states*. Washington, DC: Author.

Council of Chief State School Officers. (2006). *Creating aligned
standards and assessment systems*. Washington, DC: Author.

Cummins, J. (1980). The construct of proficiency in bilingual education.
In J. E. Alatis (Ed.), *Georgetown University Round Table on Languages
and Linguistics: Current issues in bilingual education, 1980* (pp.
81-103). Washington, DC: Georgetown University.

Cummins, J. (2005). Language proficiency, bilingualism, and academic
achievement. In P. A. Richard-Amato and M. A. Snow (Eds.), *Academic
success for English language learners* (pp. 76-86). White Plains, NY:
Longman.

Fillmore, L. W., & Snow, C. E. (2000). *What teachers need to know about
language*. Washington, DC: Center for Applied Linguistics.

Goh, D. S. (2004). *Assessment accommodations for diverse learners*.
Boston: Allyn & Bacon.

Gottlieb, M., Carnuccio, L. M., Ernst-Slavit, G., & Katz, A. (2006).
*PreK-12 English language proficiency standards*. Alexandria, VA:
Teachers of English to Speakers of Other Languages.

Haertel, E., & Herman, J. (2005). *A historical perspective on validity
arguments for accountability testing* (CSE Report No. 654). Los Angeles:
University of California, National Center for Research on Evaluation,
Standards, and Student Testing.

Halliday, M. A. K. (1994). *An introduction to functional grammar* (2nd
ed.). London: Edward Arnold.

Heritage, M., Silva, N., & Pierce, M. (2007). Academic English: A view
from the classroom. In A.L.Bailey (Ed.), *The language demands of
school: Putting academic English to the test* (pp. 171-210). New Haven,
CT: Yale University Press.

Jaeger, R. M. (1989). Certification of student competence. In R. L. Linn
(Ed.), *Educational measurement* (3rd ed., pp. 485--514). New York:
Macmillan.

Johnstone, C., Altman, J., & Thurlow, M. (2006). *Universal design
online manual*. Minneapolis: University of Minnesota, National Center on
Educational Outcomes.

Knapp, M., Copland, M. A., Plecki, M., & Portin, B. S. (2006). *Leading,
learning, and leadership support*. Seattle: University of Washington,
Center for the Study of Teaching and Policy.

Kolen, M. J., & Brennan, R. L. (2004). *Test equating, scaling, and
linking: Methods and practices* (2nd ed.). New York: Springer-Verlag.

Kopriva, R. (2000). *Ensuring accuracy in testing for English language
learners*. Washington, DC: Council of Chief State School Officers.

Kopriva, R.J. (2007). Improving testing for English language learners: A
comprehensive approach to designing, building, implementing and
interpreting better academic assessments, Erlbaum Routledge Publishers;
NY, NY.

Koretz, D. (2005). *Alignment, high stakes, and the inflation of test
scores* (CSE Report No. 655). Los Angeles: University of California,
National Center for Research on Evaluation, Standards, and Student
Testing.

LaMarca, P. M., Redfield, D., Winter, P. C., Bailey, A., & Hansche
Despriet, L. (2000). *State standards and state assessment systems: A
guide to alignment*. Washington, D.C.: Council of Chief State School
Officers.

Lazarín, M. (2006). *Improving assessment and accountability for English
language learners in the No Child Left Behind Act* (Issue Brief No. 16).
Washington, DC: The National Council of La Raza.

Lee, O., & Fradd, S. H. (1998). Science for all, including students from
non-English-language backgrounds. *Educational Researcher*, *27*(4),
12-21.

Lewis, D. M., Mitzel, H. C., & Green, D. R. (1996, June) *Standard
setting: A bookmark approach.* Paper presented at the Council of Chief
State School Officers Large-Scale Assessment Conference, Colorado
Springs, CO.

Linn, R. L. (2003). Performance standards: Utility for different uses of
assessments. *Education Policy Analysis Archives, 11*(31). Retrieved
March 8, 2006, from http://epaa.asu.edu/epaa/v11n31

Livingston, S. A., & Zeikey, M. J. (1982). *Passing scores: A manual for
setting standards of performance on educational and occupational tests*.
Princeton, NJ: Educational Testing Service.

Malagón, M. H., Rosenberg, M. B., & Winter, P. C. (2006). Developing
aligned performance level descriptors for the English language
development assessment K-2 inventories. In Council of Chief State School
Officers, *Aligning assessment to guide the learning of all students:
Six reports* (pp. 155-173). Washington, DC: Council of Chief State
School Officers.

Messick, S. (1989). Validity. In R. L. Linn (Ed.), *Educational
measurement* (3rd ed., pp. 13-103). New York: Macmillan.

Mitzel, H. C. (2005). *Consistency for state achievement standards under
NCLB*. Paper presented to CAS SCASS Study Group. Washington, DC: Council
of Chief State School Officers.

National Research Council. (2002). In J. Koenig (Ed.), *Reporting test
results for students with disabilities and English-language learners*.
Washington, DC: National Academies.

National Task Force on Early Childhood Education for Hispanics. (2007).
*Expanding and improving early education for Hispanics* (Main Report).
Tempe, AZ: Arizona State University.

Olson, B., Mead, R., & Payne, D. (2002). *A report of a standard setting
method for alternate assessments for students with significant
disabilities* (NCEO Synthesis Report No. 47). Minneapolis: University of
Minnesota, National Center on Educational Outcomes.

Rabinowitz, S. (2004). *Design considerations for building out NCLB
state assessment systems*. San Francisco: WestEd.

Rabinowitz, S., Roeber, E., Schroeder, C., & Sheinker, J. (with CAS
SCASS Study Group: Transitions in Assessment from IASA to NCLB). (2006).
*Creating aligned standards and assessment systems*. Washington, DC:
Council of Chief State School Officers.

Rabinowitz, S. N., & Sato, E. (2005). *The technical adequacy of
assessments for alternate student populations: Guidelines for consumers
and developers.* San Francisco: WestEd.

Raymond, M. R. & Reid, J. B. (2001). Who made thee judge? Selecting and
training participants for standard setting. In G. Cizek (Ed.), *Setting
performance standards: Concepts, methods, and perspectives* (pp.
119--157). Mahwah, NJ: Lawrence Erlbaum.

Rivera, C., & Collum, E. (Eds.). (2006). *State assessment policy and
practice for English language learners: A national perspective.* Mahwah,
NJ: Lawrence Erlbaum.

Ruiz-de-Velasco, J., & Fix, M. (2000). *Overlooked & underserved:
Immigrant students in U.S. secondary schools*. Washington, DC: The Urban
Institute.

Sandova, J. H., Frisby, C. L., Geisinger, K. F., Ramos-Grenier, J., &
Scheuneman, J. D. (Eds.). (1998). *Test interpretation and diversity:
Achieving equity in assessment*. Washington, DC: American Psychological
Association.

Sato, E. (2007). *Rethinking alignment for English language learner
assessments and standards: Issues and implications for extending current
models*. Paper commissioned by the Council of Chief State School
Officers.

Sato, E., & Rabinowitz, S. (forthcoming). *Knowing what English language
learners know: Implications for assessment, practice, and policy*
\[working title\]. San Francisco: WestEd.

Scarcella, R. (2003). *Academic English: A conceptual framework*. The
University of California Linguistic Minority Research Institute
Technical Report 2003-1. Retrieved July 3, 2007, from
http://www.lmri.ucsb.edu/publications/03_scarcella.pdf

Schleppegrell, M. J. (2001). Linguistic features of the language of
schooling. *Linguistics and Education, 12*, 431-459.

Schleppegrell, M. J. (2006). The challenges of academic language in
school subjects. Available in English:
http://www.soe.umich.edu/events/als/downloads/schleppegrellp.html From
I. Lindberg & K. Sandwall (Eds.), *Språket och kunskapen: att lära på
sitt andraspråk i skola och högskola* (pp. 47-69). Göteborg, Sweden:
Göteborgs universitet institutet för svenska som andraspråk.

Short, D. J., & Fitzsimmons, S. (2007). *Double the work: Challenges and
solutions to acquiring language and academic literacy for adolescent
English language learners* (A Report to Carnegie Corporation of New
York). Washington, DC: Alliance for Excellent Education.

Snow, C. E., Cancini, H., Gonzalez, P., & Shriberg, E. (1989). Giving
formal definitions: An oral language correlate of school literacy. In D.
Bloome (Ed.), *Classrooms and literacy* (pp. 233-249). Norwood, NJ:
Ablex.

Solano-Flores, G., & Trumbull, E. (2003). Examining language in context:
The need for new research and practice paradigms in the testing of
English-language learners. *Educational Researcher, 32*(2), 3-13.

The National Council of Teachers of English (NCTE) ELL Task Force.
(2006). *NCTE position paper on the role of English teachers in
educating English language learners (ELLs)*. Retrieved September 20,
2007, from http://www.ncte.org/edpolicy/ell/about/124545.htm

Thurlow, M., Minnema, J., & Treat, J. (2004). *A review of 50 states'
online large-scale assessment policies: Are English language learners
with disabilities considered?* (ELLs with Disabilities Report No. 5).
Minneapolis: University of Minnesota, National Center on Educational
Outcomes. Available:
http://education.umn.edu/NCEO/OnlinePubs/ELLsDisReport5.html

Webb, N. (1999). *Alignment of science and mathematics standards and
assessments in four states.* Madison: University of Wisconsin--Madison,
National Institute for Science Education.

Webb, N. L. (2007). Issues related to judging the alignment of
curriculum standards and assessments. *Applied Measurement in
Education*, *20*(1), 7-25.

Webb, N. L., Alt, M., Ely, R., Cormier, M., & Vesperman, B. (2006). The
Web alignment tool: Development, refinement, and dissemination. In
Council of Chief State School Officers, *Aligning assessment to guide
the learning of all students: Six reports* (pp. 1-30). Washington, DC:
Council of Chief State School Officers.

Wiley, T. G. (2005). *Literacy and language diversity in the United
States*. Washington, DC: Center for Applied Linguistics.

Winter, P. C., Kopriva, R. J., Chen, C. S., & Emick, J. E. (2006).
Exploring individual and item factors that affect assessment validity
for diverse learners: Results from a large-scale cognitive lab.
*Learning and Individual Differences, 16*(4) 267-276.

Wise, L., & Alt, M. (2006). Assessing vertical alignment. In Council of
Chief State School Officers, *Aligning assessment to guide the learning
of all students: Six reports* (pp. 57-74). Washington, DC: Council of
Chief State School Officers.

Wise, L. L., Zhang, L., Winter, P., Taylor, L., & Becker, D. E. (2006).
Vertical alignment of grade-level expectations for student achievement:
Report of a pilot study. In Council of Chief State School Officers,
*Aligning assessment to guide the learning of all students: Six reports*
(pp. 75-130). Washington, DC: Council of Chief State School Officers.

References

Abedi, J. (2004). The No Child Left Behind Act and English language
learners: Assessment and accountability issues. *Educational
Researcher*, *33*(1), 4--14.

Abedi, J., & Dietel, R. (2004). Challenges in the No Child Left Behind
Act for English language learners. *Phi Delta Kappan, 85*, 782-785.

Abedi, J., & Lord, C. (2001). The language factor in mathematics tests.
*Applied Measurement in Education, 14*(3), 219-234.

Aguirre-Munoz, Z., Parks, J.E., Benner, A., Amabisca, A. & Boscardin,
C.K. (2006). *Consequences and Validity of Performance Assessment for
English Language Learners: Conceptualizing & Developing Teachers'
Expertise in Academic Language*. CRESST/UCLA Center for the Study of
Evaluation. Available at
http://www.cse.ucla.edu/products/reports/R700.pdf.

American Educational Research Association, American Psychological
Association, & National Council on Measurement in Education. (1999).
*Standards for educational and psychological testing*. Washington, DC:
AERA.

Bailey, A. L. (2007). *The language demands of school: Putting academic
English to the test*. New Haven, CT: Yale University Press.

Bailey, A., Butler, F., & Sato, E. (2007). Standards-to-standards
linkage under Title III: Exploring common language demands in ELD and
science standards. *Applied Measurement in Education, 20 (1), 53-78.*

Butler, F. A., Bailey A. L., Stevens, R., Huang, B., & Lord, C. (2004).
*Academic English in fifth-grade mathematics, science, and social
studies textbooks.* (Final Deliverable to IES, Contract No. R305B960002;
currently available as CSE Rep. No. 642). Los Angeles: University of
California, National Center for Research on Evaluation, Standards, and
Student Testing (CRESST).

Butler, F.A. & Stevens, R. (2001). Standardized assessment of the
content knowledge of English Language Learners K-12: Current trends and
old dilemmas. *Language Testing 2001*, 18(4), 409-427.

Cummins, J. (1980). The construct of proficiency in bilingual education.
In J.E. Alatis (Ed.), Georgetown University Round Table on Languages and
Linguistics: Current issues in bilingual education, 1980 (pp. 81-103).
Washington, DC: Georgetown University.

Cummins, J. (2005). Language proficiency, bilingualism, and academic
achievement. In P.A. Richard-Amato and M.A. Snow (Eds.), Academic
success for English language learners (pp. 76-86). White Plains, NY:
Longman.

Del Vecchio, A., & Guerrero, M. (1995). *Handbook of English language
proficiency tests*. Albuquerque: New Mexico Highlands University,
Evaluation Assistance Center, Western Region. Retrieved September 21,
2007 from http://www.ncela.gwu.edu/pubs/eacwest/elptests.htm

Green, D. R. (1998). Consequential aspects of the validity of
achievement tests: A publisher's point of view. *Educational
Measurement: Issues and Practices, 17*(2), 16--19.

Halliday, M.A.K., (1994). An introduction to functional grammar (2^nd^
ed.). London: Edward Arnold.

Holmes, D. & Duron, S. (2000). *LEP students and high stakes
assessment.* Washington, DC: National Clearinghouse for Bilingual
Education, US Department of Education.

Kopriva, R. (2000). *Ensuring accuracy in testing for English language
learners*. Washington, DC: Council of Chief State School Officers.

Liu, K., Anderson, M. E., Swierzbin, B., & Thurlow, M. (1999).
*Bilingual accommodations for limited English proficient students on
statewide reading tests: Phase 1* (Minnesota Report 20). Minneapolis:
University of Minnesota, National Center on Educational Outcomes.

Menken, K. (2000). *What are the critical issues in wide-scale
assessment of English language learners?* (Issue Brief 6). Washington,
DC: National Clearinghouse for Bilingual Education, Center for the Study
of Language and Education.

Messick, S. (1989). Validity. In R. L. Linn (Ed.), *Educational
measurement* (3rd ed., pp. 13-103). New York: Macmillan.

National Clearinghouse for English Language Acquisition. (2006). *How
many school-aged English language learners (ELLs) are there in the U.S?*
(NCELA FAQ No. 1). Washington, DC: Author. Retrieved October 22, 2007
from the World Wide Web at
http://www.ncela.gwu.edu/expert/faq/01leps.html

Rabinowitz, S., & Sato, E. (2005). *A technical review of high-stakes
assessments for English language learners.* San Francisco, CA: WestEd.

Rivera, C. & Stansfield, C.W. (2001). *The effects of linguistic
simplification of science test items on performance of Limited English
Proficient and monolingual English-speaking students.* Paper presented
at the Annual Meeting of the American Educational Research Association,
Seattle, WA.

Sato, E. (2007). *Rethinking alignment for English language learner
assessments and standards: Issues and implications for extending current
models*. Paper commissioned by the Council of Chief State School
Officers.

Sato, E., Lagunoff, R., Worth, P., Bailey, A. L., & Butler, F. A.
(2005). *ELD Standards Linkage and Test Alignment Under Title III: A
Pilot Study of the CELDT and the California ELD and Content Standards*.
(Final Report to the California Department of Education). San Francisco:
WestEd.

Sato, E., & Rabinowitz, S. (forthcoming-a). *Knowing what English
language learners know: Implications for assessment, practice, and
policy* \[working title\]. San Francisco: WestEd.

Sato, E. & Rabinowitz, S. (forthcoming-b). *Linguistic modification as
an accommodation for English language learners: Intended versus actual
impact on linguistic and cognitive processing* \[working title\]. San
Francisco: WestEd.

Sato, E., Rabinowitz, S., & Gallagher, C. (forthcoming). Access and
Special Student Populations---the Similarities/Differences in the Needs
of English Language Learners and Students with Disabilities:
Implications for Standards, Assessment, and Instruction \[working
title\]. San Francisco: WestEd.

Scarcella, R., & Zimmerman, C. (1998). Academic words and gender: ESL
student performance on a test of academic lexicon. *Studies in Second
Language Acquisition*, *20*, 27--49.

Schleppegrell, M.J. (2001). Linguistic features of the language of
schooling. Linguistics and Education, 12 (4), 431-459.

Solano-Flores, G., & Li, M. (2006). The use of generalizability (g)
theory in the testing of linguistic minorities. *Educational
Measurement, Issues and Practice, 25*, 13-22.

Solano-Flores, G. & Trumbull, E. (2003). Examining language in context:
The need for new research and practice paradigms in the testing of
English-language learners. *Educational Researcher, 32*(2), 3-13.

Walqui, A. (2000). *Access and engagement: Program design and
instructional approaches for immigrant students in secondary school.*
McHenry, IL: Delta Publishing.

Webb, N. L., Horton, M., & O'Neal, S. (2002, April). *An analysis of the
alignment between language arts standards and assessments for four
states*. Paper presented at the annual meeting of the American
Educational Research Association, New Orleans, LA.

WestEd. (2004). Alignment Study: Louisiana Content Standards and the
Louisiana Alternate Assessment (LAA). San Francisco, CA: WestEd.

**Acknowledgements**

Dr. Edynn Sato and Dr. Stanley Rabinowitz of the Assessment and
Accountability Comprehensive Center (AACC) would like to thank the
following reviewers for their input on early versions of the *Framework
for High-Quality English Language Proficiency Standards and
Assessments*.

Jamal Abedi, Ph.D.\
Gary Cook, Ph.D.\
Richard Duran, Ph.D.\
Ellen Forte, Ph.D.\
Carole Gallagher, Ph.D.\
Margot Gottlieb, Ph.D.\
Kevin Huang, Ph.D.

Dorry Kenyon, Ph.D.\
Rachel Lagunoff, Ph.D.

Pamela McCabe\
Joseph McCrary D.P.A\
Theodor Rebarber

Charlene Rivera, Ph.D.\
Marla Perez-Selles\
Ursula Sexton\
Ann-Marie Wiese, Ph.D.

The authors also would like to thank Ms. Holly McKeag and Dr. Roger
Frantz for their assistance with the preparation of this document.

The information presented in this Framework may not necessarily reflect
the views of the individuals listed. The AACC and its prime host agency,
WestEd, are responsible for the contents of this document.

[^1]: Limited English proficient (LEP) students are (a) 3 to 21 years of
    age, (b) enrolled or preparing to enroll in elementary or secondary
    school, (c) either not born in the United States or have a native
    language other than English, and (d) owing to difficulty in
    speaking, reading, writing, or understanding English, not able to
    meet the State's proficient level of achievement to successfully
    achieve in English-only classrooms or not able to participate fully
    in society (Title IX, Section 9101).

[^2]: The language of the Elementary and Secondary Education Act (ESEA)
    refers to the targeted student population for this framework as
    "limited English proficient" (LEP). We recognize that many
    researchers and practitioners prefer the term English Language
    Learner (ELL). Consistent with this more general, common usage, the
    remainder of this framework will use the latter term.

[^3]: According to Title III, Section 3113, the State ELP standards are
    to be aligned with the achievement of challenging State academic
    content and student academic achievement standards. See definitions
    of *alignment* and *linkage* in footnote on pp. 10-11 of this
    document.

[^4]: Each State in a consortium should make sure that the ELP standards
    are useful and meaningful to the State vis-à-vis its ELL population,
    and the technical evidence associated with the ELP assessment is
    applicable to both the State's particular ELL population (key
    subgroups, as defined by the State, vis-à-vis the consortium's
    subgroup definitions) and context (e.g., priorities, uses of
    results) of the State.

[^5]: The distinction being made between ELP standards and ELA/Reading
    standards is that ELP skills and knowledge are necessary but not
    sufficient to demonstrate proficiency in ELA/Reading, as well as in
    other academic content areas (e.g., mathematics, science, social
    studies). However, there is obviously more overlap between ELP and
    ELA/Reading standards than between ELP and other academic content
    standards because of their focus on English language.

[^6]: While there may not be just one accepted definition of academic
    language, there are numerous resources available on the issue (e.g.,
    Aguirre-Munoz, Parks, Benner, Amabisca, & Boscardin, 2006; Bailey,
    2007; Bailey, Butler, & Sato, 2007; Butler, Bailey, Stevens, Huang,
    & Lord, 2004; Cummins, 1980, 2005; Halliday, 1994; Scarcella &
    Zimmerman, 1998; Schleppegrell, 2001) that may be considered when
    developing ELP standards and assessments.

[^7]: While there may not be just one accepted theory of English
    language development/acquisition (as there is not one accepted
    theory for teaching mathematics, for example) much has been
    validated by experts across several fields, and therefore needs to
    be considered when developing statewide systems of standards and
    assessments.

[^8]: The point is not [that]{.underline} ELLs are
    diverse/heterogeneous; rather it is [how]{.underline} they are
    diverse/heterogeneous. For example, ELLs differ in their level of
    literacy and amount of formal education in their native language.
    Some ELLs are newly arrived; however, most are second or third
    generation in the U.S. ELLs speak and write languages that may have
    sound systems, grammatical structures, and writing systems that are
    different from those of English to varying degrees, which can affect
    the rate and ease of English language acquisition and English
    literacy learning. The nature of the diversity/heterogeneity of this
    group may affect the implementation of standards (e.g., curriculum,
    instruction) and assessments.

[^9]: According to Title III, Section 3113, the State ELP standards are
    to be aligned with the achievement of challenging State academic
    content and student academic achievement standards. Therefore, in
    order to appropriately support ELL achievement of challenging
    academic content and academic achievement standards, this Suggested
    Element presents considerations for establishing the necessary
    *linkage* between State ELP standards and the language skills and
    knowledge students need to achieve in the academic content
    areas---that is, the linkage between the State ELP standards and the
    State academic content standards. The result of the process of this
    linkage should be an alignment of a State's English language
    proficiency standards to the achievement by ELL students, and former
    ELL students, of challenging academic content standards.
    **Alignment** refers to the degree to which the content (e.g.,
    skills, concepts) in two sets of standards or in an assessment and
    set of standards match in terms of breadth, depth, and range of
    complexity. Alignment relationships tend to be direct relationships
    (matches between skills, content) and are typically observed between
    standards and assessments for a single student population (e.g.,
    general education, English language learners, students with
    disabilities). **Linkage** refers to relationships that tend to be
    developmental, foundational, or proximal and is typically observed
    between standards and/or assessments developed for different student
    populations (e.g., general education, English language learners,
    students with disabilities) (WestEd, 2004; Sato, Lagunoff, Worth,
    Bailey, & Butler, 2005; Bailey, Butler, & Sato, 2007).

[^10]: Suggested elements are listed separately in this Framework to
    facilitate State processes for addressing the elements and to
    reflect their individual contribution to a valid and reliable ELP
    standards system. Although these elements are listed separately,
    States should keep in mind the inter-relationships among these
    suggested elements for establishing a comprehensive and cohesive
    plan for addressing the needs of their ELLs.

[^11]: Some considerations are repeated across suggested elements to
    ensure that States address the considerations as appropriate.

[^12]: Language skills consistent with higher-order thinking skills and
    understanding include summarization, comparison-contrast, and
    persuasion.

[^13]: ELP language skills and knowledge are necessary but not
    sufficient to demonstrate proficiency in the academic content areas
    (e.g., ELA, mathematics, science, social studies). However, there is
    obviously more overlap (convergence) between the language and
    content of ELP and ELA/Reading than between ELP and other academic
    content standards because of their focus on English language. The
    State should clearly explain its intended relationship between its
    ELP standards and its content area standards.

[^14]: Broadly defined, academic language is the language (e.g.,
    lexical, grammatical, and discourse features specific to and
    reflective of a particular context or content area) students need to
    meaningfully engage with academic content and achieve proficiency in
    English in the academic setting (Sato, 2007). While there may not be
    just one accepted definition of academic language, there are a good
    number of resources available that address the issue of academic
    language (e.g., Aguirre-Munoz, Parks, Benner, Amabisca, & Boscardin,
    2006; Bailey, 2007; Bailey, Butler, & Sato, 2007; Butler, Bailey,
    Stevens, Huang, & Lord, 2004; Cummins, 1980, 2005; Halliday, 1994;
    Scarcella & Zimmerman, 1998; Schleppegrell, 2001) which ought to be
    considered when developing ELP standards and assessments.

[^15]: The State ELP standards should reflect consideration of the
    heterogeneity of the State's ELL population. Consideration of the
    heterogeneity of the State's ELL population *should not be
    interpreted to* suggest different standards or achievement
    expectations for different ELL subgroups; only one set of ELP
    standards should be developed for all of the State's ELLs. However,
    given that standards are major levers for improving student
    achievement by articulating goals and focusing instruction, the
    standards should be accessible to all students. That is, the State's
    ELP standards should be conceptualized and designed so that they can
    guide and support instruction that can be differentiated, for
    example, to meet the varying needs of the State's ELLs vis-à-vis key
    characteristics of their backgrounds (e.g., language, culture) that
    affect their learning.

[^16]: This suggested element is not recommending that States develop
    language acquisition profiles for each major linguistic or
    sociocultural group. Rather, this suggested element encourages
    States to understand and clearly articulate who their ELLs
    are---without a clear understanding of *who* needs to develop
    English language proficiency, it is difficult to know *how* to
    effectively facilitate such development (i.e., via standards that
    guide the instruction and assessment of these students that
    appropriately consider their access needs).

[^17]: The degree to which language skills and knowledge across grade
    ranges and proficiency levels are repeated and overlap (spiraled)
    should be purposeful and grounded in theory and/or research/data.

[^18]: The State ELP standards should reflect consideration of the
    heterogeneity of the State's ELL population. Consideration of the
    heterogeneity of the State's ELL population *should not* be
    interpreted to suggest different standards or achievement
    expectations for different ELL subgroups; only one set of ELP
    standards should be developed for all of the State's ELLs. However,
    given that standards are major levers for improving student
    achievement by articulating goals and focusing instruction, the
    standards should be accessible to all students. That is, the State's
    ELP standards should be conceptualized and designed so that they can
    guide and support instruction that can be differentiated, for
    example, to meet the varying needs of the State's ELLs vis-à-vis key
    characteristics of their backgrounds (e.g., language, culture) that
    affect their learning.

[^19]: Local assessment refers to a teacher, school, or
    district-developed assessment separate from the State-mandated
    assessment. Such assessments may be designed to provide information
    on specific student English language skills and knowledge of
    interest to local practitioners, thereby augmenting information
    provided by the State-mandated assessment.

[^20]: *Articulation* is used instead of *alignment* because of the
    nature of the English language proficiency domain, which has skills
    and knowledge that fall along a continuum and may develop at
    different rates across the language domains (listening, speaking,
    reading, writing).

[^21]: According to Title III, Section 3113, the State ELP standards are
    to be aligned with the achievement of challenging State academic
    content and student academic achievement standards. Therefore, in
    order to appropriately support ELL student achievement of
    challenging academic content and academic achievement standards,
    this Suggested Element presents considerations for establishing
    necessary *linkage* between State ELP standards and the language
    skills and knowledge students need to achieve in the academic
    content areas---that is, linkage between the State ELP standards and
    the State academic content standards. The result of the process of
    this linkage should be an alignment of a State's English language
    proficiency standards to the achievement by ELL students, and former
    ELL students, of challenging academic content standards.
    **Alignment** refers to the degree to which the content (e.g.,
    skills, concepts) in two sets of standards or in an assessment and
    set of standards match in terms of breadth, depth, and range of
    complexity. Alignment relationships tend to be direct relationships
    (matches between skills, content) and are typically observed between
    standards and assessments for a single student population (e.g.,
    general education, English language learners, students with
    disabilities). **Linkage** refers to relationships that tend to be
    developmental, foundational, or proximal and is typically observed
    between standards and/or assessments developed for different student
    populations (e.g., general education, English language learners,
    students with disabilities) (WestEd, 2004; Sato, Lagunoff, Worth,
    Bailey, & Butler, 2005; Bailey, Butler, & Sato, 2007).

[^22]: The distinction being made here between *language* and *content*
    is that ELP language skills and knowledge are necessary but not
    sufficient to demonstrate proficiency in the academic content areas
    (e.g., ELA, mathematics, science, social studies). However, there is
    obviously more overlap (convergence) between the language and
    content of ELP and ELA/Reading than between ELP and other academic
    content standards because of the focus on English language in both
    ELP and ELA.

[^23]: The heterogeneity of the State's ELL population should be
    considered when developing guidance and training to LEAs; that is,
    the relevant materials should explain key characteristics of the ELL
    population that may affect instruction (e.g., linguistic background,
    cultural background, level of proficiency in native language) and
    provide guidance on how instruction can effectively be
    differentiated, for example, to meet the varying needs of the
    State's ELLs vis-à-vis the characteristics (e.g., linguistic,
    cultural) that affect their learning. Consideration of the
    heterogeneity of the State's ELL population should *not be
    interpreted to* suggest different standards or achievement
    expectations for different ELL subgroups; only one set of ELP
    standards should be developed for all of the State's ELLs. However,
    given that standards are major levers for improving student
    achievement by articulating goals and focusing instruction, the
    standards should be accessible to all students.

[^24]: The language of NCLB refers to the targeted student population
    for this framework as "limited English proficient" (LEP). We
    recognize that many researchers and practitioners prefer the term
    English Language Learner (ELL). Consistent with this more general,
    common usage, this framework will use the term English Language
    Learner (ELL).

[^25]: While there may not be just one accepted theory of English
    language development/acquisition (as there is not one accepted
    theory for teaching mathematics, for example) much is known and has
    been validated by experts across several fields, and therefore needs
    to be considered when developing Statewide systems of standards and
    assessments.

[^26]: The point is not [that]{.underline} ELLs are
    diverse/heterogeneous; rather it is [how]{.underline} they are
    diverse/heterogeneous. For example, ELLs differ in their level of
    literacy and amount of formal education in their native language.
    Some ELLs are newly arrived; however, most are second or third
    generation in the U.S. Some ELLs speak and write languages that may
    have sound systems, grammatical structures, and writing systems that
    are different from those of English to varying degrees, which can
    impact the rate and ease of English language acquisition and English
    literacy learning. The nature of the diversity/heterogeneity of this
    group may affect the implementation of standards (e.g., curriculum,
    instruction) and assessments.

[^27]: Appropriate *access* for ELLs on ELP assessments involves the
    removal of construct-irrelevant factors that may interfere with
    ELLs' demonstration of English language proficiency across the
    language domains (listening, speaking, reading, writing).

[^28]: Appropriate *accommodations* for ELLs on ELP assessments involve
    changes in testing conditions (e.g., presentation, setting,
    scheduling/timing) implemented to increase access to test content
    for the student population. Such changes are deemed fair and
    reasonable when standardized administration conditions do not
    provide an equal opportunity for all students to demonstrate what
    they know and can do. An accommodation is intended to minimize or
    remove the effects of construct-irrelevant factors on test
    performance. It is assumed that with or without the accommodation,
    the same construct is being assessed (Abedi & Lord, 2001; Butler &
    Stevens, 2001; Holmes & Duron, 2000; Rivera & Stansfield, 2001).

[^29]: Suggested elements are listed separately in this Framework to
    facilitate State processes for addressing the elements and to
    reflect their individual contribution to a valid and reliable ELP
    assessment system. Although these elements are listed separately,
    States should keep in mind the inter-relationships among these
    suggested elements for establishing a comprehensive and cohesive
    plan for addressing the needs of their ELLs.

[^30]: Some considerations are repeated across suggested elements to
    ensure that States address the considerations as appropriate.

[^31]: This Framework does not address the validity and reliability of
    State accountability systems (e.g., for State accountability
    workbooks) or how the results/uses of ELP assessments ought to
    contribute to State accountability; rather this Framework addresses
    the validity and reliability of ELP assessments for placement and
    proficiency determinations.

[^32]: This aspect of the suggested element is included to be consistent
    with the Title I Peer Review Guidance. We anticipate that, beyond
    the identification and placement process, very few States will
    incorporate local assessments to measure student progress toward
    English language proficiency.

[^33]: This element is included to be consistent with the Title I Peer
    Review Guidance. We anticipate that the matrix design may be likely
    if used to denote an embedded field test section of the test.

[^34]: This Framework is not suggesting that a single assessment
    necessarily serve all three purposes. Rather, the Framework
    encourages States to make sure that all elements of its assessment
    system are coherent and satisfy appropriate technical requirements
    (validity, reliability, freedom from bias).

[^35]: Language skills consistent with higher-order thinking skills and
    understanding include summarization, comparison-contrast, and
    persuasion.

[^36]: This Framework does not address the validity and reliability of
    State accountability systems (e.g., for State accountability
    workbooks) or how the results/uses of ELP assessments ought to
    contribute to State accountability; rather this Framework addresses
    the validity and reliability of ELP assessments for placement and
    proficiency determinations.

[^37]: This Framework does not address the validity and reliability of
    State accountability systems (e.g., for State accountability
    workbooks) or how the results/uses of ELP assessments ought to
    contribute to State accountability; rather this Framework addresses
    the validity and reliability of ELP assessments for placement and
    proficiency determination.

[^38]: As mentioned previously, this Framework does not address the
    validity and reliability of State accountability systems or how the
    results/uses of ELP assessments ought to contribute to State
    accountability. However, for the purpose of providing clarification,
    the AMAOs referred to here relate to measures of progress in
    learning English (in each of the language domains required under
    Title III) and attainment of proficiency in English.

[^39]: This Framework does not address the validity and reliability of
    State accountability systems (e.g., for State accountability
    workbooks) or how the results/uses of ELP assessments ought to
    contribute to State accountability; rather this Framework addresses
    the validity and reliability of ELP assessments for placement and
    proficiency determination.

[^40]: Broadly defined, academic language is the language (e.g.,
    lexical, grammatical, and discourse features specific to and
    reflective of a particular context or content area) students need to
    meaningfully engage with academic content and achieve proficiency in
    English in the academic setting (Sato, 2007). While there may not be
    just one accepted definition of academic language, there are
    numerous resources on the topic (e.g., Aguirre-Munoz, Parks, Benner,
    Amabisca, & Boscardin, 2006; Bailey, 2007; Bailey, Butler, & Sato,
    2007; Butler, Bailey, Stevens, Huang, & Lord, 2004; Cummins, 1980,
    2005; Halliday, 1994; Scarcella & Zimmerman, 1998; Schleppegrell,
    2001) that may be considered when developing ELP standards and
    assessments.

[^41]: This Framework does not address the validity and reliability of
    State accountability systems (e.g., for State accountability
    workbooks) or how the results/uses of ELP assessments ought to
    contribute to State accountability; rather this Framework addresses
    the validity and reliability of ELP assessments for placement and
    proficiency determination.

[^42]: Each State in a consortium should make sure that results from
    validity studies conducted by the consortium are applicable to both
    the particular ELL population (key subgroups, as defined by the
    State vis-à-vis the consortium's definition of subgroups) and
    context (e.g., priorities, uses of results) of the State.

[^43]: The list of resources is representative and not inclusive. These
    resources have not undergone a vetting process to verify their
    relevance, quality, and utility. Inclusion should not be viewed as
    an endorsement by the U.S. Department of Education or by the
    Assessment and Accountability Comprehensive Center.