**North Dakota**

**Consolidated State Application**

**Accountability Workbook**

**for State Grants under Title IX, Part C, Section 9302 of the
Elementary and**

> **Secondary Education Act (Public Law 107-110)**

**SUBMITTED: JANUARY 31, 2003**

**AMENDED: June 6, 2008**

The North Dakota Accountability Workbook herein provides amendments,
submitted to the U.S. Department of Education on June 6, 2008 for
subsequent approval, that relate to the following sections of the
Workbook:

-   Amend Section 1.1 (Page 14, subsection (i)) to clarify the manner in
    which public school districts that mutually reorganize and
    consolidate are combined for the purpose of determining the adequate
    yearly progress for the newly established public school district.
    The proscribed method joins, compiles, and analyzes multi-year,
    historical student achievement data to create a newly validated base
    upon which current and future adequate yearly progress
    determinations are made for the newly established public school
    district.

-   Renew Section 5.3 (page 46, paragraph 6) to allow the State to
    administer an alternate assessment for certain students with
    disabilities based on modified achievement standards for the
    purposes of determining adequate yearly progress and restricting any
    such determinations of proficient and advanced scores to not greater
    than 2 percent of students assessed on the North Dakota State
    Assessment or as provided by federal regulation. The application of
    this amendment is contingent wholly upon the state's submission of
    its alternate assessment based on modified achievement standards to
    the U.S. Department of Education for purposes of peer review and the
    subsequent approval of the alternate assessment for the purposes of
    determining adequate yearly progress, or any interim approval
    granted by the U.S. Department of Education.

![](media/image1.png)

**U. S. Department of Education**

**Office of Elementary and Secondary Education**

**Washington, D.C. 20202**

**Instructions for Completing Consolidated State Application
Accountability Workbook**

By January 31, 2003, States must complete and submit to the Department
this Consolidated State Application Accountability Workbook. We
understand that some of the critical elements for the key principles may
still be under consideration and may not yet be final State policy by
the January 31 due date. States that do not have final approval for some
of these elements or that have not finalized a decision on these
elements by January 31 should, when completing the Workbook, indicate
the status of each element which is not yet official State policy and
provide the anticipated date by which the proposed policy will become
effective. In each of these cases, States must include a timeline of
steps to complete to ensure that such elements are in place by May 1,
2003, and implemented during the 2002-2003 school year. By no later than
May 1, 2003, States must submit to the Department final information for
all sections of the Consolidated State Application Accountability
Workbook.

**Transmittal Instructions**

To expedite the receipt of this Consolidated State Application
Accountability Workbook, please send your submission via the Internet as
a .doc file, pdf file, rtf or .txt file or provide the URL for the site
where your submission is posted on the Internet. Send electronic
submissions to conapp@ed.gov.

A State that submits only a paper submission should mail the submission
by express courier to:

Celia Sims

U.S. Department of Education

400 Maryland Ave., SW

Room 3W300

Washington, D.C. 20202-6400

\(202\) 401-0113

# PART I: Summary of Required Elements for State Accountability Systems 

# Instructions 

# 

# The following chart is an overview of States\' implementation of the critical elements required for approval of their State accountability systems. States must provide detailed implementation information for each of these elements in Part II of this Consolidated State Application Accountability Workbook. 

# 

# For each of the elements listed in the following chart, States should indicate the current implementation status in their State using the following legend:

# 

# F: State has a final policy, approved by all the required entities in the State (e.g., State Board of Education, State Legislature), for implementing this element in its accountability system. 

# 

# P: State has a proposed policy for implementing this element in its accountability system, but must still receive approval by required entities in the State (e.g., State Board of Education, State Legislature). 

# 

# W: State is still working on formulating a policy to implement this element in its accountability system. 

# 

#  

# Summary of Implementation Status for Required Elements of

# State Accountability Systems

+---+----+--------------------------------------------------------------+
| * |    | **State Accountability System Element**                      |
| * |    |                                                              |
| S |    |                                                              |
| t |    |                                                              |
| a |    |                                                              |
| t |    |                                                              |
| u |    |                                                              |
| s |    |                                                              |
| * |    |                                                              |
| * |    |                                                              |
+---+----+--------------------------------------------------------------+
| * |    |                                                              |
| * |    |                                                              |
| [ |    |                                                              |
| P |    |                                                              |
| r |    |                                                              |
| i |    |                                                              |
| n |    |                                                              |
| c |    |                                                              |
| i |    |                                                              |
| p |    |                                                              |
| l |    |                                                              |
| e |    |                                                              |
| 1 |    |                                                              |
| : |    |                                                              |
| A |    |                                                              |
| l |    |                                                              |
| l |    |                                                              |
| S |    |                                                              |
| c |    |                                                              |
| h |    |                                                              |
| o |    |                                                              |
| o |    |                                                              |
| l |    |                                                              |
| s |    |                                                              |
| ] |    |                                                              |
| { |    |                                                              |
| . |    |                                                              |
| u |    |                                                              |
| n |    |                                                              |
| d |    |                                                              |
| e |    |                                                              |
| r |    |                                                              |
| l |    |                                                              |
| i |    |                                                              |
| n |    |                                                              |
| e |    |                                                              |
| } |    |                                                              |
| * |    |                                                              |
| * |    |                                                              |
+---+----+--------------------------------------------------------------+
| F | 1  | Accountability system includes *all schools and districts in |
|   | .1 | the state*.                                                  |
+---+----+--------------------------------------------------------------+
| F | 1  | Accountability system holds *all schools to the same         |
|   | .2 | criteria*.                                                   |
+---+----+--------------------------------------------------------------+
| F | 1  | Accountability system incorporates the *academic achievement |
|   | .3 | standards*.                                                  |
+---+----+--------------------------------------------------------------+
| F | 1  | Accountability system provides *information in a timely      |
|   | .4 | manner*.                                                     |
+---+----+--------------------------------------------------------------+
| F | 1  | Accountability system includes *report cards*.               |
|   | .5 |                                                              |
+---+----+--------------------------------------------------------------+
| F | 1  | Accountability system includes *rewards and sanctions*.      |
|   | .6 |                                                              |
+---+----+--------------------------------------------------------------+
| # |    |                                                              |
|   |    |                                                              |
| [ |    |                                                              |
| P |    |                                                              |
| r |    |                                                              |
| i |    |                                                              |
| n |    |                                                              |
| c |    |                                                              |
| i |    |                                                              |
| p |    |                                                              |
| l |    |                                                              |
| e |    |                                                              |
|   |    |                                                              |
| 2 |    |                                                              |
| : |    |                                                              |
|   |    |                                                              |
| A |    |                                                              |
| l |    |                                                              |
| l |    |                                                              |
|   |    |                                                              |
| S |    |                                                              |
| t |    |                                                              |
| u |    |                                                              |
| d |    |                                                              |
| e |    |                                                              |
| n |    |                                                              |
| t |    |                                                              |
| s |    |                                                              |
| ] |    |                                                              |
| { |    |                                                              |
| . |    |                                                              |
| u |    |                                                              |
| n |    |                                                              |
| d |    |                                                              |
| e |    |                                                              |
| r |    |                                                              |
| l |    |                                                              |
| i |    |                                                              |
| n |    |                                                              |
| e |    |                                                              |
| } |    |                                                              |
+---+----+--------------------------------------------------------------+
| F | 2  | The accountability system includes *all students*            |
|   | .1 |                                                              |
+---+----+--------------------------------------------------------------+
| F | 2  | The accountability system has a consistent definition of     |
|   | .2 | *full academic year*.                                        |
+---+----+--------------------------------------------------------------+
| F | 2  | The accountability system properly includes *mobile          |
|   | .3 | students*.                                                   |
+---+----+--------------------------------------------------------------+
| # |    |                                                              |
|   |    |                                                              |
| [ |    |                                                              |
| P |    |                                                              |
| r |    |                                                              |
| i |    |                                                              |
| n |    |                                                              |
| c |    |                                                              |
| i |    |                                                              |
| p |    |                                                              |
| l |    |                                                              |
| e |    |                                                              |
|   |    |                                                              |
| 3 |    |                                                              |
| : |    |                                                              |
|   |    |                                                              |
| M |    |                                                              |
| e |    |                                                              |
| t |    |                                                              |
| h |    |                                                              |
| o |    |                                                              |
| d |    |                                                              |
|   |    |                                                              |
| o |    |                                                              |
| f |    |                                                              |
|   |    |                                                              |
| A |    |                                                              |
| Y |    |                                                              |
| P |    |                                                              |
|   |    |                                                              |
| D |    |                                                              |
| e |    |                                                              |
| t |    |                                                              |
| e |    |                                                              |
| r |    |                                                              |
| m |    |                                                              |
| i |    |                                                              |
| n |    |                                                              |
| a |    |                                                              |
| t |    |                                                              |
| i |    |                                                              |
| o |    |                                                              |
| n |    |                                                              |
| s |    |                                                              |
| ] |    |                                                              |
| { |    |                                                              |
| . |    |                                                              |
| u |    |                                                              |
| n |    |                                                              |
| d |    |                                                              |
| e |    |                                                              |
| r |    |                                                              |
| l |    |                                                              |
| i |    |                                                              |
| n |    |                                                              |
| e |    |                                                              |
| } |    |                                                              |
+---+----+--------------------------------------------------------------+
| F | 3  | Accountability system expects *all student subgroups, public |
|   | .1 | schools, and LEAs to reach proficiency by 2013-14*.          |
+---+----+--------------------------------------------------------------+
| F | 3  | Accountability system has a method for determining whether   |
|   | .2 | *student subgroups, public schools, and LEAs made adequate   |
|   |    | yearly progress.*                                            |
+---+----+--------------------------------------------------------------+
| F | 3. | Accountability system establishes a *starting point*.        |
|   | 2a |                                                              |
+---+----+--------------------------------------------------------------+
| F | 3. | Accountability system establishes *statewide annual          |
|   | 2b | measurable objectives*.                                      |
+---+----+--------------------------------------------------------------+
| F | 3. | Accountability system establishes *intermediate goals*.      |
|   | 2c |                                                              |
+---+----+--------------------------------------------------------------+
| # |    |                                                              |
|   |    |                                                              |
| [ |    |                                                              |
| P |    |                                                              |
| r |    |                                                              |
| i |    |                                                              |
| n |    |                                                              |
| c |    |                                                              |
| i |    |                                                              |
| p |    |                                                              |
| l |    |                                                              |
| e |    |                                                              |
|   |    |                                                              |
| 4 |    |                                                              |
| : |    |                                                              |
|   |    |                                                              |
| A |    |                                                              |
| n |    |                                                              |
| n |    |                                                              |
| u |    |                                                              |
| a |    |                                                              |
| l |    |                                                              |
|   |    |                                                              |
| D |    |                                                              |
| e |    |                                                              |
| c |    |                                                              |
| i |    |                                                              |
| s |    |                                                              |
| i |    |                                                              |
| o |    |                                                              |
| n |    |                                                              |
| s |    |                                                              |
| ] |    |                                                              |
| { |    |                                                              |
| . |    |                                                              |
| u |    |                                                              |
| n |    |                                                              |
| d |    |                                                              |
| e |    |                                                              |
| r |    |                                                              |
| l |    |                                                              |
| i |    |                                                              |
| n |    |                                                              |
| e |    |                                                              |
| } |    |                                                              |
+---+----+--------------------------------------------------------------+
| F | 4  | The accountability system *determines annually the progress* |
|   | .1 | of schools and districts.                                    |
+---+----+--------------------------------------------------------------+

**STATUS Legend:**

**F** -- Final state policy

**P** -- Proposed policy, awaiting State approval

**W** -- Working to formulate policy

+---+----+--------------------------------------------------------------+
| # |    |                                                              |
|   |    |                                                              |
| [ |    |                                                              |
| P |    |                                                              |
| r |    |                                                              |
| i |    |                                                              |
| n |    |                                                              |
| c |    |                                                              |
| i |    |                                                              |
| p |    |                                                              |
| l |    |                                                              |
| e |    |                                                              |
|   |    |                                                              |
| 5 |    |                                                              |
| : |    |                                                              |
|   |    |                                                              |
| S |    |                                                              |
| u |    |                                                              |
| b |    |                                                              |
| g |    |                                                              |
| r |    |                                                              |
| o |    |                                                              |
| u |    |                                                              |
| p |    |                                                              |
|   |    |                                                              |
| A |    |                                                              |
| c |    |                                                              |
| c |    |                                                              |
| o |    |                                                              |
| u |    |                                                              |
| n |    |                                                              |
| t |    |                                                              |
| a |    |                                                              |
| b |    |                                                              |
| i |    |                                                              |
| l |    |                                                              |
| i |    |                                                              |
| t |    |                                                              |
| y |    |                                                              |
| ] |    |                                                              |
| { |    |                                                              |
| . |    |                                                              |
| u |    |                                                              |
| n |    |                                                              |
| d |    |                                                              |
| e |    |                                                              |
| r |    |                                                              |
| l |    |                                                              |
| i |    |                                                              |
| n |    |                                                              |
| e |    |                                                              |
| } |    |                                                              |
+---+----+--------------------------------------------------------------+
| F | 5  | The accountability system *includes all the required student |
|   | .1 | subgroups*.                                                  |
+---+----+--------------------------------------------------------------+
| F | 5  | The accountability system holds *schools and LEAs            |
|   | .2 | accountable for the progress of* *student subgroups*.        |
+---+----+--------------------------------------------------------------+
| F | 5  | The accountability system includes *students with            |
|   | .3 | disabilities*.                                               |
+---+----+--------------------------------------------------------------+
| F | 5  | The accountability system includes *limited English          |
|   | .4 | proficient students*.                                        |
+---+----+--------------------------------------------------------------+
| F | 5  | The State has determined the minimum number of students      |
|   | .5 | sufficient to yield statistically reliable information for   |
|   |    | each purpose for which disaggregated data are used.          |
+---+----+--------------------------------------------------------------+
| F | 5  | The State has strategies to protect the privacy of           |
|   | .6 | individual students in reporting achievement results and in  |
|   |    | determining whether schools and LEAs are making adequate     |
|   |    | yearly progress on the basis of disaggregated subgroups.     |
+---+----+--------------------------------------------------------------+
| # |    |                                                              |
|   |    |                                                              |
| [ |    |                                                              |
| P |    |                                                              |
| r |    |                                                              |
| i |    |                                                              |
| n |    |                                                              |
| c |    |                                                              |
| i |    |                                                              |
| p |    |                                                              |
| l |    |                                                              |
| e |    |                                                              |
|   |    |                                                              |
| 6 |    |                                                              |
| : |    |                                                              |
|   |    |                                                              |
| B |    |                                                              |
| a |    |                                                              |
| s |    |                                                              |
| e |    |                                                              |
| d |    |                                                              |
|   |    |                                                              |
| o |    |                                                              |
| n |    |                                                              |
|   |    |                                                              |
| A |    |                                                              |
| c |    |                                                              |
| a |    |                                                              |
| d |    |                                                              |
| e |    |                                                              |
| m |    |                                                              |
| i |    |                                                              |
| c |    |                                                              |
|   |    |                                                              |
| A |    |                                                              |
| s |    |                                                              |
| s |    |                                                              |
| e |    |                                                              |
| s |    |                                                              |
| s |    |                                                              |
| m |    |                                                              |
| e |    |                                                              |
| n |    |                                                              |
| t |    |                                                              |
| s |    |                                                              |
| ] |    |                                                              |
| { |    |                                                              |
| . |    |                                                              |
| u |    |                                                              |
| n |    |                                                              |
| d |    |                                                              |
| e |    |                                                              |
| r |    |                                                              |
| l |    |                                                              |
| i |    |                                                              |
| n |    |                                                              |
| e |    |                                                              |
| } |    |                                                              |
+---+----+--------------------------------------------------------------+
| F | 6  | Accountability system is based *primarily on academic        |
|   | .1 | assessments*.                                                |
+---+----+--------------------------------------------------------------+
| # |    |                                                              |
|   |    |                                                              |
| [ |    |                                                              |
| P |    |                                                              |
| r |    |                                                              |
| i |    |                                                              |
| n |    |                                                              |
| c |    |                                                              |
| i |    |                                                              |
| p |    |                                                              |
| l |    |                                                              |
| e |    |                                                              |
|   |    |                                                              |
| 7 |    |                                                              |
| : |    |                                                              |
|   |    |                                                              |
| A |    |                                                              |
| d |    |                                                              |
| d |    |                                                              |
| i |    |                                                              |
| t |    |                                                              |
| i |    |                                                              |
| o |    |                                                              |
| n |    |                                                              |
| a |    |                                                              |
| l |    |                                                              |
|   |    |                                                              |
| I |    |                                                              |
| n |    |                                                              |
| d |    |                                                              |
| i |    |                                                              |
| c |    |                                                              |
| a |    |                                                              |
| t |    |                                                              |
| o |    |                                                              |
| r |    |                                                              |
| s |    |                                                              |
| ] |    |                                                              |
| { |    |                                                              |
| . |    |                                                              |
| u |    |                                                              |
| n |    |                                                              |
| d |    |                                                              |
| e |    |                                                              |
| r |    |                                                              |
| l |    |                                                              |
| i |    |                                                              |
| n |    |                                                              |
| e |    |                                                              |
| } |    |                                                              |
+---+----+--------------------------------------------------------------+
| F | 7  | Accountability system includes *graduation rate for high     |
|   | .1 | school*s.                                                    |
+---+----+--------------------------------------------------------------+
| F | 7  | Accountability system includes an *additional academic       |
|   | .2 | indicator for elementary and middle school*s.                |
+---+----+--------------------------------------------------------------+
| F | 7  | Additional indicators are valid and reliable.                |
|   | .3 |                                                              |
+---+----+--------------------------------------------------------------+
| # |    |                                                              |
|   |    |                                                              |
| [ |    |                                                              |
| P |    |                                                              |
| r |    |                                                              |
| i |    |                                                              |
| n |    |                                                              |
| c |    |                                                              |
| i |    |                                                              |
| p |    |                                                              |
| l |    |                                                              |
| e |    |                                                              |
|   |    |                                                              |
| 8 |    |                                                              |
| : |    |                                                              |
|   |    |                                                              |
| S |    |                                                              |
| e |    |                                                              |
| p |    |                                                              |
| a |    |                                                              |
| r |    |                                                              |
| a |    |                                                              |
| t |    |                                                              |
| e |    |                                                              |
|   |    |                                                              |
| D |    |                                                              |
| e |    |                                                              |
| c |    |                                                              |
| i |    |                                                              |
| s |    |                                                              |
| i |    |                                                              |
| o |    |                                                              |
| n |    |                                                              |
| s |    |                                                              |
|   |    |                                                              |
| f |    |                                                              |
| o |    |                                                              |
| r |    |                                                              |
|   |    |                                                              |
| R |    |                                                              |
| e |    |                                                              |
| a |    |                                                              |
| d |    |                                                              |
| i |    |                                                              |
| n |    |                                                              |
| g |    |                                                              |
| / |    |                                                              |
| L |    |                                                              |
| a |    |                                                              |
| n |    |                                                              |
| g |    |                                                              |
| u |    |                                                              |
| a |    |                                                              |
| g |    |                                                              |
| e |    |                                                              |
|   |    |                                                              |
| A |    |                                                              |
| r |    |                                                              |
| t |    |                                                              |
| s |    |                                                              |
|   |    |                                                              |
| a |    |                                                              |
| n |    |                                                              |
| d |    |                                                              |
|   |    |                                                              |
| M |    |                                                              |
| a |    |                                                              |
| t |    |                                                              |
| h |    |                                                              |
| e |    |                                                              |
| m |    |                                                              |
| a |    |                                                              |
| t |    |                                                              |
| i |    |                                                              |
| c |    |                                                              |
| s |    |                                                              |
| ] |    |                                                              |
| { |    |                                                              |
| . |    |                                                              |
| u |    |                                                              |
| n |    |                                                              |
| d |    |                                                              |
| e |    |                                                              |
| r |    |                                                              |
| l |    |                                                              |
| i |    |                                                              |
| n |    |                                                              |
| e |    |                                                              |
| } |    |                                                              |
+---+----+--------------------------------------------------------------+
| F | 8  | Accountability system holds students, schools and districts  |
|   | .1 | separately accountable for *reading/language arts* and       |
|   |    | *mathematics*.                                               |
+---+----+--------------------------------------------------------------+
| # |    |                                                              |
|   |    |                                                              |
| [ |    |                                                              |
| P |    |                                                              |
| r |    |                                                              |
| i |    |                                                              |
| n |    |                                                              |
| c |    |                                                              |
| i |    |                                                              |
| p |    |                                                              |
| l |    |                                                              |
| e |    |                                                              |
|   |    |                                                              |
| 9 |    |                                                              |
| : |    |                                                              |
|   |    |                                                              |
| S |    |                                                              |
| y |    |                                                              |
| s |    |                                                              |
| t |    |                                                              |
| e |    |                                                              |
| m |    |                                                              |
|   |    |                                                              |
| V |    |                                                              |
| a |    |                                                              |
| l |    |                                                              |
| i |    |                                                              |
| d |    |                                                              |
| i |    |                                                              |
| t |    |                                                              |
| y |    |                                                              |
|   |    |                                                              |
| a |    |                                                              |
| n |    |                                                              |
| d |    |                                                              |
|   |    |                                                              |
| R |    |                                                              |
| e |    |                                                              |
| l |    |                                                              |
| i |    |                                                              |
| a |    |                                                              |
| b |    |                                                              |
| i |    |                                                              |
| l |    |                                                              |
| i |    |                                                              |
| t |    |                                                              |
| y |    |                                                              |
| ] |    |                                                              |
| { |    |                                                              |
| . |    |                                                              |
| u |    |                                                              |
| n |    |                                                              |
| d |    |                                                              |
| e |    |                                                              |
| r |    |                                                              |
| l |    |                                                              |
| i |    |                                                              |
| n |    |                                                              |
| e |    |                                                              |
| } |    |                                                              |
+---+----+--------------------------------------------------------------+
| F | 9  | Accountability system produces *reliable decisions*.         |
|   | .1 |                                                              |
+---+----+--------------------------------------------------------------+
| F | 9  | Accountability system produces *valid decisions*.            |
|   | .2 |                                                              |
+---+----+--------------------------------------------------------------+
| F | 9  | State has a plan for addressing *changes in assessment and   |
|   | .3 | student population*.                                         |
+---+----+--------------------------------------------------------------+
| # |    |                                                              |
|   |    |                                                              |
| [ |    |                                                              |
| P |    |                                                              |
| r |    |                                                              |
| i |    |                                                              |
| n |    |                                                              |
| c |    |                                                              |
| i |    |                                                              |
| p |    |                                                              |
| l |    |                                                              |
| e |    |                                                              |
|   |    |                                                              |
| 1 |    |                                                              |
| 0 |    |                                                              |
| : |    |                                                              |
|   |    |                                                              |
| P |    |                                                              |
| a |    |                                                              |
| r |    |                                                              |
| t |    |                                                              |
| i |    |                                                              |
| c |    |                                                              |
| i |    |                                                              |
| p |    |                                                              |
| a |    |                                                              |
| t |    |                                                              |
| i |    |                                                              |
| o |    |                                                              |
| n |    |                                                              |
|   |    |                                                              |
| R |    |                                                              |
| a |    |                                                              |
| t |    |                                                              |
| e |    |                                                              |
| ] |    |                                                              |
| { |    |                                                              |
| . |    |                                                              |
| u |    |                                                              |
| n |    |                                                              |
| d |    |                                                              |
| e |    |                                                              |
| r |    |                                                              |
| l |    |                                                              |
| i |    |                                                              |
| n |    |                                                              |
| e |    |                                                              |
| } |    |                                                              |
+---+----+--------------------------------------------------------------+
| F | 10 | Accountability system has a means for calculating the *rate  |
|   | .1 | of participation* in the statewide assessment.               |
+---+----+--------------------------------------------------------------+
| F | 10 | Accountability system has a means for *applying the 95%      |
|   | .2 | assessment criteria to student subgroups* *and small         |
|   |    | schools.*                                                    |
+---+----+--------------------------------------------------------------+

**STATUS Legend:**

F -- Final policy

P -- Proposed Policy, awaiting State approval

W-- Working to formulate policy

**PART II: State Response and Activities for Meeting State
Accountability System Requirements**

Instructions

In Part II of this Workbook, States are to provide detailed information
for each of the critical elements required for State accountability
systems. States should answer the questions asked about each of the
critical elements in the State\'s accountability system. States that do
not have final approval for any of these elements or that have not
finalized a decision on these elements by January 31, 2003, should, when
completing this section of the Workbook, indicate the status of each
element that is not yet official State policy and provide the
anticipated date by which the proposed policy will become effective. In
each of these cases, States must include a timeline of steps to complete
to ensure that such elements are in place by May 1, 2003, and
implemented during the 2002-2003 school year. By no later than May 1,
2003, States must submit to the Department final information for all
sections of the Consolidated State Application Accountability Workbook.

**PRINCIPLE 1. A single statewide Accountability System applied to all
public schools and LEAs.**

+-----------------------+-----------------------+-----------------------+
| **CRITICAL ELEMENT**  | **EXAMPLES FOR**      | **EXAMPLES OF**       |
|                       |                       |                       |
|                       | **MEETING STATUTORY   | ***NOT* MEETING       |
|                       | REQUIREMENTS**        | REQUIREMENTS**        |
+-----------------------+-----------------------+-----------------------+
| 1.  How does the      | Every public school   | A public school or    |
|     State             | and LEA is required   | LEA is not required   |
|     Accountability    | to make adequate      | to make adequate      |
|     System include    | yearly progress and   | yearly progress and   |
|     every public      | is included in the    | is not included in    |
|     school and LEA in | State Accountability  | the State             |
|     the State?        | System.               | Accountability        |
|                       |                       | System.               |
|                       | State has a           |                       |
|                       | definition of "public | State policy          |
|                       | school" and "LEA" for | systematically        |
|                       | AYP accountability    | excludes certain      |
|                       | purposes.             | public schools and/or |
|                       |                       | LEAs.                 |
|                       | -   The State         |                       |
|                       |     Accountability    |                       |
|                       |     System produces   |                       |
|                       |     AYP decisions for |                       |
|                       |     all public        |                       |
|                       |     schools,          |                       |
|                       |     including public  |                       |
|                       |     schools with      |                       |
|                       |     variant grade     |                       |
|                       |     configurations    |                       |
|                       |     (e.g., K-12),     |                       |
|                       |     public schools    |                       |
|                       |     that serve        |                       |
|                       |     special           |                       |
|                       |     populations       |                       |
|                       |     (e.g.,            |                       |
|                       |     alternative       |                       |
|                       |     public schools,   |                       |
|                       |     juvenile          |                       |
|                       |     institutions,     |                       |
|                       |     state public      |                       |
|                       |     schools for the   |                       |
|                       |     blind) and public |                       |
|                       |     charter schools.  |                       |
|                       |     It also holds     |                       |
|                       |     accountable       |                       |
|                       |     public schools    |                       |
|                       |     with no grades    |                       |
|                       |     assessed (e.g.,   |                       |
|                       |     K-2).             |                       |
+-----------------------+-----------------------+-----------------------+
| **STATE RESPONSE AND  |                       |                       |
| STATE ACTIVITIES FOR  |                       |                       |
| MEETING               |                       |                       |
| REQUIREMENTS**        |                       |                       |
+-----------------------+-----------------------+-----------------------+

The State of North Dakota stipulates that every public school and LEA is
held accountable to the provisions of adequate yearly progress and is
included in the State Accountability System. The North Dakota Assessment
System assesses *all* students within a single, unified, statewide
assessment that measures students' performance in terms of the State's
challenging content and achievement standards and that all schools and
all LEAs are measured for adequate yearly progress within a single,
unified accountability system.

North Dakota, through an agreement with the U.S. Department of
Education, has established an assessment waiver plan to bring the State
into full compliance with ESEA, Section 1111(b)(1) requirements. This
waiver plan, approved through August 2003, is enclosed as **Appendix A:
North Dakota State Assessment Waiver Agreement Plan** and can be
accessed at the following web site:
<http://www.dpi.state.nd.us/testing/assess/plan.pdf>. To date, the State
has met all objectives identified within the waiver agreement plan. The
State stipulates that it will meet all requirements identified within
the Waiver Agreement Plan.

The evidence of a single, unified, statewide assessment and
accountability system is demonstrated by the grounding authority for
State content standards and assessments in North Dakota State Law and in
the North Dakota Department of Public Instruction's adherence to the
requirements of Federal Law.

**(a) Authority within State Law for State Content Standards.**

North Dakota state law (NDCC 15.1-02-04.3) places responsibility for the
development of State academic content standards with the State
Superintendent (refer to **Appendix B: North Dakota Century Code
citations** or reference the North Dakota Century Code at the following
web site,
[http://www.state.nd.us/lr/cencode/T151C02.pdf](http://www.state.nd.us/lr/cencode/t151)
). The North Dakota Department of Public Instruction has developed and
adopted academic content standards in mathematics (reference **Appendix
D: North Dakota Mathematics Content Standards** or reference these
standards at the following web site,
<http://www.dpi.state.nd.us/standard/content/math.pdf>) and English
language arts (refer to **Appendix F: North Dakota English Language Arts
Content Standards** or reference these standards at the following web
site, <http://www.dpi.state.nd.us/standard/content/english.pdf>). These
State content standards have been developed at grades 4, 8, and 12 in
accordance with the *North Dakota Standards and Assessment Development
Protocols* (refer to **Appendix C: North Dakota Standards and Assessment
Development Protocols** or reference the following web site,
<http://www.dpi.state.nd.us/standard/content/toc.pdf>). North Dakota
mathematics and English language arts academic content standards meet
the requirements of section 1111(b)(1).

The State Superintendent oversees and approves all standards
development. A State-level advisory committee consisting of LEA and SEA
representatives, titled the *Standards, Assessment, Learning and
Teaching (SALT) Team*, advises the Department of Public Instruction on
the process and quality of standards development committee work. North
Dakota's standards development protocols currently are being revised to
incorporate improvements into the development process and to accommodate
the development of grade-level content standards in grades 3, 5, 6, and
7.

North Dakota will continue to use adopted content standards as the basis
for statewide assessments at grades 4, 8, and 12 in accordance with *No
Child Left Behind Act (NCLBA),* section 1111(b)(1). In addition, North
Dakota will expand its statewide assessments into grades 3, 5, 6, and 7,
in accordance with NCLBA section 1111(b)(1) by 2005-06, based on
state-defined, grade-level content standards in reading/English language
arts and mathematics. These grade-level content standards will be
developed and adopted in accordance with North Dakota's standards
development protocols.

North Dakota has submitted its plan for the development of
grade-specific content standards to meet the requirements of NCLBA. This
submission was a part of the State's Consolidated Application for ESEA
funding, dated June 2002. Refer to **Appendix H: North Dakota State ESEA
Consolidated Application**, pages 3-6, or refer to the following web
site: <http://www.dpi.state.nd.us/grants/DOEapp.pdf>). The North Dakota
State Consolidated Application has since been approved by the U.S.
Department of Education.

**(b) Authority within State Law for State Assessments.**

North Dakota state law (NDCC 15.1-21-08) places responsibility with the
State Superintendent for the administration of State assessments to all
public schools statewide that are aligned to the State's content
standards in reading and mathematics (refer to **Appendix B: North
Dakota Century Code citations** or reference the North Dakota Century
Code at the following web site,
<http://www.state.nd.us/lr/cencode/T151C21.pdf>). State law requires
that the assessments be administered to at least one grade level
selected within each of the following grade spans: grades three through
five; grades six through nine; and grades ten through twelve. The North
Dakota Department of Public Instruction has developed and administers
assessments at grades 4, 8, and 12 to correspond with the State's
content standards.

In April 2003, the North Dakota Legislative Assembly enacted legislation
regarding the administration of state assessments in high school.
Enrolled Senate Bill 2065, which becomes effective in August 2003,
requires that the administration of the state assessments in high school
occur during the fall of the eleventh grade effective during at least
the 2005-06 school year. The assessment development and administration
schedule presented throughout this application assumes the
administration of the high school state assessments at the twelfth grade
until the 2004-05 school year when the assessments will be administered
at the eleventh grade. Eleventh grade assessments will be aligned to the
proper grade level content standards. The content standards development
process will provide for the proper alignment of all content standards.

State law requires that the State assessments compile aggregated results
and disaggregated results. The State assessments must compile student
achievement data that allows for a comparison of individual students,
classrooms within a given school and school district, schools within the
district, and school districts within the state. The test scores must
also allow for comparisons based on students' gender, ethnicity,
economic status, service status (i.e., migrant, disability, limited
English proficient) and assessment status (i.e., enrollment period
within a school and LEA), unless doing so enables the identification of
any student. (Refer to NDCC 15.1-21-08 within **Appendix B: North Dakota
Century Code citations** or reference the North Dakota Century Code at
the following web site, <http://www.state.nd.us/lr/cencode/T151C21.pdf>.

State law requires the State Superintendent to present to the
legislative council the test scores publicly for the first time at a
meeting of a legislative committee designated by the legislative
council. At the meeting, the superintendent and representatives of the
testing service that created the tests shall provide detailed testimony
regarding the testing instrument, the methodology used to test and
assess the students, and the significance of the test scores. (Refer to
NDCC 15.1-21-09 within **Appendix B: North Dakota Century Code
citations** or reference the North Dakota Century Code at the following
web site,
[http://www.state.nd.us/lr/cencode/T151C21.pdf](http://www.state.nd.us/lr/assembly/57-2001/cencode/CCT15x1.pdf)
).

State law requires the State Superintendent to require that the entity
developing a test to be administered under section 15.1-21-08 not
include questions that might be deemed personal to a student or to the
student's family and that the entity developing the test not include
questions requiring responses that might be deemed personal to a student
or to the student's family. Before a test is finalized for use in North
Dakota, the State Superintendent must require that the test be reviewed
by a standards alignment committee appointed by the State Superintendent
to ensure that the test meets the requirement of privacy. (Refer to NDCC
15.1-21-11 within **Appendix B: North Dakota Century Code citations** or
reference the North Dakota Century Code at the following web site,
[http://www.state.nd.us/lr/cencode/T151C21.pdf](http://www.state.nd.us/lr/assembly/57-2001/cencode/CCT15x1.pdf)
).

State law requires school districts to allow any individual over the age
of twenty to view any test administered under sections 15.1-21-08 as
soon as the test is in the possession of the school district. (Refer to
NDCC 15.1-21-14 within **Appendix B: North Dakota Century Code
citations** or reference the North Dakota Century Code at the following
web site,
[http://www.state.nd.us/lr/cencode/T151C21.pdf](http://www.state.nd.us/lr/assembly/57-2001/cencode/CCT15x1.pdf)
).

**(c) Fulfilling the Requirements of the ESEA Waiver Plan**

North Dakota, through an agreement with the U.S. Department of
Education, has established an assessment waiver plan to bring the State
into full compliance with ESEA, Section 1111(b)(1) requirements. . This
waiver plan, approved through August 2003, is enclosed as Appendix A:
North Dakota State Assessment Waiver Agreement Plan and can be accessed
at the following web site:
[**http://www.dpi.state.nd.us/testing/assess/plan.pdf**](http://www.dpi.state.nd.us/testing/assess/plan.pdf).
During the 2001-02 school year, North Dakota administered its state
assessment and is on schedule to meet fully all provisions set forth
within the waiver plan.

State assessments have been developed and adopted thus far in
mathematics and reading at grades 4, 8, and 12 in accordance with North
Dakota's approved assessment waiver agreement and the *North Dakota
Standards and Assessment Development Protocols*
(<http://www.dpi.state.nd.us/standard/content/toc.pdf>). North Dakota
will proceed to develop state assessments in mathematics and reading at
additional grades (grades 3, 5, 6, and 7) by 2005-2006 in accordance
with State protocols and section 1111(b)(1) requirements. North Dakota
will proceed to develop state assessments in science at grades 4, 8, and
12 by 2007-2008 in accordance with State protocols and section
1111(b)(1) requirements. Additionally, North Dakota will expand its
science assessment, voluntarily, at grades 3, 5, 6, and 7 by 2007-2008,
based on the availability of ESEA Title VI funds, in accordance with
State protocols and section 1111(b)(1) standards.

North Dakota has submitted its plan to expand the development of grade
specific assessments to meet the requirements of NCLBA. This submission
was an element of the State's Consolidated Application for ESEA funding,
dated June 2002. Refer to **Appendix H: North Dakota State ESEA
Consolidated Application**, pages 7-10, or refer to the following web
site: <http://www.dpi.state.nd.us/grants/DOEapp.pdf>). The North Dakota
State Consolidated Application has since been approved by the U.S.
Department of Education.

The North Dakota Assessment System provides for a single, unified,
statewide assessment that measures the performance of *all* students in
*all* schools and *all* LEAs in terms of the State's challenging content
and achievement standards.

**(d) Fulfilling the requirements of ESEA Consolidated Application.
Agreement to administer a statewide accountability system based on
adequate yearly progress.**

State law grants to the State Superintendent of Public Instruction
authority to apply for, abide by the requirements of, and administer any
federal funded program on behalf of the State of North Dakota. In June
2002, the State Superintendent of Public Instruction signed the North
Dakota Consolidated Application for programs administered under the
*Elementary and Secondary Education Act*. This application included a
signed certificate of assurances that obligated the State to administer
a single, unified assessment and accountability system based on adequate
yearly progress. With the signature of the State Superintendent, the
State of North Dakota entered into an agreement with the U.S. Department
of Education to abide with all provisions of Section 1111 of the ESEA,
including all elements of accountability based on adequate yearly
progress.

\(e\) Accountability System applies to all public schools within North
Dakota, including schools with variant grade configurations, schools
serving special populations, and schools that with no grades assessed.

The State of North Dakota stipulates that all public schools, regardless
of grade configuration or service population, will participate in the
state accountability system. State law defines any public school to
include any educational institution supported through State funding. The
state accountability system will include all public schools identified
as K-12, all alternative public schools, the North Dakota School for the
Deaf and the North Dakota State Youth Correctional Center.

Most schools within North Dakota minimally cover grade spans of K-6,
6-8, or 9-12. However, a review of School Year 2001-02 statewide student
enrollments reveals 10 individual schools with student populations that
do not fit within the typical grade span observed statewide. The
following data indicate the respective number and type of school grade
spans that do not correspond to the general assessment grade spans.
Refer to Appendix I: Schools Falling Outside Assessment System Grade
Spans for a list of schools identified with a type of organization that
does not allow for any assessments within the State Assessment System.

+--------+-----------+--------+--------+--------+--------+--------+
|        | **Type of |        |        |        |        |        |
|        | School    |        |        |        |        |        |
|        | Org       |        |        |        |        |        |
|        | anization |        |        |        |        |        |
|        | (grade    |        |        |        |        |        |
|        | span)**   |        |        |        |        |        |
+--------+-----------+--------+--------+--------+--------+--------+
|        | #### Kin  | *      | *      | *      | *      | **     |
|        | dergarten | *K-1** | *K-2** | *K-3** | *6-7** | 9-10** |
+--------+-----------+--------+--------+--------+--------+--------+
| **     | 3         | 2      | 2      | 1      | 1      | 1      |
| Number |           |        |        |        |        |        |
| of     |           |        |        |        |        |        |
| Sch    |           |        |        |        |        |        |
| ools** |           |        |        |        |        |        |
+--------+-----------+--------+--------+--------+--------+--------+

> Students who attend any of the schools above will eventually graduate
> to a higher grade level in another designated school. As such, there
> is a clearly identified school that will receive each student from
> their school-of-origin listed above. Where schools-of-origin exist
> with grade spans that do not allow for the administration of the State
> Assessment, as are the cases above, student achievement reports from
> the receiving school will be forwarded to the school-of-origin by the
> State. No reports will be issued that might identify an individual
> student. Each school in which no assessments occur will be link
> directly to the supporting district. As students are promoted to
> school plants where assessments occur, students will participate in
> the assessment and accountability system. Every school, regardless of
> classification, resides within a district that participates in the
> State accountability system. Listed below are the linkages for schools
> that do not assess students currently because of their classification.
> Some schools will begin assessing students by at least 2005-06 when
> the State begins assessing all grades 3-8 and high school.
>
> ***Non-Assessed Schools Linked to District Accountability***

1.  Naughton School linked to Naughton Public School District;

2.  Agassiz Middle School linked to Fargo Public School District;

3.  Eagle Kindergarten Center linked to Fargo Public School District;

4.  Davenport Elementary School linked to Kindred Public School
    > District;

5.  Early Childhood Center linked to West Fargo Public School District;

6.  Griggs County Central Elementary School linked to Griggs County
    > Central District;

7.  Stevenson School linked to Bowline Butte Public School District;

8.  Zimmerman Elementary School linked to Wahpeton Public School
    > District;

9.  Central Campus School linked to Minot Public School District;

10. New Kindergarten School linked to New 8 Public School District
    > District.

> **(f) Rules for Performance, Participation, and Graduation For
> Alternative High Schools**

***Rules regarding performance and participation rate.***

*General rule regarding performance and participation rate:* Given the
inherently high transfer rate between traditional and alternative high
schools, and the cumbersome nature of tracking such transfers, the
student performance and participation measures for alternative high
schools will be rolled up to the traditional high school, school
district of residence, or the State.

*Beginning of the year definition.* To identify the status of students
within the Accountability System, the State will employ a "beginning of
the school year" definition. For the 2001-02 school year, the beginning
of the school year is defined as 150 school days prior to the first day
of the spring testing window. In subsequent years, the number of days
will be defined as the number of school days preceding the first day of
the testing window, as determined by the State. This will accommodate
both the fall and spring testing windows.

*Specific rules for performance and participation.*

1.  If a student is enrolled in an alternative high school and is a
    resident of the school district in which the alternative high school
    is located, the student's performance and participation are
    attributed to the resident school district.

2.  If a student attends an alternative high school but is enrolled in
    the school district's traditional high school, then the student's
    performance and participation are attributed to the traditional high
    school.

3.  If a student is a resident of a school district other than the one
    in which the alternative high school is located, and the student's
    resident school district claims the student for pupil membership,
    the student's performance and participation are attributed to the
    student's school district of residence.

4.  If a student transfers from one school district to another (whether
    the receiving school district is the location of the alternative
    high school or not) since the beginning of the school year, then the
    student's performance and participation are attributed to the State.

***General rule regarding graduation rate.***

*General rule for graduation.* Given the inherently high transfer rate
between traditional and alternative high schools, and the cumbersome
nature of tracking such transfers, the student graduation measure for
alternative high schools will be rolled up to the traditional high
school, school district of residence, or the State.

*Cohort definition.* To identify the status of students within the
Accountability System, the State will employ a cohort definition. A
cohort begins from entry as identified by the school's definition (grade
9 for a grade 9-12 school, or grade 10 for grade 10-12 school) and
extends until age 21 or until graduation, whichever occurs first. The
formula to determine graduation rate is stated in the Accountability
Workbook, page 50.

*Specific rules for graduation.*

1.  If a student is enrolled in an alternative high school and is a
    resident of the school district in which the alternative high school
    is located, the student's graduation is attributed to the resident
    school district.

2.  If student attends an alternative high school but is enrolled in the
    school district's traditional high school, then the student's
    graduation is attributed to the traditional high school.

3.  If the student is a resident of a school district other than the one
    in which the alternative high school is located, and the student's
    resident school district claims the student for pupil membership,
    the student's graduation is attributed to the student's school
    district of residence.

4.  If a student transfers from one school district to another since the
    beginning of the cohort, then the student's graduation is attributed
    to the State.

> **(g) Rules for Performance and Participation For Atypical Education
> Settings**

*General rules for performance and participation.* The following general
rules apply when determining the educational entity to which a student's
performance and participation will be attributed.

1.  If the student physically attends the public school, performance and
    participation are attributed to that school, the school district,
    and the State.

2.  If the public or private school or facility serves the student on a
    contract basis, the student's school district of residence is
    responsible; student performance and participation are attributed to
    the school district of residence.

3.  If the student is served in a state facility, student performance
    and participation are attributed to the State.

*Beginning of the year definition.* To identify the status of students
within the Accountability System, the State will employ a "beginning of
the school year" definition. For the 2001-02 school year, the beginning
of the school year is defined as 150 school days prior to the first day
of the spring testing window. In subsequent years, the number of days
will be defined as the number of school days preceding the first day of
the testing window, as determined by the State. This will accommodate
both the fall and spring testing windows.

*Specific rules for performance and participation.* Students may attend
school in other than the public school in their school district of
residence for either a brief or extended period of time due to (1)
choice; (2) developmental or health concerns; or (3) behavior/discipline
issues or adjudication.

*(1) Specific rules for performance and participation related to
choice.*

> If a student attends a school and school district other than his or
> her school or school district of residence and the serving school
> district claims pupil membership for the student, performance and
> participation are attributed to the serving school, school district,
> and State. This applies to:

a.  Job Corps students

b.  Air Force Base students

c.  Open enrolled students

*(2) Specific rules for performance and participation related to
developmental and health concerns.*

a.  If a student is served under contract to a public or private
    facility or to another public or private school or school district,
    performance and participation are attributed to the school district
    of residence and State. This applies to:

```{=html}
<!-- -->
```
i.  Anne Carlsen Center

ii. Developmental Center

iii. Adolescent Unit of Jamestown State Hospital

iv. Students attending psychiatric treatment or mental health facilities

v.  Some students receiving special education services

```{=html}
<!-- -->
```
b.  If a student is placed in a treatment facility out of North Dakota
    and the North Dakota school district of residence claims pupil
    membership, performance and participation are attributed to the
    school district and State.

c.  If a student is served at the North Dakota School for the Deaf
    (NDSD), performance and participation are attributed to NDSD and the
    State.

*(3) Specific rules for performance and participation related to
behavior/discipline or adjudication issues.*

a.  If a student is served at the Youth Correctional Center or State
    Penitentiary, performance and participation are attributed to the
    State.

b.  If a student is incarcerated and is claimed by the school district
    of residence for pupil membership, performance and participation are
    attributed to the school district. However, if a student is
    incarcerated and is not claimed by the school district of residence
    for pupil membership, performance and participation are attributed
    to the State.

c.  If a student is served at the Adolescent Unit at the North Dakota
    State Hospital, performance and participation are attributed to the
    State.

d.  If a student is served at Dakota Boys Ranch (Minot and Fargo),
    performance and participation are attributed to the State.

e.  If a student is served at Home on the Range (Beach), performance and
    participation are attributed to the Beach school district.

f.  If the student who is less than 16 years of age is truant and the
    school district of residence claims pupil membership for the
    student, participation for that student is attributed to that school
    district of residence and the State. However, if the student who is
    less than 16 years of age is truant and the school district of
    residence does not claim pupil membership for the student,
    participation for that student is attributed to the State.

g.  If a student who is less than 16 years of age is suspended or
    expelled from school, as evidenced through appropriate
    documentation, and who is claimed by the school district of
    residence for purposes of pupil membership, performance and
    participation are attributed to the school district of residence and
    the State according to the beginning of year rule. However, if a
    student who is less than 16 years of age is suspended or expelled
    from school and who is not claimed by the school district of
    residence for purposes of pupil membership, participation is
    attributed to the State.

h.  If a student is placed with foster parents who reside in a North
    Dakota school district, performance and participation are attributed
    to the serving school district and State.

> **(h) Definition of "public school" for AYP determination.**
>
> For the purposes of determining AYP, a public school within North
> Dakota is identified by the grade levels it serves and is approved to
> operate based upon its meeting criteria established in State law (NDCC
> 15.1-06-06). Schools report their approval status annually, as
> identified on the State's MIS 02 report for school approval. The
> Department of Public Instruction will reference this grade level
> approval status for the purposes of classifying and reporting public
> schools.

**(i) Reorganized and/or consolidated public school districts.**

The State of North Dakota stipulates that all public school districts,
regardless of any operational or planned reorganization or
consolidation, will receive a full and complete determination of
adequate yearly progress wholly consistent with all provisions and
assurances contained within the state's Consolidated State Application
Accountability Workbook. Any public school districts that were to
reorganize and/or consolidate through mutual agreement pursuant to
provisions of state law will be recognized as a newly established,
unified public school district for the purposes of determining adequate
yearly progress upon the specified date of their reorganization and/or
consolidation. The Department of Public Instruction will join, compile
and analyze the combined multi-year, historical student achievement data
of all reorganizing and/or consolidating public school districts to
create a newly validated base upon which current and future adequate
yearly progress determinations are made for the newly established public
school district. Any newly established public school district will
receive a complete determination, based on all applicable rules and
assurances, and the resulting determination will be issued as the
official standing determination of the newly established public school
district, effective the date of its reorganization and/or consolidation.

+-----------------------+-----------------------+-----------------------+
| **CRITICAL ELEMENT**  | **EXAMPLES FOR**      | **EXAMPLES OF**       |
|                       |                       |                       |
|                       | **MEETING STATUTORY   | ***NOT* MEETING       |
|                       | REQUIREMENTS**        | REQUIREMENTS**        |
+-----------------------+-----------------------+-----------------------+
| 1.  How are all       | All public schools    | Some public schools   |
|     public schools    | and LEAs are          | and LEAs are          |
|     and LEAs held to  | systematically judged | systematically judged |
|     the same criteria | on the basis of the   | on the basis of       |
|     when making an    | same criteria when    | alternate criteria    |
|     AYP               | making an AYP         | when making an AYP    |
|     determination?    | determination.        | determination.        |
|                       |                       |                       |
|                       | If applicable, the    |                       |
|                       | AYP definition is     |                       |
|                       | integrated into the   |                       |
|                       | State Accountability  |                       |
|                       | System.               |                       |
+-----------------------+-----------------------+-----------------------+
| **STATE RESPONSE AND  |                       |                       |
| STATE ACTIVITIES FOR  |                       |                       |
| MEETING               |                       |                       |
| REQUIREMENTS**        |                       |                       |
+-----------------------+-----------------------+-----------------------+

The State of North Dakota stipulates that all public schools and LEAs
will be judged systematically on the basis of the same criteria when
making AYP determination. The State will adopt the definition of AYP as
set forth within ESEA section 1111.

All schools and LEAs will be measured for AYP in terms of their
demonstrated achievement of each of the following criteria:

-   A school's or LEA's aggregate proficiency in both mathematics and
    reading, determined independently;

-   A school's or LEA's proficiency, determined on the disaggregated
    achievement results for each subgroup (ethnicity, disability,
    limited English proficient, and economic disadvantaged), determined
    independently;

-   A school's or LEA's aggregate participation rate that equals or
    exceeds 95%;

-   A school's or LEA's disaggregated participation rate that equals or
    exceeds 95% within each subgroup, determined independently;

-   A secondary school's or LEA's achievement of the required graduation
    rate;

-   An elementary or middle school's or LEA's achievement of the
    required attendance rate;

-   A school's or LEA's achievement of Safe Harbor in the aggregate or
    disaggregated for each subgroup, determined independently.

-   The rules of statistical reliability apply to all independent
    measures of AYP. Any application of statistical reliability for safe
    harbor is contingent on the study of the effects of the binomial
    distribution on safe harbor and an agreement between the U.S.
    Department of Education and the North Dakota Department of Public
    Instruction. Refer to section 9.1 for an overview of this study.

Each criteria stated above will apply to all public schools and LEAs,
without exception.

*The incorporation of federal guidance regarding the election of a local
school district to determine adequate yearly progress for a Title I
targeted assistance school based solely on the achievement of students
who are served by Title I or who are eligible to receive Title I
service.*

On February 4, 2004, the U.S. Department of Education issued guidance
regarding the determination of adequate yearly progress under Section
1116(b)(1)(D) in schools with a targeted assistance Title I program.
This provision is a restricted privilege accorded to Title I targeted
assistance schools. The following provisions apply.

a\. *Minimum group size and subgroup accountability.* The authority to
invoke the provisions of Section 1116(b)(1)(D) of the Act may be
exercised only if the number of students served or eligible to be served
by Title I services meets the minimum group size of ten students as
provided within Sections 5.5 and 5.6 of the North Dakota Accountability
Workbook. If, among the targeted assistance students, there are
sufficient numbers of students in various subgroups to meet the minimum
group size, subgroup accountability will be required. As in other
schools, students who are members of subgroups that fall below the
minimum group size at the school level will be included in the
appropriate subgroups for district and State adequate yearly progress
calculations.

b\. *Separate decisions for reading and mathematics.* In the case of a
targeted assistance school where Title I services are offered in only
one subject, the school will still be held accountable for both
reading/language arts and mathematics. An LEA will have the flexibility
to calculate adequate yearly progress based on either of the following
conditions:

-   The reading and mathematics assessment scores/participation rates of
    only students served or who are eligible for Title I services; or

-   The assessment scores/participation rates of only students served or
    eligible for Title I services in the subject in which Title I
    services are provided and of all students in the school for the
    subject in which Title I services are not provided.

c\. *Other academic indicators.* The LEA may examine the data for the
entire school or only those students receiving or eligible to receive
Title I services to determine if adequate yearly progress was made for
the other indicator. When adequate yearly progress is being calculated
based upon only the students receiving or eligible for Title I services,
those same students would be used to calculate progress on the other
indicator if the LEA wants to take advantage of the safe harbor
provisions. If data for the other indicators (i.e., graduation and
attendance) cannot be collected on those students receiving or eligible
for school regardless of student participation in Title I services, the
State will calculate adequate yearly progress based on all students in
the school regardless of student participation in Title I services.

d\. *Including results at the district and state level.* When
calculating adequate yearly progress for a LEA, the State will review
the progress of all students attending schools without Title I programs
and with schoolwide Title I programs; in schools operating targeted
assistance programs, the State may review the progress of only the
students who are served or eligible for Title I services or may review
the progress of all students in the school. The calculation of adequate
yearly progress decisions for LEAs will be conducted consistent with the
USDE guidance of February 4, 2004. The progress of all students,
regardless of eligibility for Title I services, will be reviewed in
adequate yearly progress calculations for the State. All students must
participate in the assessment at the school level, even if the school
only uses results from students receiving or eligible for Title I
services for accountability purposes.

e\. *Adequate yearly progress for schools without tested grades.* When
Title I services are provided in grades that are not assessed, an LEA
has several options for determining adequate yearly progress:

-   Back-mapping the performance of students in the grade assessed to
    the grades in which services are provided; or

-   Forward-mapping by examining the achievement of students who
    received services once those students are in the grades where
    assessments are given; or

-   Examining the achievement of the students who would have been
    eligible for Title I services in the grades assessed.

The Department stipulates that it will actively inform schools of their
opportunity to invoke the privilege of this provision, draft procedures
regarding the administration of this privilege, require schools to
submit a written request indicating their desire to exercise this
privilege, provide technical assistance and guidance to each school
regarding the administration of this privilege, monitor for the correct
application of this privilege according to the guidelines set forth
within the February 4, 2004 guidance, and maintain records concerning
the use of this privilege. Furthermore, the Department commits itself to
develop clear procedures and guidance to schools that encourages and
simplifies the determination of adequate yearly progress according to
this privilege.

+-----------------------+-----------------------+-----------------------+
| **CRITICAL ELEMENT**  | **EXAMPLES FOR**      | **EXAMPLES OF**       |
|                       |                       |                       |
|                       | **MEETING STATUTORY   | ***NOT* MEETING       |
|                       | REQUIREMENTS**        | REQUIREMENTS**        |
+-----------------------+-----------------------+-----------------------+
| 2.  Does the State    | State has defined     | Standards do not meet |
|     have, at a        | three levels of       | the legislated        |
|     minimum, a        | student achievement:  | requirements.         |
|     definition of     | *basic*, *proficient* |                       |
|     *basic*,          | and *advanced*.[^1]   |                       |
|     *proficient* and  |                       |                       |
|     *advanced*        | Student achievement   |                       |
|     student           | levels of             |                       |
|     achievement       | *proficient* and      |                       |
|     levels in         | *advanced* determine  |                       |
|     reading/language  | how well students are |                       |
|     arts and          | mastering the         |                       |
|     mathematics?      | materials in the      |                       |
|                       | State's academic      |                       |
|                       | content standards;    |                       |
|                       | and the *basic* level |                       |
|                       | of achievement        |                       |
|                       | provides complete     |                       |
|                       | information about the |                       |
|                       | progress of           |                       |
|                       | lower-achieving       |                       |
|                       | students toward       |                       |
|                       | mastering the         |                       |
|                       | *proficient* and      |                       |
|                       | *advanced* levels.    |                       |
+-----------------------+-----------------------+-----------------------+
| **STATE RESPONSE AND  |                       |                       |
| STATE ACTIVITIES FOR  |                       |                       |
| MEETING               |                       |                       |
| REQUIREMENTS**        |                       |                       |
+-----------------------+-----------------------+-----------------------+

The State of North Dakota has established achievement standards in
reading and mathematics, based on four distinct levels of student
achievement: novice, partially proficient, proficient, and advanced.

The State of North Dakota has developed and adopted academic achievement
standards in mathematics (refer to **Appendix E: North Dakota
Mathematics Achievement Standards**, at the following web site,
<http://www.dpi.state.nd.us/standard/perform/index.shtm>) and English
language arts (refer to **Appendix G: North Dakota English Language Arts
Achievement Standards**, at the following web site,
<http://www.dpi.state.nd.us/standard/perform/index.shtm>). These State
achievement standards have been developed at grades 4, 8, and 12 in
accordance with North Dakota's content and achievement standards
protocols (<http://www.dpi.state.nd.us/standard/content/toc.pdf>). North
Dakota mathematics and English language arts academic achievement
standards meet the requirements of section 1111(b)(1). North Dakota's
achievement standards in science will be completed by the winter 2003,
in accordance with State standards development protocols and section
1111(b)(1) requirements.

North Dakota will continue to use adopted achievement standards as the
basis for statewide assessments at grades 4, 8, and 12 in accordance
with section 1111(b)(1). In addition, North Dakota will expand its
statewide assessments into grades 3, 5, 6, and 7 in mathematics and
reading, in accordance with section 1111(b)(1) by 2005-06, based on
State-defined, grade-level achievement standards. Additionally, North
Dakota will expand its statewide assessments, voluntarily, based on the
availability of ESEA Title VI funding, into grades 3, 5, 6, and 7 in
science by 2007-2008, based on State-defined, achievement standards. All
achievement standards at grades 3, 5, 6, and 7 will be aligned with
North Dakota's corresponding grade-level content standards. These
achievement standards will be developed and adopted in accordance with
North Dakota's standards development protocols.

North Dakota proposes to develop narrative achievement standards at
grades 3, 5, 6, and 7 contemporaneously with the development of
grade-level content standards. The content expectation committees will
also draft the narrative achievement standards. These narrative
achievement standards will act as the primary calibration tool for the
cut-point standards setting performed to align the State assessment
scale scores to State achievement standards.

The State of North Dakota will only recognize and reference student
achievement proficiency ratings generated by the North Dakota State
Assessment and its Alternate Assessment. No other student achievement
assessment tools or means will be recognized. No local assessments will
be recognized as an alternative to the North Dakota State Assessment.

The only definitions of proficiency levels recognized by the State AYP
accountability system are those proficiency levels set for the North
Dakota State Assessment through the standards-setting process. Refer to
**Appendix J: North Dakota State Assessment, Bookmark Standards Setting
Technical Report, 2002** for the established definitions of the North
Dakota proficiency levels. The State cut scores for the North Dakota
State Assessment will constitute the defining scales for identifying
schools and districts for AYP. Refer to **Appendix K: State
Superintendent's Approval Notification of North Dakota State Assessment
Cut Scores** for the State's announced policy regarding the
establishment of proficiency level cut scores.

+-----------------------+-----------------------+-----------------------+
| **CRITICAL ELEMENT**  | **EXAMPLES FOR**      | **EXAMPLES OF**       |
|                       |                       |                       |
|                       | **MEETING STATUTORY   | ***NOT* MEETING       |
|                       | REQUIREMENTS**        | REQUIREMENTS**        |
+-----------------------+-----------------------+-----------------------+
| 4.  How does the      | > State provides      | Timeline does not     |
|     State provide     | > decisions about     | provide sufficient    |
|     accountability    | > adequate yearly     | time for LEAs to      |
|     and adequate      | > progress in time    | fulfill their         |
|     yearly progress   | > for LEAs to         | responsibilities      |
|     decisions and     | > implement the       | before the beginning  |
|     information in a  | > required provisions | of the next academic  |
|     timely manner?    | > before the          | year.                 |
|                       | > beginning of the    |                       |
|                       | > next academic year. |                       |
|                       |                       |                       |
|                       | State allows enough   |                       |
|                       | time to notify        |                       |
|                       | parents about public  |                       |
|                       | school choice or      |                       |
|                       | supplemental          |                       |
|                       | educational service   |                       |
|                       | options, time for     |                       |
|                       | parents to make an    |                       |
|                       | informed decision,    |                       |
|                       | and time to implement |                       |
|                       | public school choice  |                       |
|                       | and supplemental      |                       |
|                       | educational services. |                       |
+-----------------------+-----------------------+-----------------------+
| **STATE RESPONSE AND  |                       |                       |
| STATE ACTIVITIES FOR  |                       |                       |
| MEETING               |                       |                       |
| REQUIREMENTS**        |                       |                       |
+-----------------------+-----------------------+-----------------------+

During the 2001-02 school year and pursuant to the State's Assessment
Waiver Plan approved by the U.S. Department of Education, the State
received its baseline assessment results from its assessment vendor
during the fall, 2002. The Department of Public Instruction used these
baseline assessment results to conduct the 2001-02 AYP review of each
school and LEA in February 2003.

The State released its initial AYP reports in February 2003. The State
provides technical assistance on programmatic issues related to AYP
reports to LEAs and schools. In the spring of each school year, the
Department of Public Instruction conducts a workshop for all schools
identified as not achieving AYP. At this workshop, schools are provided
with a timeline of required activities and information on implementing
all required AYP provisions. Schools are informed of their
responsibilities on parent notification, school choice, supplemental
services, and other corrective actions sanctions, and are given guidance
on writing a school improvement plan. The schools prepare and implement
these requirements before the beginning of the next academic school
year. Additionally, schools receive ongoing guidance throughout the
school year including informative memos on required procedures, example
forms and ideas for implementation. The Title I website for program
improvement also contains the information distributed at the spring
workshop to help schools as they implement required provisions before
and during the school year. Refer to **Appendix L: Program Improvement
Activities** at the following website:
[www.dpi.state.nd.us/title1/progress/index.shtm](http://www.dpi.state.nd.us/title1/progress/index.shtm).

For the 2002-03, and 2003-04 school years, all final assessment scores
will be made available to the State from the State's assessment
contractor by June of each respective year. It is anticipated that the
State will be in a position to conduct its AYP determination and report
dissemination by late July of each respective year. Schools will receive
their AYP status reports during the summer of each respective year. This
notification will arrive in time for schools and LEAs to notify, in
turn, parents regarding their right to seek a supplemental service,
travel service, or school choice option under program improvement with
ESEA section 1116.

Beginning during the 2004-05 school year and for every school year
thereafter, the State will conduct fall assessments that will ensure the
State's ability to conduct ongoing achievement cut-score analyses and
AYP determinations well in advance of the end of a given school year.
The advancement of a fall assessment schedule is designed to improve the
quality of cut-score analyses, the generation and dissemination of
reports, the timely notification of schools and LEAs, the more conducive
turn-around time for school- and LEA-reporting to parents, and the more
relaxed deliberation of parents in determining their parental rights
options identified within ESEA, section 1116.

+-----------------------+-----------------------+-----------------------+
| **CRITICAL ELEMENT**  | **EXAMPLES FOR**      | **EXAMPLES OF**       |
|                       |                       |                       |
|                       | **MEETING STATUTORY   | ***NOT* MEETING       |
|                       | REQUIREMENTS**        | REQUIREMENTS**        |
+-----------------------+-----------------------+-----------------------+
| 5.  Does the State    | The State Report Card | The State Report Card |
|     Accountability    | includes all the      | does not include all  |
|     System produce an | required data         | the required data     |
|     annual State      | elements \[see        | elements.             |
|     Report Card?      | Appendix A for the    |                       |
|                       | list of required data | The State Report Card |
|                       | elements\].           | is not available to   |
|                       |                       | the public.           |
|                       | The State Report Card |                       |
|                       | is available to the   |                       |
|                       | public at the         |                       |
|                       | beginning of the      |                       |
|                       | academic year.        |                       |
|                       |                       |                       |
|                       | The State Report Card |                       |
|                       | is accessible in      |                       |
|                       | languages of major    |                       |
|                       | populations in the    |                       |
|                       | State, to the extent  |                       |
|                       | possible.             |                       |
|                       |                       |                       |
|                       | Assessment results    |                       |
|                       | and other academic    |                       |
|                       | indicators (including |                       |
|                       | graduation rates) are |                       |
|                       | reported by student   |                       |
|                       | subgroups             |                       |
+-----------------------+-----------------------+-----------------------+
| **STATE RESPONSE AND  |                       |                       |
| STATE ACTIVITIES FOR  |                       |                       |
| MEETING               |                       |                       |
| REQUIREMENTS**        |                       |                       |
+-----------------------+-----------------------+-----------------------+

The State stipulates that it will produce and disseminate a State Report
Card and Profile for the state as a whole, for each LEA, and for each
public school to meet all accountability requirements specified within
ESEA section 1111. The State Report Card and Profile will publish all
aggregate student achievement data, all disaggregate student achievement
data by subgroup, graduation rates, attendance rates, participation
rates, and AYP status for the State, each LEA, and each school
respectively.

# State law requires the dissemination of individual student assessment reports to parents and schools in an understandable format. State law also requires the presentation of State assessment results to the Legislative Council summarizing overall student achievement. Further, State law requires that aggregated and disaggregated student achievement results be published for the review of the public. Refer to Appendix B: North Dakota Century Code, Assessment Statutes for a summary of State's reporting requirements. Refer to Appendix M: Testimony Before the Education Committee by the Department of Public Instruction, October 10, 2002 for a summary outline of the testimony delivered to the Legislative Council's Interim Education Committee on October 10, 2002.

The State's assessment contractor scores, prints, packages and ships all
student achievement reports to the respective schools. Teachers are
instructed to review the results of each student's assessment with each
student and subsequently with each student's parents. Teachers are
instructed to review a student's performance at the subject level, the
standards level, and at the benchmark level. Further, teachers are
instructed to clarify the meaning of the State content standards and
achievement standards. The back-side of all reports offers a summary of
these standards and identifies a web site for a more detailed
presentation of the State's standards. Refer to Appendix N: North Dakota
State Assessment, Student Achievement Reports for illustrations of the
various achievement reports.

The Department of Public Instruction receives all student achievement
data for each school and district from the State's assessment contractor
through a comprehensive data transfer. The Department compiles the data,
identifies and corrects any inconsistencies, generates disaggregated
reports according to defined subgroup populations, and prints summative
reports for each school, each district, and the State. The results of
these reports are forwarded to each school and district. These results
are also listed on the State Report Card and School Profile of the
Department's web site. Refer to Appendix O: North Dakota State Report
Card and Profile for an illustration of the content. The 2001-02 State
Report Card and Profile and its web site are under development and will
be completed in early February 2002.

The State will produce all district- and school-level reports for the
districts and schools. These reports will include both aggregated and
disaggregated student achievement data. Refer to Appendix O: North
Dakota School Report Card and Profile for an illustration of the content
of these student achievement profiles. Districts may use these reports
as the foundation for their locally produced report cards and profiles.
These State-generated reports will offer quality assurances regarding
the generation of any district achievement data.

All public information will be disseminated through the public media, as
described below.

\(1\) The Department of Public Instruction will present an annual report
to the North Dakota Legislative Council as required by law (refer **to
Appendix B: North Dakota Century Code, Assessment Statutes** for an
overview of State statutes on public disclosure of State Assessment
results, and **Appendix M: Testimony Before the Education Committee by
the Department of Public Instruction, October 10, 2002** for the 2002
presentation to the Legislative Council).

\(2\) The Department of Public Instruction will publish press releases
for use by radio/television, the print media, and other publication
media. The content for these press releases will reflect the school
profile and report card. Refer to **Appendix O: North Dakota School
Report Card and Profile** for an illustration of this content. Refer to
**Appendix P: State Superintendent's Release of State Assessment
Results** for the November 2002 press release on the 2001-02 school year
achievement data, submitted by the State Superintendent.

\(3\) The Department of Public Instruction will publish the school
report card and profile electronically through the Department's website,
Refer to **Appendix O: North Dakota School Report Card and Profile** or
refer to the following web site,
<http://www.dpi.state.nd.us/dpi/reports/profile/0102/50128.htm>

> This public information process supplements the Department's
> communication to parents regarding standards and assessment.
>
> The State will publish all aggregated and disaggregated student
> achievement data by school, district, and the State on the Department
> of Public Instruction web site. This publication will allow school
> districts to access information on their district and other districts
> for use in general school improvement activities. Refer to **Appendix
> T: North Dakota Sample School Report Card and Profile** or refer to
> the following web site,
> <http://www.dpi.state.nd.us/dpi/reports/profile/0102/50128.htm>.

Parents will have access to the information through their students'
individual achievement reports, the Department of Public Instruction
website, the dissemination of their district's local school report card
and profile, and other forms of public documents. The Department of
Public Instruction will analyze data and review policies on a regular
basis in order to assure that data are used to advance school
improvement plans.

+-----------------------+-----------------------+-----------------------+
| **CRITICAL ELEMENT**  | **EXAMPLES FOR**      | **EXAMPLES OF**       |
|                       |                       |                       |
|                       | **MEETING STATUTORY   | ***NOT* MEETING       |
|                       | REQUIREMENTS**        | REQUIREMENTS**        |
+-----------------------+-----------------------+-----------------------+
| 6.  How does the      | State uses one or     | State does not        |
|     State             | more types of rewards | implement rewards or  |
|     Accountability    | and sanctions, where  | sanctions for public  |
|     System include    | the criteria are:     | schools and LEAs      |
|     rewards and       |                       | based on adequate     |
|     sanctions for     | -   Set by the State; | yearly progress.      |
|     public schools    |                       |                       |
|     and LEAs?[^2]     | -   Based on adequate |                       |
|                       |     yearly progress   |                       |
|                       |     decisions; and,   |                       |
|                       |                       |                       |
|                       | -   Applied uniformly |                       |
|                       |     across public     |                       |
|                       |     schools and LEAs. |                       |
+-----------------------+-----------------------+-----------------------+
| **STATE RESPONSE AND  |                       |                       |
| STATE ACTIVITIES FOR  |                       |                       |
| MEETING               |                       |                       |
| REQUIREMENTS**        |                       |                       |
+-----------------------+-----------------------+-----------------------+

*Sanctions and Rewards Based on Adequate Yearly Progress.*

The state of North Dakota has established an accountability system that
is based on the state's definition of adequate yearly progress and is
applied uniformly across all public schools and districts in the state.
All schools and districts are held to the same standards. All schools
and districts will receive written notification of whether they are
satisfactory in making adequate yearly progress. However, the state does
not hold schools and districts not receiving Title I funds to the
requirements of Section 1116 of the *No Child Left Behind* Act.

The state of North Dakota is in the process of revising the previous
system of rewards and sanctions to align with the *No Child Left Behind*
Act. The Department of Public Instruction has always had a system of
rewards and sanctions in place. The previous and new system for rewards
and sanctions for public schools and LEAs in North Dakota is based
solely on a district's or school's adequate yearly progress status.

The state's previous system of rewards was based on a state assessment
that measured student progress using national percentiles. Schools that
scored above the 65 percentile for 3 consecutive years were identified
as Title I distinguished schools. These districts and schools were
recognized and served as models under the statewide school support
system.

As of 2001-2002, the state assessment measures student progress against
our North Dakota state standards in reading and mathematics. North
Dakota teachers, under the direction of our state assessment contractor,
went through a standard setting process and identified cut scores for
proficiency on the state assessment. Schools that meet or exceed the
standard are declared satisfactory in making adequate yearly progress.

*System of Sanctions*

The state's previous system of sanctions remains intact. Schools that
were in program improvement status in the old law remained in the same
category after the *No Child Left Behind* Act was enacted. North Dakota
currently has twenty-three schools identified for program improvement.
Twenty-one of the schools are currently in the fourth year of program
improvement. Two schools are in their third year of program improvement.
All twenty-three schools have submitted a program improvement plan that
is currently being peer reviewed against established rubrics that assess
the quality of the plans. All twenty-three schools have notified parents
and community members of their program improvement identification and
the appropriate parent options available to them. Annual workshops for
schools identified for program improvement were held during April 2002
and April 2003. School personnel were apprised of the new regulations in
the *No Child Left Behind* Act.

The school choice and supplemental service provisions are currently
being implemented for the twenty-three schools in program improvement
status. The state of North Dakota created a supplemental service
application and went through a request for proposal process in August
2002. This process resulted in two supplemental service providers being
approved to offer supplemental services. In December 2002, the
Department of Public Instruction went through a second request for
proposals process. A second round of supplemental service providers was
announced in March 2003.

Current North Dakota law allows for open enrollment so the choice
provision can be implemented in districts with more than one school per
grade span. State law authorizes a process for the SEA to take at least
one of the actions against LEAs in corrective action, listed in the NCLB
legislation. In April 2003 the North Dakota Legislative Assembly enacted
law to authorize open enrollment across school district lines. This
provision effectively authorizes the Department to implement at least
one of four alternative governance options, or another option that leads
to "major restructuring" to improve student achievement for schools in
year seven of the program improvement timeline.

*System of Rewards.*

The State of North Dakota is developing a system of rewards that
includes distinguished schools designations or financial rewards. These
strategies will recognize schools that have significantly closed the
achievement gap, exceeded adequate yearly progress, or have made the
greatest gains in student performance. The Department of Public
Instruction is working with various advisory groups, including the
Committee of Practitioners, Title I School Support Team, the Standards
Assessment and Learning Team, and the State's Assessment and
Accountability Technical Advisory Committee to develop criteria on what
constitutes a significant gain for recognition awards and financial
rewards.

All districts and schools in the state that significantly exceed the
adequate yearly progress expectations for any given year will be
recognized as a distinguished school. Distinguished schools and
districts will receive certificates of distinction and receive public
recognition of this distinguished status.

Title I districts and schools that have significantly closed the
achievement gap or have made the greatest gains in student performance
in achieving status or improvement will be eligible for financial
rewards. Financial rewards will vary and most likely be limited
depending on the number of eligible schools. The Department of Public
Instruction will conduct an annual review of the established
distribution method in order to optimize the financial impact to
schools. This annual review will be conducted based on the
recommendations of the Committee of Practitioners, School Support Team,
SALT Team, and the State Technical Advisory Committee. The State will
reward those districts and schools that have achieved the upper 10% to
25% of ranked achievement gains, allowing for statistical significance.
The State Superintendent will review, amend, and approve the annual
distribution formula to meet the available funds and established
criteria.

**PRINCIPLE 2. All students are included in the State Accountability
System.**

+-----------------------+-----------------------+-----------------------+
| **CRITICAL ELEMENT**  | **EXAMPLES FOR**      | **EXAMPLES OF**       |
|                       |                       |                       |
|                       | **MEETING STATUTORY   | ***NOT* MEETING       |
|                       | REQUIREMENTS**        | REQUIREMENTS**        |
+-----------------------+-----------------------+-----------------------+
| 1.  How does the      | All students in the   | Public school         |
|     State             | State are included in | students exist in the |
|     Accountability    | the State             | State for whom the    |
|     System include    | Accountability        | State Accountability  |
|     all students in   | System.               | System makes no       |
|     the State?        |                       | provision.            |
|                       | The definitions of    |                       |
|                       | "public school" and   |                       |
|                       | "LEA" account for all |                       |
|                       | students enrolled in  |                       |
|                       | the public school     |                       |
|                       | district, regardless  |                       |
|                       | of program or type of |                       |
|                       | public school.        |                       |
+-----------------------+-----------------------+-----------------------+
| **STATE RESPONSE AND  |                       |                       |
| STATE ACTIVITIES FOR  |                       |                       |
| MEETING               |                       |                       |
| REQUIREMENTS**        |                       |                       |
+-----------------------+-----------------------+-----------------------+

The North Dakota Assessment System assesses *all* students, regardless
of status, within a single, unified, statewide assessment that measures
students' performance in terms of the State's challenging content and
achievement standards.

North Dakota state law (NDCC 15.1-21-08) places responsibility with the
State Superintendent for the administration of State assessments to all
public school students that are aligned to the State's content standards
in reading and mathematics (refer to **Appendix B: North Dakota Century
Code citations** or reference the North Dakota Century Code at the
following web site,
[http://www.state.nd.us/lr/cencode/T151C21.pdf](http://www.state.nd.us/lr/assembly/57-2001/cencode/CCT15x1.pdf)
). State law requires that the assessments be administered to at least
one grade level selected within each of the following grade spans:
grades three through five; grades six through nine; and grades ten
through twelve. The North Dakota Department of Public Instruction has
developed and administers assessments at grades 4, 8, and 12 to
correspond with the State's content standards.

State law provides for the assessment of all students within the
designated grade levels. Therefore, all students are to be included
within the State assessment and accountability system. No exceptions or
systematic exemptions to the State assessment and accountability system
are allowed.

+-----------------------+-----------------------+-----------------------+
| **CRITICAL ELEMENT**  | **EXAMPLES FOR**      | **EXAMPLES OF**       |
|                       |                       |                       |
|                       | **MEETING STATUTORY   | ***NOT* MEETING       |
|                       | REQUIREMENTS**        | REQUIREMENTS**        |
+-----------------------+-----------------------+-----------------------+
| 2.  How does the      | The State has a       | LEAs have varying     |
|     State define      | definition of "full   | definitions of "full  |
|     "full academic    | academic year" for    | academic year."       |
|     year" for         | determining which     |                       |
|     identifying       | students are to be    | The State's           |
|     students in AYP   | included in decisions | definition excludes   |
|     decisions?        | about AYP.            | students who must     |
|                       |                       | transfer from one     |
|                       | The definition of     | district to another   |
|                       | full academic year is | as they advance to    |
|                       | consistent and        | the next grade.       |
|                       | applied statewide.    |                       |
|                       |                       | The definition of     |
|                       |                       | full academic year is |
|                       |                       | not applied           |
|                       |                       | consistently.         |
+-----------------------+-----------------------+-----------------------+
| **STATE RESPONSE AND  |                       |                       |
| STATE ACTIVITIES FOR  |                       |                       |
| MEETING               |                       |                       |
| REQUIREMENTS**        |                       |                       |
+-----------------------+-----------------------+-----------------------+

The State of North Dakota requires all students enrolled in public
schools within North Dakota to participate in the State Assessment
system. Refer to **Appendix B: North Dakota Century Code** citations or
reference the North Dakota Century Code at the following web site,
[http://www.state.nd.us/lr/cencode/T151C21.pdf](http://www.state.nd.us/lr/assembly/57-2001/cencode/CCT15x1.pdf)
). All students, regardless of their enrollment status, participate in
the State Assessment. This total inclusion policy includes those
students who may have enrolled in a district or school after the
beginning of a school year.

For the purpose of identifying students whose achievement results are to
be included within a school's or LEA's AYP determination, a student must
be in school for the full academic year. A 'full academic year means a
student has been enrolled at a school or within a LEA for a period equal
to or exceeding 173 instructional days, as provided by State statute and
recorded within the school's pupil membership file on the State's Online
Reporting System.

Any student who may have been enrolled in a school or district after the
beginning of a school year is identified on their assessment demographic
sheet. Students or school personnel mark a special code on the
assessment demographic sheet that identifies their late enrollment
status. This code identifies the student and to remove them from the
school's student roll for AYP identification purposes. Refer to page 29
for codes "R" and "S" of **Appendix Q: North Dakota State Assessment,
Test Coordinator's Manual 2002-03** for the enrollment code
identification fields. A student who has not been enrolled in a school
for the entire year but has been enrolled in the district for the entire
year will be included into AYP consideration for the school in which the
assessment is administered and will be included into AYP consideration
for the district.

All students must be accounted for regarding their enrollment status.
This is a required entry on the demographic sheet of all students.
Student participation rates will be compared to a school's and
district's Average Daily Membership student count used to reimburse
schools and districts for their State foundation aid. Therefore, the
State references reimbursement census data to confirm student
participation rates. Refer to page 29 for codes "R" and "S" of Appendix
Q: North Dakota State Assessment, Test Coordinator's Manual 2002-03 for
the enrollment code identification fields. Student participation rates
will be compared to the schools and districts Average Daily Membership
student count used to reimburse school's and district's for their State
foundation aid. Therefore, the State references reimbursement census
data to confirm student participation rates.

The State is developing a statewide student data analysis and reporting
system to aid the State in monitoring the enrollment patterns and
participation rates of students. The TetraData application will allow
the State to link district enrollment files with the State's assessment
participation files in order to assure that all enrolled students are
accounted for in the State Assessment system files. Refer to **Appendix
R: TetraData Data Analysis and Reporting System Summary** for an
overview of the TetraData system's purpose and design.

The State currently is reviewing its ESEA and accreditation monitoring
policies. The State is pursuing an amendment to its monitoring
requirements that would mandate districts to produce evidence regarding
the enrollment dates of all students. Monitors would check for any
students who had enrolled after the beginning of a school year and
cross-check their participation status in the State Assessment data
file. Any failures to include such students would be identified as a
compliance violation of the school's and district's ESEA compliance
agreement. Refer to **Appendix S: Consolidated Application Certification
and Assurances** for the State Assessment requirement for receipt of
federal ESEA funding. A school or district may be sanctioned for any
compliance violation of their ESEA assurances agreement.

It is the expressed policy of the State of North Dakota to include all
students within the North Dakota State Assessment.

+-----------------------+-----------------------+-----------------------+
| **CRITICAL ELEMENT**  | **EXAMPLES FOR**      | **EXAMPLES OF**       |
|                       |                       |                       |
|                       | **MEETING STATUTORY   | ***NOT* MEETING       |
|                       | REQUIREMENTS**        | REQUIREMENTS**        |
+-----------------------+-----------------------+-----------------------+
| 3.  How does the      | State holds public    | State definition      |
|     State             | schools accountable   | requires students to  |
|     Accountability    | for students who were | attend the same       |
|     System determine  | enrolled at the same  | public school for     |
|     which students    | public school for a   | more than a full      |
|     have attended the | full academic year.   | academic year to be   |
|     same public       |                       | included in public    |
|     school and/or LEA | State holds LEAs      | school                |
|     for a full        | accountable for       | accountability.       |
|     academic year?    | students who transfer |                       |
|                       | during the full       | State definition      |
|                       | academic year from    | requires students to  |
|                       | one public school     | attend school in the  |
|                       | within the district   | same district for     |
|                       | to another public     | more than a full      |
|                       | school within the     | academic year to be   |
|                       | district.             | included in district  |
|                       |                       | accountability.       |
|                       |                       |                       |
|                       |                       | State holds public    |
|                       |                       | schools accountable   |
|                       |                       | for students who have |
|                       |                       | not attended the same |
|                       |                       | public school for a   |
|                       |                       | full academic year.   |
+-----------------------+-----------------------+-----------------------+
| **STATE RESPONSE AND  |                       |                       |
| STATE ACTIVITIES FOR  |                       |                       |
| MEETING               |                       |                       |
| REQUIREMENTS**        |                       |                       |
+-----------------------+-----------------------+-----------------------+

All students participating in the State assessment must be accounted for
regarding their enrollment status. This is a required entry on the
student demographic sheet of all students. Student participation rates
will be compared to the school's and district's Average Daily Membership
student count used to reimburse schools and districts for their State
foundation aid. Therefore, the State references reimbursement census
data to confirm student participation rates.

The State requires all schools to account for all students regarding
their enrollment status within the school and district and their
inclusion within the State Assessment. The enrollment status of each
student is a required entry on the demographic sheet of all students.
Refer to page 29 for codes "R" and "S" of **Appendix Q: North Dakota
State Assessment, Test Coordinator's Manual 2002-03** for the enrollment
code identification fields. Student participation rates will be compared
to the school's and district's Average Daily Membership student count
used to reimburse school's and district's for their State foundation
aid. Therefore, the State references reimbursement census data to
confirm student participation rates.

The State is developing a statewide student data analysis and reporting
system to aid the State in monitoring the enrollment patterns and
participation rates of students. The TetraData application will allow
the State to link district enrollment files with the State's assessment
participation files in order to assure that all enrolled students are
accounted for in the State Assessment system files. Refer to **Appendix
R: TetraData Data Analysis and Reporting System Summary** for an
overview of the TetraData system's purpose and design.

The State currently is reviewing its ESEA and accreditation monitoring
policies. The State is pursuing an amendment to its monitoring
requirements that would mandate districts to produce evidence regarding
the enrollment dates of all students. Monitors would check for any
students who had enrolled after the beginning of a school year and
cross-check their participation status in the State Assessment data
file. Any failures to include such students would be identified as a
compliance violation of the school's and district's ESEA compliance
agreement. Refer to **Appendix S: Consolidated Application Certification
and Assurances** for the State assessment requirement for receipt of
federal ESEA funding. A school or district may be sanctioned for any
compliance violation of their ESEA assurances agreement.

It is the expressed policy of the State of North Dakota to include all
students within the North Dakota State Assessment.

**PRINCIPLE 3. State definition of AYP is based on expectations for
growth in student achievement that is continuous and substantial, such
that all students are proficient in reading/language arts and
mathematics no later than 2013-2014.**

+----------------------+-----------------------+-----------------------+
| ## CRITICAL ELEMENT  | **EXAMPLES FOR\       | **EXAMPLES OF**       |
|                      | MEETING               |                       |
|                      | REQUIREMENTS**        | ***NOT* MEETING       |
|                      |                       | REQUIREMENTS**        |
+======================+=======================+=======================+
| 1.  How does the     | The State has a       | State definition does |
|     State's          | timeline for ensuring | not require all       |
|     definition of    | that all students     | students to achieve   |
|     adequate yearly  | will meet or exceed   | proficiency by        |
|     progress require | the State's           | 2013-2014.            |
|     all students to  | proficient level of   |                       |
|     be proficient in | academic achievement  | State extends the     |
|     reading/language | in reading/language   | timeline past the     |
|     arts and         | arts[^3] and          | 2013-2014 academic    |
|     mathematics by   | mathematics, not      | year.                 |
|     the 2013-2014    | later than 2013-2014. |                       |
|     academic year?   |                       |                       |
+----------------------+-----------------------+-----------------------+
| **STATE RESPONSE AND |                       |                       |
| STATE ACTIVITIES FOR |                       |                       |
| MEETING              |                       |                       |
| REQUIREMENTS**       |                       |                       |
+----------------------+-----------------------+-----------------------+

The State of North Dakota stipulates that its State Accountability
System provides for an established timeline that ensures that all
students will be proficient in reading and mathematics by the 2013-14
academic year, as specified within ESEA section 1111.

The State of North Dakota has developed and adopted academic achievement
standards in mathematics (refer to **Appendix E: North Dakota
Mathematics Achievement Standards** at the following web site,
<http://www.dpi.state.nd.us/standard/perform/index.shtm>) and English
language arts (refer to **Appendix G: North Dakota English Language Arts
Achievement Standards** at the following web site,
(<http://www.dpi.state.nd.us/standard/perform/index.shtm>). These State
achievement standards have been developed at grades 4, 8, and 12 in
accordance with North Dakota's content and achievement standards
protocols (<http://www.dpi.state.nd.us/standard/content/toc.pdf>). North
Dakota mathematics and English language arts academic achievement
standards meet the requirements of section 1111(b)(1). North Dakota's
achievement standards in science will be completed by the winter 2003,
in accordance with State standards development protocols and section
1111(b)(1) requirements.

It is the policy of the State that all students achieve proficiency as
defined within the State's challenging achievement standards by the
2013-14 academic year.

For the purposes of determining AYP, proficiency means the aggregation
of all student achievement within the "proficient" and "advanced"
performance levels of the State's achievement standards. Schools and
LEAs must evidence a steady improvement of student achievement from the
below-proficient performance level (the aggregate of the novice and
partially-proficient performance level) to the proficient performance
level.

+----------------------+-----------------------+-----------------------+
| ## CRITICAL ELEMENT  | **EXAMPLES FOR\       | **EXAMPLES OF**       |
|                      | MEETING               |                       |
|                      | REQUIREMENTS**        | ***NOT* MEETING       |
|                      |                       | REQUIREMENTS**        |
+======================+=======================+=======================+
| 2.  How does the     | For a public school   | State uses different  |
|     State            | and LEA to make       | method for            |
|     Accountability   | adequate yearly       | calculating how       |
|     System determine | progress, each        | public schools and    |
|     whether each     | student subgroup must | LEAs make AYP.        |
|     student          | meet or exceed the    |                       |
|     subgroup, public | State annual          |                       |
|     school and LEA   | measurable            |                       |
|     makes AYP?       | objectives, each      |                       |
|                      | student subgroup must |                       |
|                      | have at least a 95%   |                       |
|                      | participation rate in |                       |
|                      | the statewide         |                       |
|                      | assessments, and the  |                       |
|                      | school must meet the  |                       |
|                      | State's requirement   |                       |
|                      | for other academic    |                       |
|                      | indicators.           |                       |
|                      |                       |                       |
|                      | However, if in any    |                       |
|                      | particular year the   |                       |
|                      | student subgroup does |                       |
|                      | not meet those annual |                       |
|                      | measurable            |                       |
|                      | objectives, the       |                       |
|                      | public school or LEA  |                       |
|                      | may be considered to  |                       |
|                      | have made AYP, if the |                       |
|                      | percentage of         |                       |
|                      | students in that      |                       |
|                      | group who did not     |                       |
|                      | meet or exceed the    |                       |
|                      | proficient level of   |                       |
|                      | academic achievement  |                       |
|                      | on the State          |                       |
|                      | assessments for that  |                       |
|                      | year decreased by 10% |                       |
|                      | of that percentage    |                       |
|                      | from the preceding    |                       |
|                      | public school year;   |                       |
|                      | that group made       |                       |
|                      | progress on one or    |                       |
|                      | more of the State's   |                       |
|                      | academic indicators;  |                       |
|                      | and that group had at |                       |
|                      | least 95%             |                       |
|                      | participation rate on |                       |
|                      | the statewide         |                       |
|                      | assessment.           |                       |
+----------------------+-----------------------+-----------------------+

  -----------------------------------------------------------------------
  **STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS**

  -----------------------------------------------------------------------

The State of North Dakota stipulates that the State will determine AYP
for each public school and LEA as provided for within ESEA section 1111,
including emphasis on the school identification method referenced to
proficiency ratings, safe harbor provisions, statistical reliability
assurances, graduation rates for secondary schools, attendance rates for
elementary and middle schools, and a minimum assessment participation
rate of 95%.

The State of North Dakota will only recognize and reference student
achievement proficiency ratings generated by the North Dakota State
Assessment and its Alternate Assessment. No other student achievement
assessment tools or means will be recognized. No local assessments will
be recognized as an alternative to the North Dakota State Assessment.

The only definitions of achievement levels recognized by the State AYP
accountability system are those proficiency levels set for the North
Dakota State Assessment through the standards-setting process. Refer to
**Appendix J: North Dakota State Assessment, Bookmark Standards Setting
Technical Report, 2002** for the established definitions of the North
Dakota achievement levels. The State cut scores for the North Dakota
State Assessment will constitute the defining scales for identifying
schools and districts for AYP. Refer to **Appendix K: State
Superintendent's Approval Notification of North Dakota State Assessment
Cut Scores** for the State's announced policy regarding the
establishment of achievement level cut scores.

All student achievement data collected during the administration of the
State Assessment will be used to aggregate overall student achievement
and to disaggregate student achievement results into each of the
required student subgroups to determine AYP. Refer to **Appendix X:
North Dakota Assessment System 2001-02 Baseline Impact Data** to review
the State's impact data.

AYP will be determined using 2001-02 school year data as the baseline.
The starting points are calculated pursuant to the prescribed ESEA
section 1111 requirements. The same starting point and annual,
measurable objectives apply to all student subgroups resulting in 100%
proficiency of all students by 2013-2014.

In calculating AYP for student aggregated and subgroup populations, the
State will employ a binomial distribution statistical model to ensure
high levels of reliability. Ninety-five percent of students, considering
statistical reliability, in each applicable student sub-group must be
tested in order for the school to make AYP.

In calculating AYP for any student subgroup that did not meet the AYP
goal but did decrease the percentage of students in the applicable
student sub-group by 10% or more, the school or district will then be
judged to have made AYP if the respective subgroup also meets the
state's other criteria when using the safe harbor provision (graduation
rate for high school and attendance rates for elementary and secondary).
Goals must be met for all applicable student subgroups in accordance
with 34 CFR 200.19(d)(2)(i). Following a study of the effects of
statistical reliability on safe harbor and with the concurrence of the
State and the U.S. Department of Education, the State will employ the
binomial distribution statistical method within the calculation of
safe-harbor status, including safe harbor for subgroups.

All students' scores will be used as an aggregate to determine the AYP
of schools as a whole. All schools' scores will be used as an aggregate
to determine the AYP of LEAs. All LEAs scores will be used as an
aggregate to determine the AYP of the State

Please refer to **Appendix T: State AYP Computation Rules** for the
working rules used to determine AYP.

+----------------------+-----------------------+-----------------------+
| ## CRITICAL ELEMENT  | **EXAMPLES FOR\       | **EXAMPLES OF**       |
|                      | MEETING               |                       |
|                      | REQUIREMENTS**        | ***NOT* MEETING       |
|                      |                       | REQUIREMENTS**        |
+======================+=======================+=======================+
| 3.2a What is the     | Using data from the   | The State             |
| State's starting     | 2001-2002 school      | Accountability System |
| point for            | year, the State       | uses a different      |
| calculating Adequate | established separate  | method for            |
| Yearly Progress?     | starting points in    | calculating the       |
|                      | reading/language arts | starting point (or    |
|                      | and mathematics for   | baseline data).       |
|                      | measuring the         |                       |
|                      | percentage of         |                       |
|                      | students meeting or   |                       |
|                      | exceeding the State's |                       |
|                      | proficient level of   |                       |
|                      | academic achievement. |                       |
|                      |                       |                       |
|                      | Each starting point   |                       |
|                      | is based, at a        |                       |
|                      | minimum, on the       |                       |
|                      | higher of the         |                       |
|                      | following percentages |                       |
|                      | of students at the    |                       |
|                      | proficient level: (1) |                       |
|                      | the percentage in the |                       |
|                      | State of proficient   |                       |
|                      | students in the       |                       |
|                      | lowest-achieving      |                       |
|                      | student subgroup; or, |                       |
|                      | (2) the percentage of |                       |
|                      | proficient students   |                       |
|                      | in a public school at |                       |
|                      | the 20^th^ percentile |                       |
|                      | of the State's total  |                       |
|                      | enrollment among all  |                       |
|                      | schools ranked by the |                       |
|                      | percentage of         |                       |
|                      | students at the       |                       |
|                      | proficient level.     |                       |
|                      |                       |                       |
|                      | A State may use these |                       |
|                      | procedures to         |                       |
|                      | establish separate    |                       |
|                      | starting points by    |                       |
|                      | grade span; however,  |                       |
|                      | the starting point    |                       |
|                      | must be the same for  |                       |
|                      | all like schools      |                       |
|                      | (e.g., one same       |                       |
|                      | starting point for    |                       |
|                      | all elementary        |                       |
|                      | schools, one same     |                       |
|                      | starting point for    |                       |
|                      | all middle            |                       |
|                      | schools...).          |                       |
+----------------------+-----------------------+-----------------------+

  -----------------------------------------------------------------------
  **STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS**

  -----------------------------------------------------------------------

Based on the administration of the rules identified within section 3.2
above, the State has established starting points for calculating
Adequate Yearly Progress.

Using baseline data from the 2001-02 school year, the State has
established starting points of proficiency separately in reading and
math for each grade level. Refer to **Appendix X: North Dakota
Assessment System 2001-02 Baseline Impact Data** to review the State's
impact data. The same starting point for reading and math will apply to
the aggregate student population within each subject and to each student
subgroup for each of the three grade levels. Each starting point is
based, at a minimum, on the higher of the following percentages of
students at the proficient level: (1) the percentage in the State of
proficient students in the lowest-achieving student subgroup; or, (2)
the percentage of proficient students in a public school at the 20^th^
percentile of the State's total enrollment among all schools ranked by
the percentage of students at the proficient level.

The State has established the following starting points for AYP.

  -------------------- ----------------- ----------------- -----------------
  **Subject**          **Grade Level**                     

                       **Four**          **Eight**         **Twelve**

  **Reading/language   65.1%             61.4%             42.9%
  arts**                                                   

  **Mathematics**      45.7%             33.3%             24.1%
  -------------------- ----------------- ----------------- -----------------

+----------------------+-----------------------+-----------------------+
| ## CRITICAL ELEMENT  | **EXAMPLES FOR\       | **EXAMPLES OF**       |
|                      | MEETING               |                       |
|                      | REQUIREMENTS**        | ***NOT* MEETING       |
|                      |                       | REQUIREMENTS**        |
+======================+=======================+=======================+
| 2.  What are the     | State has annual      | The State             |
|     State's annual   | measurable objectives | Accountability System |
|     measurable       | that are consistent   | uses another method   |
|                      | with a state's        | for calculating       |
| > objectives for     | intermediate goals    | annual measurable     |
| > determining        | and that identify for | objectives.           |
| > adequate yearly    | each year a minimum   |                       |
| > progress?          | percentage of         | The State             |
|                      | students who must     | Accountability System |
|                      | meet or exceed the    | does not include      |
|                      | proficient level of   | annual measurable     |
|                      | academic achievement  | objectives.           |
|                      | on the State's        |                       |
|                      | academic assessments. |                       |
|                      |                       |                       |
|                      | The State's annual    |                       |
|                      | measurable objectives |                       |
|                      | ensure that all       |                       |
|                      | students meet or      |                       |
|                      | exceed the State's    |                       |
|                      | proficient level of   |                       |
|                      | academic achievement  |                       |
|                      | within the timeline.  |                       |
|                      |                       |                       |
|                      | The State's annual    |                       |
|                      | measurable objectives |                       |
|                      | are the same          |                       |
|                      | throughout the State  |                       |
|                      | for each public       |                       |
|                      | school, each LEA, and |                       |
|                      | each subgroup of      |                       |
|                      | students.             |                       |
+----------------------+-----------------------+-----------------------+
| **STATE RESPONSE AND |                       |                       |
| STATE ACTIVITIES FOR |                       |                       |
| MEETING              |                       |                       |
| REQUIREMENTS**       |                       |                       |
+----------------------+-----------------------+-----------------------+

Based on the administration of the rules identified within section 3.2
above, the State has established measurable objectives for determining
Adequate Yearly Progress.

Using baseline data from the 2001-02 school year, the State has
established measurable objectives for proficiency separately in reading
and math for each year from 2001-02 to 2013-14. Refer to **Appendix X:
North Dakota Assessment System 2001-02 Baseline Impact Data** to review
the State's impact data. The same measurable objectives for reading and
math will apply to the aggregate student population within each subject
and to each student subgroup for each of the three grade levels.

The measurable objectives are determined using the baseline percentage
of proficient students statewide from the 2001-02 school year and
prorating the expected annual growth required to achieve 100% by
2013-14. The following chart identifies the measurable objectives
established for Adequate Yearly Progress.

<table>
<colgroup>
<col style="width: 12%" />
<col style="width: 6%" />
<col style="width: 6%" />
<col style="width: 6%" />
<col style="width: 6%" />
<col style="width: 6%" />
<col style="width: 6%" />
<col style="width: 6%" />
<col style="width: 6%" />
<col style="width: 6%" />
<col style="width: 6%" />
<col style="width: 6%" />
<col style="width: 5%" />
<col style="width: 6%" />
</colgroup>
<tbody>
<tr class="odd">
<td rowspan="2"><p><strong>Subject/</strong></p>
<p><strong>Grades</strong></p></td>
<td colspan="13"><strong>School Years</strong></td>
</tr>
<tr class="even">
<td><strong>01- 02</strong></td>
<td><strong>02-03</strong></td>
<td><strong>03-04</strong></td>
<td><strong>04-05</strong></td>
<td><strong>05-06</strong></td>
<td><strong>06-07</strong></td>
<td><strong>07-08</strong></td>
<td><strong>08-09</strong></td>
<td><strong>09-10</strong></td>
<td><strong>10-11</strong></td>
<td><strong>11-12</strong></td>
<td><strong>12-13</strong></td>
<td><strong>13-14</strong></td>
</tr>
<tr class="odd">
<td><strong>Reading</strong></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="even">
<td><strong>4</strong></td>
<td>65.1</td>
<td>68.0</td>
<td>70.9</td>
<td>73.8</td>
<td>76.7</td>
<td>79.6</td>
<td>82.6</td>
<td>85.5</td>
<td>88.4</td>
<td>91.3</td>
<td>94.2</td>
<td>97.1</td>
<td>100.0</td>
</tr>
<tr class="odd">
<td><strong>8</strong></td>
<td>61.4</td>
<td>64.6</td>
<td>67.8</td>
<td>71.1</td>
<td>74.3</td>
<td>77.5</td>
<td>80.7</td>
<td>83.9</td>
<td>87.1</td>
<td>90.4</td>
<td>93.6</td>
<td>96.8</td>
<td>100.0</td>
</tr>
<tr class="even">
<td><strong>12</strong></td>
<td>42.9</td>
<td>47.7</td>
<td>52.4</td>
<td>57.2</td>
<td>61.9</td>
<td>66.7</td>
<td>71.5</td>
<td>76.2</td>
<td>81.0</td>
<td>85.7</td>
<td>90.5</td>
<td>95.2</td>
<td>100.0</td>
</tr>
<tr class="odd">
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="even">
<td><strong>Math</strong></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="odd">
<td><strong>4</strong></td>
<td>45.7</td>
<td>50.2</td>
<td>54.8</td>
<td>59.3</td>
<td>63.8</td>
<td>68.3</td>
<td>72.9</td>
<td>77.4</td>
<td>81.9</td>
<td>86.4</td>
<td>91.0</td>
<td>95.5</td>
<td>100.0</td>
</tr>
<tr class="even">
<td><strong>8</strong></td>
<td>33.3</td>
<td>38.9</td>
<td>44.4</td>
<td>50.0</td>
<td>55.5</td>
<td>61.1</td>
<td>66.7</td>
<td>72.2</td>
<td>77.8</td>
<td>83.3</td>
<td>88.9</td>
<td>94.4</td>
<td>100.0</td>
</tr>
<tr class="odd">
<td><strong>12</strong></td>
<td>24.1</td>
<td>30.4</td>
<td>36.8</td>
<td>43.1</td>
<td>49.4</td>
<td>55.7</td>
<td>62.1</td>
<td>68.4</td>
<td>74.7</td>
<td>81.0</td>
<td>87.4</td>
<td>93.7</td>
<td>100.0</td>
</tr>
<tr class="even">
<td><strong>Graduation</strong></td>
<td>89.9</td>
<td></td>
<td></td>
<td>TBD</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="odd">
<td><strong>Attendance</strong></td>
<td>93.0</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
</tbody>
</table>

\* 2004-05 will initiate a new method for determining graduation rates
based on the disaggregated tracking of 9^th^ grade cohorts through to
graduation. In 2005 the State will recalculate the target graduation
rate using the 20% ranking rule for graduation rates.

+----------------------+-----------------------+-----------------------+
| ## CRITICAL ELEMENT  | **EXAMPLES FOR\       | **EXAMPLES OF**       |
|                      | MEETING               |                       |
|                      | REQUIREMENTS**        | ***NOT* MEETING       |
|                      |                       | REQUIREMENTS**        |
+======================+=======================+=======================+
| 3.2c What are the    | State has established | The State uses        |
| State's intermediate | intermediate goals    | another method for    |
| goals for            | that increase in      | calculating           |
| determining adequate | equal increments over | intermediate goals.   |
| yearly progress?     | the period covered by |                       |
|                      | the State timeline.   | The State does not    |
|                      |                       | include intermediate  |
|                      | -   The first         | goals in its          |
|                      |     incremental       | definition of         |
|                      |     increase takes    | adequate yearly       |
|                      |     effect not later  | progress.             |
|                      |     than the          |                       |
|                      |     2004-2005         |                       |
|                      |     academic year.    |                       |
|                      |                       |                       |
|                      | -   Each following    |                       |
|                      |     incremental       |                       |
|                      |     increase occurs   |                       |
|                      |     within three      |                       |
|                      |     years.            |                       |
+----------------------+-----------------------+-----------------------+
| **STATE RESPONSE AND |                       |                       |
| STATE ACTIVITIES FOR |                       |                       |
| MEETING              |                       |                       |
| REQUIREMENTS**       |                       |                       |
+----------------------+-----------------------+-----------------------+

The State of North Dakota stipulates that the State Superintendent has
established State intermediate goals for determining adequate yearly
progress that meet the provisions of ESEA section 1111. The intermediate
goals are based on the respective measurable objectives established from
the 2001-02 baseline data, set forth within Principle 3.2b above. Refer
to **Appendix X: North Dakota Assessment System 2001-02 Baseline Impact
Data** to review the State's impact data.

The intermediate goals will be based on the respective measurable
objectives for reading, mathematics, graduation, and attendance
determined independently and defined for the following years:

Step 1: 2001-02 through 2003-04 and set at the 2001-02 baseline AYP
cut-point.

Step 2: 2004-05 through to 2006-07 and set at the 2004-05 measurable
objective;

Step 3: 2007-08 through to 2009-10 and set at the 2008-09 measurable
objective;

Step 4: 2010-11 through to 2012-13 and set at the 2010-11 measurable
objective; and

Step 5: 2013-14 and set at the 2013-14 measurable objective.

The intermediate goals will be the effective AYP cut-point for all years
within each respective step. The intermediate goal will constitute the
AYP cut-point upon which all school and district program improvement
identification will be made. The following chart identifies the
respective intermediate goals for each respective subject and grade
level.

<table style="width:100%;">
<colgroup>
<col style="width: 11%" />
<col style="width: 6%" />
<col style="width: 6%" />
<col style="width: 6%" />
<col style="width: 6%" />
<col style="width: 6%" />
<col style="width: 6%" />
<col style="width: 6%" />
<col style="width: 6%" />
<col style="width: 6%" />
<col style="width: 6%" />
<col style="width: 6%" />
<col style="width: 6%" />
<col style="width: 6%" />
</colgroup>
<tbody>
<tr class="odd">
<td rowspan="2"><p><strong>Subject/</strong></p>
<p><strong>Grades</strong></p></td>
<td colspan="13"><strong>School Years</strong></td>
</tr>
<tr class="even">
<td><strong>01-02</strong></td>
<td><strong>02-03</strong></td>
<td><strong>03-04</strong></td>
<td><strong>04-05</strong></td>
<td><strong>05-06</strong></td>
<td><strong>06-07</strong></td>
<td><strong>07-08</strong></td>
<td><strong>08-09</strong></td>
<td><strong>09-10</strong></td>
<td><strong>10-11</strong></td>
<td><strong>11-12</strong></td>
<td><strong>12-13</strong></td>
<td><strong>13-14</strong></td>
</tr>
<tr class="odd">
<td><strong>Reading</strong></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td></td>
</tr>
<tr class="even">
<td><strong>4</strong></td>
<td colspan="3">65.1</td>
<td colspan="3">73.8</td>
<td colspan="3">82.6</td>
<td colspan="3">91.3</td>
<td>100.0</td>
</tr>
<tr class="odd">
<td><strong>8</strong></td>
<td colspan="3">61.4</td>
<td colspan="3">71.1</td>
<td colspan="3">80.7</td>
<td colspan="3">90.4</td>
<td>100.0</td>
</tr>
<tr class="even">
<td><strong>12</strong></td>
<td colspan="3">42.9</td>
<td colspan="3">57.2</td>
<td colspan="3">71.5</td>
<td colspan="3">85.7</td>
<td>100.0</td>
</tr>
<tr class="odd">
<td></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td></td>
</tr>
<tr class="even">
<td><strong>Math</strong></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td></td>
</tr>
<tr class="odd">
<td><strong>4</strong></td>
<td colspan="3">45.7</td>
<td colspan="3">59.3</td>
<td colspan="3">72.9</td>
<td colspan="3">86.4</td>
<td>100.0</td>
</tr>
<tr class="even">
<td><strong>8</strong></td>
<td colspan="3">33.3</td>
<td colspan="3">50.0</td>
<td colspan="3">66.7</td>
<td colspan="3">83.3</td>
<td>100.0</td>
</tr>
<tr class="odd">
<td><strong>12</strong></td>
<td colspan="3">24.1</td>
<td colspan="3">43.1</td>
<td colspan="3">62.1</td>
<td colspan="3">81.0</td>
<td>100.0</td>
</tr>
<tr class="even">
<td><strong>Graduation</strong></td>
<td colspan="13">89.9 *</td>
</tr>
<tr class="odd">
<td><strong>Attendance</strong></td>
<td colspan="13">93.0</td>
</tr>
</tbody>
</table>

**\*** 2004-05 will initiate a new method for determining graduation
rates based on the disaggregated tracking of 9^th^ grade cohorts through
to graduation. In 2005 the State will recalculate the target graduation
rate using the 20% ranking rule for graduation rates.

**PRINCIPLE 4. State makes annual decisions about the achievement of all
public schools and LEAs.**

+----------------------+-----------------------+-----------------------+
| ## CRITICAL ELEMENT  | ## EXAMPLES FOR       | **EXAMPLES OF**       |
|                      |                       |                       |
|                      | **MEETING             | ***NOT* MEETING       |
|                      | REQUIREMENTS**        | REQUIREMENTS**        |
+======================+=======================+=======================+
| 1.  How does the     | AYP decisions for     | AYP decisions for     |
|     State            | each public school    | public schools and    |
|     Accountability   | and LEA are made      | LEAs are not made     |
|     System make an   | annually.[^4]         | annually.             |
|     annual           |                       |                       |
|     determination of |                       |                       |
|     whether each     |                       |                       |
|     public school    |                       |                       |
|     and LEA in the   |                       |                       |
|     State made AYP?  |                       |                       |
+----------------------+-----------------------+-----------------------+
| **STATE RESPONSE AND |                       |                       |
| STATE ACTIVITIES FOR |                       |                       |
| MEETING              |                       |                       |
| REQUIREMENTS**       |                       |                       |
+----------------------+-----------------------+-----------------------+

The State of North Dakota stipulates that it will conduct annual reviews
of school and district achievement data for the purposes of determining
whether each public school and LEA had made AYP as provided within ESEA
section 1111.

North Dakota state law (NDCC 15.1-21-08) places responsibility with the
State Superintendent for the annual administration of State assessments
that are aligned to the State's content standards in reading and
mathematics (refer to **Appendix B: North Dakota Century Code
citations** or reference the North Dakota Century Code at the following
web site, <http://www.state.nd.us/lr/cencode/T151C21.pdf> ). State law
requires that the assessments be administered annually to at least one
grade level selected within each of the following grade spans: grades
three through five; grades six through nine; and grades ten through
twelve. The North Dakota Department of Public Instruction has developed
and administers assessments at grades 4, 8, and 12 to correspond with
the State's content standards. Effective with the 2006-07 administration
of adequate yearly progress, the State will conduct adequate yearly
progress determinations based on student achievement data for grades
3-8, and 11.

State law further requires that the State assessments compile aggregated
results and disaggregated results. The annual State assessments must
compile student achievement data that allows for a comparison of
individual students, classrooms within a given school and school
district, schools within the district, and school districts within the
state. The test scores must also allow for comparisons based on
students' gender, ethnicity, economic status, service status (i.e.,
migrant, LEP, disability), and assessment status (i.e., enrollment
status and participation status), unless doing so enables the
identification of any student. (Refer to NDCC 15.1-21-08 within
**Appendix B: North Dakota Century Code citations** or reference the
North Dakota Century Code at the following web site,
[http://www.state.nd.us/lr/cencode/T151C21.pdf](http://www.state.nd.us/lr/assembly/57-2001/cencode/CCT15x1.pdf)
).

The State will make its annual AYP review and determinations based
solely on student achievement data generated by the annual State
Assessment and on official graduation and attendance data reported to
and monitored by the State. The State will issue annual AYP status
reports to all LEAs and schools identifying each LEA's and school's
overall performance in terms of AYP performance goals.

> The State will produce all district- and school-level reports for
> districts and schools regarding their respective student achievement
> levels. These reports will include both aggregated and disaggregated
> student achievement data. Refer to **Appendix O: North Dakota School
> Report Card and Profile** for an illustration of the content of these
> student achievement profiles. Districts may use these profile reports
> as the foundation for their locally produced profiles. The
> State-generated reports will offer quality assurances regarding the
> generation of any district achievement data.
>
> The State requires all districts to disseminate student achievement
> report cards and profiles to their communities. This mandate is
> required as a condition of their receipt of federal funds. Refer to
> **Appendix S: Consolidated Application Certification and Assurances**
> for the State assessment requirement for receipt of federal ESEA
> funding. To assure compliance with this provision for the development
> and dissemination of performance profiles per Title I funding, the
> Department will require timely and comprehensive reports as a
> condition of receiving uninterrupted Title I funds. Further, evidence
> of these profiles will be one of the criteria in the Department's
> Title I monitoring program.

The State currently is reviewing its ESEA and accreditation monitoring
policies. The State is pursuing an amendment to its monitoring
requirements that would mandate districts to produce evidence regarding
the dissemination of achievement profiles to their communities. Monitors
would check for the production and dissemination of such achievement
profiles. Any failures to disseminate such profiles would be identified
as a compliance violation of the school's and district's ESEA compliance
agreement. Refer to **Appendix S: Consolidated Application Certification
and Assurances** for the State Assessment requirement for receipt of
federal ESEA funding. A school or district may be sanctioned for any
compliance violation of their ESEA assurances agreement.

All AYP review and determination activity will be conducted annually and
completed by July of each respective year.

**PRINCIPLE 5. All public schools and LEAs are held accountable for the
achievement of individual subgroups.**

+---------------------+-----------------------+-----------------------+
| ## CRITICAL ELEMENT | **EXAMPLES FOR**      | **EXAMPLES OF**       |
|                     |                       |                       |
|                     | **MEETING             | ***NOT* MEETING       |
|                     | REQUIREMENTS**        | REQUIREMENTS**        |
+=====================+=======================+=======================+
| 1.  How does the    | Identifies subgroups  | State does not        |
|     definition of   | for defining adequate | disaggregate data by  |
|     adequate yearly | yearly progress:      | each required student |
|     progress        | economically          | subgroup.             |
|     include all the | disadvantaged, major  |                       |
|     required        | racial and ethnic     |                       |
|     student         | groups, students with |                       |
|     subgroups?      | disabilities, and     |                       |
|                     | students with limited |                       |
|                     | English proficiency.  |                       |
|                     |                       |                       |
|                     | Provides definition   |                       |
|                     | and data source of    |                       |
|                     | subgroups for         |                       |
|                     | adequate yearly       |                       |
|                     | progress.             |                       |
+---------------------+-----------------------+-----------------------+
| **STATE RESPONSE    |                       |                       |
| AND STATE           |                       |                       |
| ACTIVITIES FOR      |                       |                       |
| MEETING             |                       |                       |
| REQUIREMENTS**      |                       |                       |
+---------------------+-----------------------+-----------------------+

The State of North Dakota stipulates that the State's definition of
adequate yearly progress includes all required subgroups as provided
within ESEA section 1111.

State law requires that the State assessments compile aggregated results
and disaggregated results. The State assessments must compile student
achievement data that allows for a comparison of individual students,
classrooms within a given school and school district, schools within the
district, and school districts within the state. The test scores must
also allow for comparisons based on students' gender, ethnicity,
economic status, service status (i.e., migrant, LEP, and disability),
and assessment status (i.e., enrollment and participation status),
unless doing so enables the identification of any student. (Refer to
NDCC 15.1-21-08 within **Appendix B: North Dakota Century Code
citations** or reference the North Dakota Century Code at the following
web site, <http://www.state.nd.us/lr/cencode/T151C21.pdf> ).

The results generated by the North Dakota State Assessment are reported
in mathematics and reading for grades 4, 8, and 12. Results are reported
at the individual student, school, district, and State level. Results
are disaggregated by gender, ethnicity, disability, limited English
proficiency status, migrant status, and economic disadvantaged status.
AYP determination includes consideration for ethnicity, disability,
limited English proficiency, and economic status. The following tables
summarize the level of the disaggregated reports.

+-----------------+--------------+------------+------------+----------+
| # Di            |              |            |            |          |
| saggregated Lev |              |            |            |          |
| els for ND Stat |              |            |            |          |
| e Assessment in |              |            |            |          |
|                 |              |            |            |          |
| > **Mathematics |              |            |            |          |
| > and Reading** |              |            |            |          |
| >               |              |            |            |          |
| > **Grades 4,   |              |            |            |          |
| > 8, and 12**   |              |            |            |          |
| >               |              |            |            |          |
| > **(\*\*       |              |            |            |          |
| > refers to AYP |              |            |            |          |
| > subgroups)**  |              |            |            |          |
+-----------------+--------------+------------+------------+----------+
| > **Reporting   | > **         | >          | > **       | > *      |
| > Level**       | Individual** | **School** | District** | *State** |
|                 | >            |            |            |          |
|                 | >            |            |            |          |
|                 |  **Student** |            |            |          |
+-----------------+--------------+------------+------------+----------+
| ### **Gender**  | > N/A        | > \*       | > \*       | > \*     |
+-----------------+--------------+------------+------------+----------+
| **Ethnicity     | > N/A        | > \*       | > \*       | > \*     |
| \*\***          |              |            |            |          |
+-----------------+--------------+------------+------------+----------+
| **Disability    | > N/A        | > \*       | > \*       | > \*     |
| \*\***          |              |            |            |          |
+-----------------+--------------+------------+------------+----------+
| **Limited       | > N/A        | > \*       | > \*       | > \*     |
| English         |              |            |            |          |
| Proficient      |              |            |            |          |
| \*\***          |              |            |            |          |
+-----------------+--------------+------------+------------+----------+
| **Migrant**     | > N/A        | > \*       | > \*       | > \*     |
+-----------------+--------------+------------+------------+----------+
| *               | > N/A        | > \*       | > \*       | > \*     |
| *Economically** |              |            |            |          |
|                 |              |            |            |          |
| **Disadvantaged |              |            |            |          |
| \*\***          |              |            |            |          |
+-----------------+--------------+------------+------------+----------+

The State and its assessment contractor assume the full responsibility
for generating aggregate and disaggregated student achievement reports.
Local districts do not generate these reports.

Student demographic information is gathered at the time of the
assessment administration on the individual student's assessment
demographic sheet. On this sheet the student or a school official
completes basic information about the student, including their name and
other essential information. The assessment requires completion of
certain demographic and special codes that are included on the
demographic sheet and detailed for testing coordinators within the *Test
Coordinator's Manual*. Refer to pages 28-31 of **Appendix Q: North
Dakota State Assessment, Test Coordinator's Manual 2002-03** for a
listing of the various demographic and special categories used to
describe a student. These codes are then used during the process of
classifying student achievement by subgroup populations.

A central concern of any demographic collection process is the
introduction of erroneous information on the part of an individual. This
is especially troublesome within an assessment system where information
can be inadvertently omitted. The State of North Dakota has established
a plan to centralize student identification information with the use of
a data analysis and reporting application contracted through TetraData
Corporation. Refer to **Appendix R: TetraData Data Analysis and
Reporting System Summary** for an overview of the application. This
application will allow the State to routinely link student
identification information statewide with the database supplied by
CTB/McGraw-Hill in order to identify and reconcile incorrect
information. The use of this data linkage application will enhance the
accuracy, reliability, and speed of collecting the demographic
information used to classify school, district, and State subgroup
achievement reports.

Disaggregated reports will approximate the presentation format
identified with the State Report Card and Profile. Refer to **Appendix
O: North Dakota State Report Card and Profile** for an example of the
presentation format used to report disaggregated subgroup achievement
data.

The State alone may authorize the publication of any reports regarding
the State Assessment for accountability purposes based on State
Assessment data. The State's contractor (CTB/McGraw-Hill) produces all
reports for the State Assessment. The State recognizes no other
assessment reports produced by other outside sources, including
districts and schools, as authoritative regarding the State Assessment.

+---------------------+-----------------------+-----------------------+
| ## CRITICAL ELEMENT | **EXAMPLES FOR**      | **EXAMPLES OF**       |
|                     |                       |                       |
|                     | **MEETING             | ***NOT* MEETING       |
|                     | REQUIREMENTS**        | REQUIREMENTS**        |
+=====================+=======================+=======================+
| 2.  How are public  | Public schools and    | State does not        |
|     schools and     | LEAs are held         | include student       |
|     LEAs held       | accountable for       | subgroups in its      |
|     accountable for | student subgroup      | State Accountability  |
|     the progress of | achievement:          | System.               |
|     student         | economically          |                       |
|     subgroups in    | disadvantaged, major  |                       |
|     the             | ethnic and racial     |                       |
|     determination   | groups, students with |                       |
|     of adequate     | disabilities, and     |                       |
|     yearly          | limited English       |                       |
|     progress?       | proficient students.  |                       |
+---------------------+-----------------------+-----------------------+
| **STATE RESPONSE    |                       |                       |
| AND STATE           |                       |                       |
| ACTIVITIES FOR      |                       |                       |
| MEETING             |                       |                       |
| REQUIREMENTS**      |                       |                       |
+---------------------+-----------------------+-----------------------+

The State of North Dakota stipulates that it will conduct an AYP review
and determination for each school and LEA based on the progress of
student subgroups as provided within ESEA section 1111.

As identified in Principle 5.1 above, the State provides a method to
identify, record, and report student achievement for all subgroups. The
State will disaggregate and hold schools and LEAs accountable for the
performance of each of the following student subgroups:

-   All Students

-   Asian/Pacific

-   Black

-   Hispanic

-   Native American

-   White

-   Economic disadvantaged

-   Limited English Proficient

-   Students with Disabilities

The State will determine whether each subgroup within each school and
LEA achieved the annual measurable objective, or met the "Safe Harbor"
provision, and met the 95% participation rate criteria. For a school or
LEA to make AYP, every group for which a school or LEA is accountable
must make AYP. Any subgroup that makes AYP based on the safe harbor
provision must also make AYP based on the appropriate secondary
indicator (i.e., graduation rate or attendance rate). The rules for
statistical reliability will apply in reviewing and determining subgroup
accountability.

+---------------------+-----------------------+-----------------------+
| ## CRITICAL ELEMENT | **EXAMPLES FOR**      | **EXAMPLES OF**       |
|                     |                       |                       |
|                     | **MEETING             | ***NOT* MEETING       |
|                     | REQUIREMENTS**        | REQUIREMENTS**        |
+=====================+=======================+=======================+
| 1.  How are         | All students with     | The State             |
|     students with   | disabilities          | Accountability System |
|     disabilities    | participate in        | or State policy       |
|     included in the | statewide             | excludes students     |
|     State's         | assessments: general  | with disabilities     |
|     definition of   | assessments with or   | from participating in |
|     adequate yearly | without               | the statewide         |
|     progress?       | accommodations or an  | assessments.          |
|                     | alternate assessment  |                       |
|                     | based on grade level  | State cannot          |
|                     | standards for the     | demonstrate that      |
|                     | grade in which        | alternate assessments |
|                     | students are          | measure grade-level   |
|                     | enrolled.             | standards for the     |
|                     |                       | grade in which        |
|                     | State demonstrates    | students are          |
|                     | that students with    | enrolled.             |
|                     | disabilities are      |                       |
|                     | fully included in the |                       |
|                     | State Accountability  |                       |
|                     | System.               |                       |
+---------------------+-----------------------+-----------------------+
| **STATE RESPONSE    |                       |                       |
| AND STATE           |                       |                       |
| ACTIVITIES FOR      |                       |                       |
| MEETING             |                       |                       |
| REQUIREMENTS**      |                       |                       |
+---------------------+-----------------------+-----------------------+

The State of North Dakota stipulates that all students with disabilities
will be included in the State's definition of adequate yearly progress
as provided within ESEA section 1111.

State law requires that the State assessments compile aggregated results
and disaggregated results. The State assessments must compile student
achievement data that allows for a comparison of classrooms within a
given school and school district, schools within the district, and
school districts within the state. The test scores must also allow for
comparisons based on students' gender, ethnicity, economic status,
service status (i.e., migrant, LEP, and disability), and assessment
status (i.e., enrollment and participation status), unless doing so
enables the identification of any student. (Refer to NDCC 15.1-21-08
within **Appendix B: North Dakota Century Code citations** or reference
the North Dakota Century Code at the following web site,
[http://www.state.nd.us/lr/cencode/T151C21.pdf](http://www.state.nd.us/lr/assembly/57-2001/cencode/CCT15x1.pdf)
).

It is the policy of the Department of Public Instruction to include all
students with disabilities in the North Dakota accountability system.
See enclosed **Appendix ­­U: Individualized Education Program Planning
Proces*s****,* or access this document at
<http://www.dpi.state.nd.us/speced/guide/iep/index.shtm>. The State's
individualized education program (IEP) form (page 4), required for every
student eligible under the Individuals with Disabilities Education Act
(IDEA), includes a section requiring the description of the student's
participation in district-wide and statewide assessments. The IEP team
must indicate whether the student will participate without
accommodations, with accommodations (which must be stated), or in the
Alternate Assessment. This element of the IEP is addressed by the school
district as it conducts a self-assessment in preparation for the Office
of Special Education monitoring. If violations are found, corrective
actions are determined and evidence of completion is required.

The State Assessment Program *Test Coordinator's Manual, 2002-2003,*
**(Appendix Q**) provides very limited opportunity for a school to
exclude a student from participation in the State Assessment by
invalidating an assessment. Any school that proposes to invalidate a
student's test must provide written documentation to the Department of
Public Instruction stating the reason for test invalidation. The
authorizing administrator must sign the form. If a school systematically
fails to include students in the State Assessment, sanctions will be
imposed. Any non-participating student or any invalidated assessment
will be included into the calculation to determine the participation
rate of the school, district and State.

All students who participate in the North Dakota Alternate Assessment
will have levels of performance included within the State accountability
system. The State Assessment Program *Test Coordinator's Manual,
2002-2003,* provides guidance for use of the Alternate Assessment
(**Appendix Q**), and in the use of accommodations (pages 33 -- 35).

As provided under pending federal regulations and allowances offered by
the U.S. Department of Education, the State may use alternate
achievement standards to calculate AYP for schools and districts for
students with the most significant cognitive disabilities who take an
alternate assessment. This allowance is offered until the newly proposed
regulations related to alternate assessments become finalized. The State
attests that the State's alternate assessments and achievement standards
are aligned with North Dakota's academic content standards and reflect
qualified, professional judgment of the highest learning standards
possible for these students.

Additionally, the State acknowledges that the percentage of students
identified as proficient on the alternate assessment at the district and
State levels may not exceed 1.0 percent of all students in the grades
assessed. The State stipulates that it will monitor the participation
and achievement rates of students in the alternate assessments within
each district and across the State; no district or the State will be
permitted to exceed this limit. The State will calculate and monitor the
overall enrollment of students, the participation rates of students
within the State Assessment, and the participation and achievement rates
of students within the Alternate Assessment.

*The incorporation of an alternate assessment provision identified in
federal guidance, dated May 10, 2005, affecting students with persistent
cognitive disabilities, beginning with the 2004-05 school year.*

On May 10, 2005, U.S. Secretary of Education Margaret Spellings
announced the details of a new policy designed to assist students with
persistent cognitive disabilities to more appropriately participate in
their state assessment and accountability system. Under this policy a
limited number of students with persistent cognitive disabilities,
approximating two percent, will be allowed to have the results from
alternate assessments classified as proficient when specifically based
on alternate achievement standards. This provision is a policy separate
from and in addition to the current regulation that allows up to one
percent of all students being tested, i.e., those students with a
significant cognitive disability, to have the results from alternate
assessments classified as proficient based on alternate achievement
standards.

Effective with the 2007-08 academic year, the State will administer an
alternate assessment for certain students with disabilities based on
modified achievement standards for the purposes of determining adequate
yearly progress and restricting any such determinations of proficient
and advanced scores to not greater than 2 percent of students assessed
on the North Dakota State Assessment or as provided by federal
regulation. The application of this amendment is contingent wholly upon
the state's submission of its alternate assessment based on modified
achievement standards to the U.S. Department of Education for purposes
of peer review and the subsequent approval of the alternate assessment
for the purposes of determining adequate yearly progress, or any interim
approval granted by the U.S. Department of Education.

The Department of Public Instruction stipulates that it will abide by
the established core principles identified within the May 10, 2005
guidance.

1.  Statewide assessment participation rates for students with
    disabilities, for purposes of measuring adequate yearly progress,
    will remain at or above 95%. During the 2003-04 school year, the
    statewide students with disabilities participation rate equaled
    98.05% in reading and 97.83% in mathematics. (Reference the
    following web address for the state report:
    <http://www.dpi.state.nd.us/dpi/reports/profile/0304/AYPState/aypstate2004.pdf>.);

2.  Alternate assessments in reading/language arts and mathematics are
    available for students with disabilities who are unable to
    participate in the regular assessment even with accommodations (as
    required by the Individuals with Disabilities Education Act) and the
    State will report results to the Secretary and the public based on
    these alternate assessments in reading/language arts and
    mathematics;

3.  Appropriate accommodations are available for the state assessment
    for students with disabilities;

4.  The State stipulates that all students with disabilities will
    continue to be included in the state's assessment system. The
    Department of Public Instruction will cross-check that all
    identified special education students will be included in the
    statewide assessment file, as captured on the state's Online
    Reporting System. This system also monitors special education
    enrollment for payment purposes.

The Department of Public Instruction has previously submitted support
data and narrative within the state's Consolidated Performance Report,
which is available to the U.S. Department of Education, to establish the
state's eligibility status. Following a review by the U.S. Department of
Education on the state's eligibility status to receive this proposed
provision, the Department of Public Instruction will submit additional
evidence as identified within the "Sound State Education Policies"
within the May 10, 2005 guidance.

+---------------------+-----------------------+-----------------------+
| ## CRITICAL ELEMENT | **EXAMPLES FOR**      | **EXAMPLES OF**       |
|                     |                       |                       |
|                     | **MEETING             | ***NOT* MEETING       |
|                     | REQUIREMENTS**        | REQUIREMENTS**        |
+=====================+=======================+=======================+
| 1.  How are         | All LEP student       | LEP students are not  |
|     students with   | participate in        | fully included in the |
|     limited English | statewide             | State Accountability  |
|     proficiency     | assessments: general  | System.               |
|     included in the | assessments with or   |                       |
|     State's         | without               |                       |
|     definition of   | accommodations or a   |                       |
|     adequate yearly | native language       |                       |
|     progress?       | version of the        |                       |
|                     | general assessment    |                       |
|                     | based on grade level  |                       |
|                     | standards.            |                       |
|                     |                       |                       |
|                     | State demonstrates    |                       |
|                     | that LEP students are |                       |
|                     | fully included in the |                       |
|                     | State Accountability  |                       |
|                     | System.               |                       |
+---------------------+-----------------------+-----------------------+
| **STATE RESPONSE    |                       |                       |
| AND STATE           |                       |                       |
| ACTIVITIES FOR      |                       |                       |
| MEETING             |                       |                       |
| REQUIREMENTS**      |                       |                       |
+---------------------+-----------------------+-----------------------+

The State of North Dakota stipulates that all limited English proficient
students will be included in the State's definition of adequate yearly
progress as provided within ESEA section 1111.

State law requires that the State assessments compile aggregated results
and disaggregated results. The State assessments must compile student
achievement data that allows for a comparison of classrooms within a
given school and school district, schools within the district, and
school districts within the state. The test scores must also allow for
comparisons based on students' gender, ethnicity, economic status,
service status (i.e., migrant, LEP, and disability) and assessment
status (i.e., enrollment and participation status), unless doing so
enables the identification of any student. (Refer to NDCC 15.1-21-08
within **Appendix B: North Dakota Century Code citations** or reference
the North Dakota Century Code at the following web site,
[http://www.state.nd.us/lr/cencode/T151C21.pdf](http://www.state.nd.us/lr/assembly/57-2001/cencode/CCT15x1.pdf)
).

It is the policy of the Department of Public Instruction to include all
LEP students in the State Assessment program. The State Consolidated
Application (Part I-H), which was approved by the U.S. Department of
Education (see **Appendix H**), indicates the State's commitment to
include all students in the State Assessment.

The School Report Card and Profile, as illustrated in **Appendix O**,
reports LEP student achievement against the State standards, compared
with other students.

Accountability for LEP student achievement is predicated on the ability
of schools and LEAs to assess all students suspected of having limited
English proficiency, to identify those meeting the federal definition of
LEP, and to record all LEP students who participate in the State
Assessment. The Department of Public Instruction reconciles all
discrepancies in LEP student numbers reported via the Survey of the
State's Limited English Proficient Students and Available Educational
Programs and Services compared with State Assessment statistics and the
TetraData data analysis and reporting system.

The State provides to LEP student the right to accommodations in the
classroom and in the State Assessment. Accommodations are listed in the
Test Coordinator's Manual for the statewide achievement testing program
on pages 33 - 35, located in **Appendix Q**, and at the following
website: <http://www.dpi.state.nd.us/testing/assess/testmanl.pdf>. The
North Dakota State Task Force on Limited English Proficiency, convened
by the Department of Public Instruction in 2000, developed guidance for
LEP students and state content standards. During the 2002-2003 school
year, the Task Force will further refine the guidance for limited
English proficient students and statewide achievement testing. This will
be provided to schools and districts, along with specific accommodations
for the levels of English language proficiency. See **Appendix H: North
Dakota State ESEA Consolidated Application** for further discussion of
accommodations.

*The incorporation of federal guidance regarding the assessment of new
limited English proficient (LEP) students and the extended
classification of LEP students for two years beyond program status for
the purposes of determining adequate yearly progress.*

On February 20, 2004, the U.S. Department of Education released guidance
regarding the assessment of new LEP students and the inclusion of new
LEP students within the State's accountability system. This guidance
acknowledges that teachers need sufficient time with their students to
provide high-quality instruction before it is appropriate to hold a
particular school accountable. The following provisions are limited by
federal regulation and restricted only to those LEP students who are
newly arrived to the United States within the allowance of the current
instructional year. Any LEP student identified outside this restricted
definition of "new" will be covered by provisions of the current State
Accountability Workbook. The following provisions apply.

a\. *Definition of first year of enrollment*. A "new" LEP student is
understood as any student who has arrived in the United States and whose
direct instruction is initiated and conducted within the course of the
current school year. Under the State's accountability plan for schools
and districts, within the limits of federal regulation, only those
students who have been under the direct instruction of a school since
the beginning of the school year will be included in the determination
of adequate yearly progress under the assessment provisions outlined in
subsection b below. The designation of entry into the United States is
recorded within the LEP record of the pupil membership file on the
State's Online Reporting System.

b\. *Assessing new limited English proficient students.* For purposes of
participation in the State assessment system, LEP students, during their
first year of enrollment in U.S. schools, must take an English
proficiency assessment, as provided within Section 1111(b)(7) of the
Act, and may participate in the reading/language arts assessment, as
provided within Section 1111(b)(3) of the Act. The assessment results of
first-year LEP students who take the reading/language arts assessment
will not be included in determining adequate yearly progress for
achievement,. New LEP students who take the English language proficiency
assessment and/or the reading/language arts assessment, will be counted
as participants toward meeting the 95% assessment participation required
for adequate yearly progress determinations for reading.

New LEP students will take the State's mathematics assessment, as
provided within Section 1111(b)(3) of the Act, with accommodations as
necessary; however, the State will not include these new LEP students'
assessment results from the mathematics assessments during their first
year in U.S. schools. New LEP students who take the mathematics
assessment will be counted as participants toward meeting the 95%
assessment participation required for adequate yearly progress
determinations for mathematics.

c\. *Two-year extension of LEP status for accountability reporting.* The
State will include in the LEP subgroup any student who had previously
been considered an LEP student during the past one or two years, to
calculate adequate yearly progress for schools, districts, and the
State. The determination of when a student has attained English language
proficiency and is no longer an LEP student will follow the State's
method and definition as provided within the State's Consolidated
Application. New LEP students will not be included into the base
calculation of determining a minimum LEP subgroup size.

The Department of Public Instruction will conduct an independent review
of all new LEP student reports, as captured within the State's pupil
membership file, and validate all data points used to substantiate the
claim of a new LEP student's status.

+---------------------+-----------------------+-----------------------+
| ## CRITICAL ELEMENT | **EXAMPLES FOR**      | **EXAMPLES OF**       |
|                     |                       |                       |
|                     | **MEETING             | ***NOT* MEETING       |
|                     | REQUIREMENTS**        | REQUIREMENTS**        |
+---------------------+-----------------------+-----------------------+
| 2.  What is the     | State defines the     | State does not define |
|     State\'s        | number of students    | the required number   |
|     definition of   | required in a         | of students in a      |
|     the minimum     | subgroup for          | subgroup for          |
|     number of       | reporting and         | reporting and         |
|     students in a   | accountability        | accountability        |
|     subgroup        | purposes, and applies | purposes.             |
|     required for    | this definition       |                       |
|     reporting       | consistently across   | Definition is not     |
|     purposes? For   | the State.[^5]        | applied consistently  |
|     accountability  |                       | across the State.     |
|     purposes?       | Definition of         |                       |
|                     | subgroup will result  | Definition does not   |
|                     | in data that are      | result in data that   |
|                     | statistically         | are statistically     |
|                     | reliable.             | reliable.             |
+---------------------+-----------------------+-----------------------+
| **STATE RESPONSE    |                       |                       |
| AND STATE           |                       |                       |
| ACTIVITIES FOR      |                       |                       |
| MEETING             |                       |                       |
| REQUIREMENTS**      |                       |                       |
+---------------------+-----------------------+-----------------------+

The State of North Dakota has established a definition for the minimum
number of students in a subgroup for both reporting and accountability
purposes. The definition is consistent with the minimum number
identified within Principle 9. The State has established a test of
statistical significance for the method of determining a minimum number
within a given population and referenced to the established measurable
objective, safe harbor, participation rate, graduation rate, or
attendance rate. Refer to Principle 9.1 for a detailed overview of this
method of statistical significance.

+---------------------+-----------------------+-----------------------+
| ## CRITICAL ELEMENT | **EXAMPLES FOR**      | **EXAMPLES OF**       |
|                     |                       |                       |
|                     | **MEETING             | ***NOT* MEETING       |
|                     | REQUIREMENTS**        | REQUIREMENTS**        |
+=====================+=======================+=======================+
| 3.  How does the    | Definition does not   | Definition reveals    |
|     State           | reveal personally     | personally            |
|     Accountability  | identifiable          | identifiable          |
|     System protect  | information.[^6]      | information.          |
|     the privacy of  |                       |                       |
|     students when   |                       |                       |
|     reporting       |                       |                       |
|     results and     |                       |                       |
|     when            |                       |                       |
|     determining     |                       |                       |
|     AYP?            |                       |                       |
+---------------------+-----------------------+-----------------------+
| **STATE RESPONSE    |                       |                       |
| AND STATE           |                       |                       |
| ACTIVITIES FOR      |                       |                       |
| MEETING             |                       |                       |
| REQUIREMENTS**      |                       |                       |
+---------------------+-----------------------+-----------------------+

The Department of Public Instruction employs a four level procedure,
described below, to eliminate the possibility of compromising student
identification through an inadvertent publication of student achievement
results. These procedures are designed to eliminate any violation of
FERPA law regarding student privacy.

\(1\) *Minimal N Value Rule.* The Department employs an N\<10 value,
where any population value N less than 10 will prohibit the reporting of
students within an identified population. Any population value N of 10
or greater will allow the reporting of students within an identified
subgroup.

\(2\) *Single-populated Level Rule.* The Department employs a rule where
if all students within a school or subgroup report at a certain
performance level and no other performance levels record any students,
then the Department will record a limited percentage of students,
presented as an inequality, to serve as a representative finding. As
such, if all students were to reside within a given level, for example
"partially proficient", then reporting on that level will identify any
and all students. This would be a violation. To remedy this situation, a
representative inequality (e.g. \<5% or \>95%) will be recorded.

\(3\) *Total Population Below Proficient Rule.* The Department employs a
rule to allow for the proper identification of a school or district
where all students' achievement scores fall below proficient (i.e., the
combination of partially proficient and novice). It is in the interest
of the public and students that any school or district with 100%
below-proficient achievement scores be identified for not making
Adequate Yearly Progress. To eliminate the possibility of identifying
any student, the reports for schools and districts with 100%
below-proficient achievement scores will record an inequality to serve
as a representative finding (e.g., \<5% or \>95%). This representative
finding would eliminate any possible student identification and also
allow for the proper identification of the school or district. In the
absence of this rule, extremely low performing schools would be exempt
from not making Adequate Yearly Progress, thereby violating the
principle of validity.

\(4\) *Distinguished Students Rule.* The Department employs a rule to
allow for the proper identification of a school or district where all
students' achievement scores rest above proficient (i.e., the
combination of proficient and advanced). It is in the interest of the
public and students that any school or district with 100%
above-proficient achievement scores be identified as making Adequate
Yearly Progress. To eliminate the possibility of identifying any
student, the reports for schools and districts with 100%
above-proficient achievement scores will record an inequality to serve
as a representative finding (e.g., \<5% or \>95%). This representative
finding would eliminate any possible student identification and also
allow for the proper identification of the school or district. In the
absence of this rule, high performing schools would not be recognized
for making Adequate Yearly Progress.

**PRINCIPLE 6. State definition of AYP is based primarily on the State's
academic assessments.**

+----------------------+-----------------------+-----------------------+
| ## CRITICAL ELEMENT  | ## EXAMPLES FOR       | ## EXAMPLES OF        |
|                      |                       |                       |
|                      | **MEETING             | ***NOT* MEETING       |
|                      | REQUIREMENTS**        | REQUIREMENTS**        |
+======================+=======================+=======================+
| 1.  How is the       | Formula for AYP shows | Formula for AYP shows |
|     State's          | that decisions are    | that decisions are    |
|     definition of    | based primarily on    | based primarily on    |
|     adequate yearly  | assessments.[^7]      | non-academic          |
|     progress based   |                       | indicators or         |
|     primarily on     | Plan clearly          | indicators other than |
|     academic         | identifies which      | the State             |
|     assessments?     | assessments are       | assessments.          |
|                      | included in           |                       |
|                      | accountability.       |                       |
+----------------------+-----------------------+-----------------------+
| **STATE RESPONSE AND |                       |                       |
| STATE ACTIVITIES FOR |                       |                       |
| MEETING              |                       |                       |
| REQUIREMENTS**       |                       |                       |
+----------------------+-----------------------+-----------------------+

The State of North Dakota stipulates that the State's definition of
adequate yearly progress is based primarily on academic assessments as
provided within ESEA section 1111.

North Dakota state law (NDCC 15.1-21-08) places responsibility with the
State Superintendent for the administration of State assessments that
are aligned to the State's content standards in reading and mathematics
(refer to **Appendix B: North Dakota Century Code citations** or
reference the North Dakota Century Code at the following web site,
<http://www.state.nd.us/lr/cencode/T151C21.pdf> ). State law requires
that the assessments be administered to at least one grade level
selected within each of the following grade spans: grades three through
five; grades six through nine; and grades ten through twelve. The North
Dakota Department of Public Instruction has developed and administers
assessments at grades 4, 8, and 12 to correspond with the State's
content standards.

The State AYP plan meets the requirements of the ESEA*,* including
emphasis on the school identification method referenced to student
achievement proficiency rating, safe harbor provisions, statistical
reliability assurances, graduation rates for secondary schools,
attendance rates for elementary schools, and a minimum assessment
participation rate of 95%. The primary means for the identification of
schools and LEAs is, nevertheless, student achievement data.

The State of North Dakota will only recognize and reference student
achievement proficiency ratings generated by the North Dakota State
Assessment and its Alternate Assessment. No other student achievement
assessment tools or means will be recognized. No local assessments will
be recognized as an alternative to the North Dakota State Assessment.

The only definitions of achievement levels recognized by the State AYP
accountability system are those proficiency levels set for the North
Dakota State Assessment through the standards-setting process. Refer to
**Appendix J: North Dakota State Assessment, Bookmark Standards Setting
Technical Report, 2002** for the established definitions of the North
Dakota achievement levels. The State cut scores for the North Dakota
State Assessment will constitute the defining scales for identifying
schools and districts for AYP. Refer to **Appendix K: State
Superintendent's Approval Notification of North Dakota State Assessment
Cut Scores** for the State's announced policy regarding the
establishment of performance level cut scores.

PRINCIPLE 7. State definition of AYP includes graduation rates for
public High schools and an additional indicator selected by the State
for public Middle and public Elementary schools (such as attendance
rates).

+---------------------+-----------------------+-----------------------+
| ## CRITICAL ELEMENT | **EXAMPLES FOR**      | **EXAMPLES OF**       |
|                     |                       |                       |
|                     | **MEETING             | ***NOT* MEETING       |
|                     | REQUIREMENTS**        | REQUIREMENTS**        |
+=====================+=======================+=======================+
| 1.  What is the     | State definition of   | State definition of   |
|     State           | graduation rate:      | public high school    |
|     definition for  |                       | graduation rate does  |
|     the public high | -   Calculates the    | not meet these        |
|     school          |     percentage of     | criteria.             |
|     graduation      |     students,         |                       |
|     rate?           |     measured from the |                       |
|                     |     beginning of the  |                       |
|                     |     school year, who  |                       |
|                     |     graduate from     |                       |
|                     |     public high       |                       |
|                     |     school with a     |                       |
|                     |     regular diploma   |                       |
|                     |     (not including a  |                       |
|                     |     GED or any other  |                       |
|                     |     diploma not fully |                       |
|                     |     aligned with the  |                       |
|                     |     state's academic  |                       |
|                     |     standards) in the |                       |
|                     |     standard number   |                       |
|                     |     of years; or,     |                       |
|                     |                       |                       |
|                     | -   Uses another more |                       |
|                     |     accurate          |                       |
|                     |     definition that   |                       |
|                     |     has been approved |                       |
|                     |     by the Secretary; |                       |
|                     |     and               |                       |
|                     |                       |                       |
|                     | -   Must avoid        |                       |
|                     |     counting a        |                       |
|                     |     dropout as a      |                       |
|                     |     transfer.         |                       |
|                     |                       |                       |
|                     | Graduation rate is    |                       |
|                     | included (in the      |                       |
|                     | aggregate) for AYP,   |                       |
|                     | and disaggregated (as |                       |
|                     | necessary) for use    |                       |
|                     | when applying the     |                       |
|                     | exception clause[^8]  |                       |
|                     | to make AYP.          |                       |
+---------------------+-----------------------+-----------------------+

  -----------------------------------------------------------------------
  **STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS**

  -----------------------------------------------------------------------

The State of North Dakota stipulates that it has established the
graduation rate of each high school as a component for determining
adequate yearly progress, as provided within ESEA section 1111.

The graduation rate defined within ESEA section 1111 requires the State
to report graduates, retentions, and dropouts, within cohorts, in the
aggregate and disaggregated by subgroups. The State has initiated
measures to collect and report this information to the specification of
the Act; however, the State's full capacity to do so will not become
effective until 2005. In the interim, until State data to perform the
required calculations becomes available, the State will define and use
an alternative measure, based on schools' reported dropout and
graduation data within cohorts where graduation occurs in a standard
number of years. The interim measure, effective for the graduating
classes of 2003 and 2004, will be defined by the following equation:

Number of Graduates

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

(divided by)

Number of Graduates + Dropouts Yr1 + Dropouts Yr2 + Dropouts Yr3 +
Dropouts Yr4

The State stipulates that, as required under final Title I regulations,
this definition will avoid counting a dropout as a transfer and will not
include students who receive a non-standard diploma (e.g., attendance
certificate, GED). Students that transfer in or out of the school after
the State Assessment administration will not be included in the
denominator or numerator.

The State has established the target graduation rate based on the same
20% ranking rule used for determining achievement targets. Any district
with a graduation rate lower than this target point will be identified
for not making Adequate Yearly Progress. This target point will remain
as the State reference for graduation throughout the duration of the
2001-2005 school years. Based on this interim definition, the State has
established a graduation target point of 89.9% based on North Dakota
2001-02 graduation baseline impact data. This 89.9% target rate will be
applied for the first time to 2002-03 graduation rates. Refer to
**Appendix Y: North Dakota 2001-02 Graduation Impact Data** for a
summary of the impact data.

In 2005 when the State transfers from its current definition of
graduation to that used within NCLBA, the State will recalculate the
target graduation rate using the 20% ranking rule for graduation rates.
This target point will remain as the State definition for graduation
throughout the duration of the 2005-2014 school years. Therefore, it is
anticipated that the State's interim graduation target point of 89.9%
will be revised with the scheduled 2005 recalculation.

The State will begin reporting graduation rates using the NCLBA
definition in 2005, using collected cohort State data from 2001 -- 2005.
The rate will be calculated based on the following equation:

> \# Graduates (with regular diploma) who completed high school in four
> years

#### (divided by)

> \[# Graduates (same as above) + \# of 9^th^ grade
> dropouts/retentions + \# 10^th^ grade dropouts/retentions + \# 11^th^
> grade dropouts/retentions + \# 12^th^ grade dropouts/retentions + \#
> students who complete 12^th^ grade without a regular diploma\]

The data for each class will be tracked forward from 9^th^ grade.
Dropouts are defined as students who leave school prior to graduation
for reasons other than transfer to another school. Students who are
retained in grade, and thus leave their original class, will not count
toward the number of graduates, but will be included in the denominator
as members of the original class.

The State stipulates that any school or district that has met the
requirements of safe harbor for any specified subgroup must also
demonstrate that it has met the requirements for graduation rate for
that same specified subgroup as required under 34 CFR 200.19(d)(2)(i).
The State anticipates having a student data warehouse in place by 2005
to accommodate the monitoring and reporting of disaggregated graduation
rates. Prior to its full implementation, the State will require schools
or districts that have met safe harbor within a specified subgroup to
also evidence the achievement of the graduation rate for that specified
subgroup. The State will independently review all school and district
information to validate the authenticity of these data. Following 2005,
the State anticipates an ability to automate this activity with the
statewide student data warehouse.

*The restricted extension of a graduation cohort beyond four years, as
defined within an individualized education program, for students served
within provisions of the Individuals with Disabilities Education Act or
as provided within a LEP service program for specified LEP students;*

The graduation rate defined within ESEA section 1111 requires the State
to report graduates, retentions, and dropouts, within cohorts, in the
aggregate and disaggregated by subgroups. The Department reaffirms its
commitment to proceed with the determination of adequate yearly progress
based on these elements.

The Department of Public Instruction is also mindful of its
responsibility to administer the provisions of the Individuals with
Disabilities Education Act. Inherent throughout the Act is the paramount
importance of schools to provide appropriate instruction to each student
with a disability according to the design of that student's unique
individualized education program. By definition, a student's
individualized education program sets the course of study for that
student, including curriculum, instructional strategy, assessment,
service supports, and educational schedule, including anticipated
graduation. Within North Dakota law, services to students with
disabilities may extend beyond the traditional twelve years up to the
age of 21 inclusive. In such circumstances, a student with a disability
may properly graduate, according to the dictates of the student's
individualized education program, at age 21, several years beyond a
traditional graduation that, for a typical student, may be completed
within four years of entering high school.

To ensure that schools properly exercise their instructional duties
according to a student's individualized education program, the
Department of Public Instruction monitors school's compliance with the
provisions of the IDEA, including the proper development and
administration of a student's individualized education program. The
Department monitors graduation rates of all students with disabilities,
including those students whose graduation rates extend to age 21. The
State provides oversight on all services provided to students with
disabilities, including the proper conclusion of their services and the
bestowal of graduation at a time prescribed within the student's
individualized education program.

Given the high educational standards and service schedules set forth
within a student's individualized education program and the civil rights
granted to students with disabilities to receive the full benefit of
these standards and service schedules, it is incumbent on the State to
offer every support to schools to provide the full benefit of
instruction to all students with disabilities, regardless of the
duration of their education. It is likewise incumbent on the State to
eliminate any barriers that might impede or otherwise deter schools from
properly administering their duties to all students, regardless of
disability status. This concern includes the bestowal of a standard
graduation on students with disabilities, whose individualized education
programs require a high school instruction period that extends beyond
the traditional four years. Any policy that places pressures on schools
to divert their full attention on the needs of students with
disabilities must be reviewed and amended accordingly.

It is to this aim, to eliminate any impediment to the proper delivery of
high school instruction to some students with disabilities, that the
Department of Public Instruction allows for the extension of the
standard graduation date beyond the traditional four years for students
with disabilities whose individual education programs identify such a
need. This extension meets the spirit of the Individuals with
Disabilities Education Act and to advance a considered policy that
encourages the properly paced delivery of instruction for all students
with disabilities.

Based on the above stated principles, but uniquely referenced to limited
English proficient students, the Department of Public Instruction
additionally extends the standard graduation provision for those LEP
students identified by a school's LEP service team as requiring
additional time to achieve the full benefit of their high school
education. Any extended standard graduation schedule must be documented
in a service plan for that student, signed by the members of the LEP
team, and submitted to the Department as partial confirmation of the
request.

The Department stipulates that this privilege of an extended standard
graduation date must be documented within each student's individualized
education program or LEP service program and that all programs will be
monitored for proper compliance, according to monitoring protocols. The
Department will require districts to submit evidence of any extension of
a standard graduation in order to confirm its proper use. The Department
will maintain records regarding incidence rates for any such claims;
however, no student identifiable reports will be issued. The State
stipulates, furthermore, that any awarded extension of a standard
graduation schedule will culminate in the award of a standard diploma to
students with individualized education programs and to LEP students with
a LEP service program.

*Graduation Indicator, Transition Extension*

To determine the adequate yearly progress graduation indicator for
2004-05, the State will calculate graduation rates according to the
current definition, which has been applicable for determinations for
2001-04, within Section 7.1 of the State Accountability Plan. The State
will delay the implementation of the anticipated graduation rate
formula, originally set for implementation for 2004-05, until the
2005-06 school year. At that point determinations will be referenced on
graduation data collected on June 30 of the previous year and will allow
for the disaggregated reporting of subgroups as stipulated in the State
Accountability Plan.

+---------------------+-----------------------+-----------------------+
| ## CRITICAL ELEMENT | **EXAMPLES FOR**      | **EXAMPLES OF**       |
|                     |                       |                       |
|                     | **MEETING             | ***NOT* MEETING       |
|                     | REQUIREMENTS**        | REQUIREMENTS**        |
+=====================+=======================+=======================+
| 1.  What is the     | State defines the     | State has not defined |
|     State's         | additional academic   | an additional         |
|     additional      | indicators, e.g.,     | academic indicator    |
|     academic        | additional State or   | for elementary and    |
|     indicator for   | locally administered  | middle schools.       |
|     public          | assessments not       |                       |
|     elementary      | included in the State |                       |
|     schools for the | assessment system,    |                       |
|     definition of   | grade-to-grade        |                       |
|     AYP? For public | retention rates or    |                       |
|     middle schools  | attendance rates.[^9] |                       |
|     for the         |                       |                       |
|     definition of   | An additional         |                       |
|     AYP?            | academic indicator is |                       |
|                     | included (in the      |                       |
|                     | aggregate) for AYP,   |                       |
|                     | and disaggregated (as |                       |
|                     | necessary) for use    |                       |
|                     | when applying the     |                       |
|                     | exception clause to   |                       |
|                     | make AYP.             |                       |
+---------------------+-----------------------+-----------------------+
| **STATE RESPONSE    |                       |                       |
| AND STATE           |                       |                       |
| ACTIVITIES FOR      |                       |                       |
| MEETING             |                       |                       |
| REQUIREMENTS**      |                       |                       |
+---------------------+-----------------------+-----------------------+

The State of North Dakota has adopted the attendance rate for elementary
and middle schools as the additional academic indicator for determining
AYP.

The State has established an attendance target point based on North
Dakota 2001-02 attendance baseline impact data. Refer to **Appendix Z:
North Dakota 2001-02 Attendance Impact Data** for a summary of the
impact data. The State has set the target attendance rate at the second
standard deviation below the norm of ranked district attendance rates.
Any district with an attendance rate lower than this target point will
be identified for not making Adequate Yearly Progress. This target point
will remain as the State definition for attendance throughout the
duration of the 2001-2014 school years. Based on the State's attendance
rate definition, the State attendance target point has been set at 93%.

Attendance rate is defined as the aggregate days of attendance in a
school or school district divided by the aggregate days of enrollment.
The attendance rate is included in the aggregate for AYP. Attendance
data are collected through the State's ADM (average daily membership)
reporting system.

*Attendance Indicator, Transition Extension*

To determine the adequate yearly progress attendance indicator for
2004-05, the State will calculate attendance rates according to the
current definition within Section 7.2 of the State Accountability Plan.
To allow for a successful transition to previous year reporting,
attendance rate determinations for 2004-05 will reference data collected
on June 30, 2004. This will result in 2004-05 attendance determinations
that are identical to the 2003-04 attendance determinations; however, no
school or district will be held to a double-jeopardy effect. Schools
previously identified in 2003-04 as not making adequate yearly progress
for attendance will be held harmless with any subsequent 2004-05
identification. All reports will indicate that the attendance data
references previous year data; no "did not make adequate yearly
progress" designations will be reported. Effective for the 2005-06
school year, determinations will be referenced on attendance data
collected on June 30 of the previous year, data that would not have been
previously reported.

+---------------------+-----------------------+-----------------------+
| ## CRITICAL ELEMENT | **EXAMPLES FOR**      | **EXAMPLES OF**       |
|                     |                       |                       |
|                     | **MEETING             | ***NOT* MEETING       |
|                     | REQUIREMENTS**        | REQUIREMENTS**        |
+=====================+=======================+=======================+
| 2.  Are the State's | State has defined     | State has an academic |
|     academic        | academic indicators   | indicator that is not |
|     indicators      | that are valid and    | valid and reliable.   |
|     valid and       | reliable.             |                       |
|     reliable?       |                       | State has an academic |
|                     | State has defined     | indicator that is not |
|                     | academic indicators   | consistent with       |
|                     | that are consistent   | nationally recognized |
|                     | with nationally       | standards.            |
|                     | recognized standards, |                       |
|                     | if any.               | State has an academic |
|                     |                       | indicator that is not |
|                     |                       | consistent within     |
|                     |                       | grade levels.         |
+---------------------+-----------------------+-----------------------+
| **STATE RESPONSE    |                       |                       |
| AND STATE           |                       |                       |
| ACTIVITIES FOR      |                       |                       |
| MEETING             |                       |                       |
| REQUIREMENTS**      |                       |                       |
+---------------------+-----------------------+-----------------------+

Attendance rates are widely recognized as a valid indicator of student
success. Attention to student achievement in addition to attendance and
graduation rates offers a balanced presentation of key student
performance indicators.

The State's ADM reporting system provides a reliable means of
identifying students and monitoring student attendance rates.

The State of North Dakota has established a definition for the minimum
number of students in a subgroup for both reporting and accountability
purposes. The definition is consistent with the minimum number
identified within Principle 9. The State has established a test of
statistical significance for the method of determining a minimum number
within a given population and referenced to the established measurable
objective, participation rate, graduation rate, or attendance rate.
Refer to Principle 9.1 for a detailed overview of this method of
statistical significance.

*Safe Harbor for Academic Indicators*

If a high school whose calculated graduation rate is below the
state-defined measurable objective, as determined by the method
prescribed in Section 7.1, reports sufficient improvement in the
graduation rate to meet the terms of safe harbor, then the high school
will be identified as having made adequate yearly progress for the
graduation rate. Safe harbor for the graduation rate is defined as a ten
percent reduction in the not-graduated rate.

If an elementary or middle school whose calculated attendance rate is
below the state-defined measurable objective, as determined by the
method prescribed in Section 7.2, reports sufficient improvement in the
attendance rate to meet the terms of safe harbor, then the elementary or
middle school will be identified as having made adequate yearly progress
for the attendance rate. Safe harbor for the attendance rate is defined
as a ten percent reduction in the absence rate.

No statistical significance test will be conducted during the
calculation of safe harbor. The safe harbor provision for graduation and
attendance will be applied in the determination of adequate yearly
progress for school districts and the State.

**PRINCIPLE 8. AYP is based on reading/language arts and mathematics
achievement objectives.**

+----------------------+-----------------------+-----------------------+
| ## CRITICAL ELEMENT  | **EXAMPLES FOR\       | **EXAMPLES OF**       |
|                      | MEETING               |                       |
|                      | REQUIREMENTS**        | ***NOT* MEETING       |
|                      |                       | REQUIREMENTS**        |
+----------------------+-----------------------+-----------------------+
| 1.  Does the state   | State AYP             | State AYP             |
|     measure          | determination for     | determination for     |
|     achievement in   | student subgroups,    | student subgroups,    |
|     reading/language | public schools and    | public schools and    |
|     arts and         | LEAs separately       | LEAs averages or      |
|     mathematics      | measures              | combines achievement  |
|     separately for   | reading/language arts | across                |
|     determining AYP? | and mathematics.      | reading/language arts |
|                      | [^10]                 | and mathematics.      |
|                      |                       |                       |
|                      | AYP is a separate     |                       |
|                      | calculation for       |                       |
|                      | reading/language arts |                       |
|                      | and mathematics for   |                       |
|                      | each group, public    |                       |
|                      | school, and LEA.      |                       |
+----------------------+-----------------------+-----------------------+
| **STATE RESPONSE AND |                       |                       |
| STATE ACTIVITIES FOR |                       |                       |
| MEETING              |                       |                       |
| REQUIREMENTS**       |                       |                       |
+----------------------+-----------------------+-----------------------+

The State of North Dakota stipulates that the State will measure
achievement in reading and mathematics separately for determining AYP.

North Dakota state law (NDCC 15.1-21-08) places responsibility with the
State Superintendent for the administration of State assessments that
are aligned to the State's content standards in reading and mathematics
(refer to **Appendix B: North Dakota Century Code citations** or
reference the North Dakota Century Code at the following web site,
<http://www.state.nd.us/lr/cencode/T151C21.pdf> ). State law requires
that the assessments be administered to at least one grade level
selected within each of the following grade spans: grades three through
five; grades six through nine; and grades ten through twelve. The North
Dakota Department of Public Instruction has developed and administers
assessments at grades 4, 8, and 12 to correspond with the State's
content standards.

State law requires that the State assessments compile aggregated results
and disaggregated results. The State assessments must compile student
achievement data that allows for a comparison of classrooms within a
given school and school district, schools within the district, and
school districts within the state. The test scores must also allow for
comparisons based on students' gender, ethnicity, economic status,
service status (i.e., migrant, LEP, disability), and assessment status
(i.e., enrollment and participation status), unless doing so enables the
identification of any student. (Refer to NDCC 15.1-21-08 within
**Appendix B: North Dakota Century Code citations** or reference the
North Dakota Century Code at the following web site,
[http://www.state.nd.us/lr/cencode/T151C21.pdf](http://www.state.nd.us/lr/assembly/57-2001/cencode/CCT15x1.pdf)
).

> The State will produce all district- and school-level reports for
> districts and schools regarding their respective student achievement
> levels in both reading and mathematics separately. These profile
> reports will include both aggregated and disaggregated student
> achievement data. Refer to **Appendix O: North Dakota Sample School
> Report Card and Profile** for an illustration of the content of these
> student achievement profiles. Districts may use these profile reports
> as the foundation for their locally produced profiles. These
> State-generated reports will offer quality assurances regarding the
> generation of any district achievement data.
>
> The State requires all districts to disseminate student achievement
> profiles to their communities. This mandate is required as an element
> of their receipt of federal funds. Refer to **Appendix S: Consolidated
> Application Certification and Assurances** for the State assessment
> requirement for receipt of federal ESEA funding. To assure compliance
> with this provision for the development and dissemination of
> performance profiles per Title I funding, the Department will require
> timely and comprehensive reports as a condition of receiving
> uninterrupted Title I funds. Further, evidence of these profiles will
> be one of the criteria in the Department's Title I monitoring program.

The State currently is reviewing its ESEA and accreditation monitoring
policies. The State is pursuing an amendment to its monitoring
requirements that would mandate districts to produce evidence regarding
the dissemination of achievement profiles to their communities. Monitors
would check for the production and dissemination of any such achievement
profiles. Any failures to disseminate these profiles would be identified
as a compliance violation of the school's and district's ESEA compliance
agreement. Refer to **Appendix S: Consolidated Application Certification
and Assurances** for the State Assessment requirement for receipt of
federal ESEA funding. A school or district may be sanctioned for any
compliance violation of their ESEA assurances agreement.

**PRINCIPLE 9. State Accountability System is statistically valid and
reliable.**

+----------------------+-----------------------+-----------------------+
| ## CRITICAL ELEMENT  | **EXAMPLES FOR\       | **EXAMPLES OF**       |
|                      | MEETING               |                       |
|                      | REQUIREMENTS**        | ***NOT* MEETING       |
|                      |                       | REQUIREMENTS**        |
+======================+=======================+=======================+
| 1.  How do AYP       | State has defined a   | State does not have   |
|     determinations   | method for            | an acceptable method  |
|     meet the State's | determining an        | for determining       |
|     standard for     | acceptable level of   | reliability (decision |
|     acceptable       | reliability (decision | consistency) of       |
|     reliability?     | consistency) for AYP  | accountability        |
|                      | decisions.            | decisions, e.g., it   |
|                      |                       | reports only          |
|                      | State provides        | reliability           |
|                      | evidence that         | coefficients for its  |
|                      | decision consistency  | assessments.          |
|                      | is (1) within the     |                       |
|                      | range deemed          | State has parameters  |
|                      | acceptable to the     | for acceptable        |
|                      | State, and (2) meets  | reliability; however, |
|                      | professional          | the actual            |
|                      | standards and         | reliability (decision |
|                      | practice.             | consistency) falls    |
|                      |                       | outside those         |
|                      | State publicly        | parameters.           |
|                      | reports the estimate  |                       |
|                      | of decision           | State's evidence      |
|                      | consistency, and      | regarding             |
|                      | incorporates it       | accountability        |
|                      | appropriately into    | reliability (decision |
|                      | accountability        | consistency) is not   |
|                      | decisions.            | updated.              |
|                      |                       |                       |
|                      | State updates         |                       |
|                      | analysis and          |                       |
|                      | reporting of decision |                       |
|                      | consistency at        |                       |
|                      | appropriate           |                       |
|                      | intervals.            |                       |
+----------------------+-----------------------+-----------------------+
| **STATE RESPONSE AND |                       |                       |
| STATE ACTIVITIES FOR |                       |                       |
| MEETING              |                       |                       |
| REQUIREMENTS**       |                       |                       |
+----------------------+-----------------------+-----------------------+

The State of North Dakota stipulates that all AYP determinations meet
the State's standard for acceptable validity and reliability. The State
has adopted, with the technical assistance of Richard Hill of the
National Center for the Improvement of Educational Assessments, a test
for statistical significance that establishes a balance between systemic
validity and reliability.

# Background

Each state must create an accountability system in response to the
requirements of the No Child Left Behind Act (NCLBA). Among the
requirements is the determination of whether schools and subgroups
within the school either have achieved a particular percentage of
students at the proficient level or higher (i.e., met the "status"
requirement) or have improved their percentage of students achieving at
the proficient level or higher over the prior year's level (i.e., met
the "improvement" or "safe harbor" requirement). If a school or a
subgroup fails one or both those tests, it is identified as not making
Adequate Yearly Progress (AYP). Results for subgroups are not required
to be included "in a case in which the number of students in a category
is insufficient to yield statistically reliable information." States are
left to determine what that proper number might be.

One issue to be addressed is how low reliability can go before it is
"insufficient." If the stakes for identification are lower, then a
fairly low level of reliability might be acceptable. If the stakes for
identification are higher, however, then one would want to be fairly
certain that a school had been correctly classified before applying the
proscribed consequences to the school. In NCLBA, annual judgments are
made about whether a school has made AYP. If a school is identified as
not making AYP two years in a row, a series of proscribed consequences
is set in motion. So, unquestionably, one would want the decision about
whether a school had not made AYP two years in a row to be highly
reliable. Given the effects of identification for not making adequate
yearly progress, a reasonable argument can be constructed for wanting a
reliable decision to be made every year for every school.

# Selecting a Fixed N

There exists an approach that requires a school or subgroup to have a
particular number of assessed students (for example, 30) in order to be
considered for AYP identification, regardless of the performance of the
school or subgroup. This appears to be an approach that will not work
well for either measuring status or safe harbor. If a certain fixed
number is chosen, schools will not be directly accountable for subgroups
with fewer than that number (i.e., those subgroups will be included in
the school's total score, but the performance of that subgroup by itself
will not be considered). No matter how small a number is chosen, this
will exclude many subgroups, leading to an incomplete look at the
performance of the school. Thus, one could argue that a number like 30
is *far* too large a number---a requirement that subgroups meet this
minimum N will eliminate the vast majority of subgroups in North Dakota.

On the other hand, the results for schools and subgroups are supposed to
be "statistically reliable." That would mean, at a minimum, that if a
subgroup causes a school to not make AYP, another sample of students in
that subgroup drawn for that school would be likely to have the same
result. While reasonably modest numbers of students often (but not
always) can be used to reliably determine whether a subgroup has met the
status requirement, it takes large numbers (hundreds of students) to
reliability detect whether a school has made sufficient improvement.

So, on the one hand, a state should pick a fairly small N for purposes
of validity (say, certainly something no larger than 10), but needs a
very high N (say, 300 or more) for purposes of reliability. A value that
provides reasonable validity is wholly inadequate for reliability
purposes; conversely, a value that provides reasonable reliability is
wholly inadequate for validity purposes. A figure between those two is
largely inadequate for *both* purposes. This is the reason that the
selection of any given fixed value for minimum N remains problematic.
Until one looks carefully at the issue, one presumes that a modest fixed
N will be a reasonable compromise between reliability and validity. A
careful look tells us that choosing any value is wholly inadequate for
at least one of the two concerns, if not both. In short, there isn't a
reasonable answer to this dilemma. One is not faced with a reasonable
balancing of concerns over reliability and validity when arbitrary N
values are considered; any purported answer will be clearly wrong for at
least one of the two.

Given that one cannot have validity without reliability, it would be
justifiable for a state to select a minimum N of 300. Granted, an N of
this size will eliminate virtually every subgroup in a state,
essentially eliminating this aspect of NCLBA. But such an N would at
least ensure that decisions would be sufficiently reliable.

**Selecting an Alternative Method: the North Dakota Model**

An alternative to selecting a fixed N is to run a test of statistical
significance. That way, schools and subgroups that are far from the
standard do not need to have a large N for a reliable decision to be
made. For example, suppose the standard for a state is 50 percent
proficient. If no students in a subgroup are proficient, a reliable
decision (i.e., one that has less than a 1 percent probability of
misclassifying the subgroup) that the subgroup does not meet the status
test can be made if there are just seven students in the subgroup. That
is, if 50 percent of the students in a subgroup are proficient, there is
less than 1 chance out of 100 that no students within a sample of seven
would be proficient. Thus, in cases where results are extremely low, the
inadequate performance of the subgroup can be reliably detected even
with small Ns. On the other hand, if 499 out of 1000 students were
proficient, one would not be certain that another sample of students
from that same subgroup wouldn't have at least 50 percent proficient.
So, this system will select a group that is far away from the standard
even if the group is small, but will not select a group that is very,
very close to the standard even if the group is quite large. Not only is
this a better application of statistics than the fixed N approach, it
also is more fair and valid. Certainly, one would want to identify and
target resources to very low-achieving subgroups before doing the same
to subgroups that are very close to the state's standard.

However, even this system cannot solve the problem of measuring subgroup
improvement. Measuring improvement over one year is difficult to do
because the amount of improvement desired is small, relative to status
(10 percentage points or less) and the measurement is made between two
samples of students rather than one. A subgroup's status might be 50
percentage points away from the state's standard; as such, detecting
differences that large can be done with samples as small as seven (as
shown above). But a subgroup's required annual improvement can be no
larger than 10 percent, and is often considerably smaller than that.
Given that measurement of improvement is made by comparing one sample to
another, each with its own sampling error, reliable judgments require,
at a minimum, scores of students, and more typically require hundreds of
students.

Take this specific example. Suppose a subgroup has 50 percent of its
students passing one year. To make AYP through the safe harbor
provision, the subgroup must improve to 55 percent passing the following
year. If the subgroup really does improve its performance by 5
percentage points, how many students will it take, each year, to have at
least a 95 percent probability that the subgroup's performance will
*increase* over the previous year, much less go up the required amount?

A school with 50 percent of its students failing is supposed to reduce
that percentage by 5 in one year, and a z-score of 2.33 cuts off the
upper 1 percent of the area under a normal curve. So, to reject the null
hypothesis at the .01 level one-tailed, the standard error of the
difference can be no bigger than 5/2.33, or 2.15.

Now, suppose we hypothesize that a school has *N* students in each of
two years, and its proportion of students passing goes from 50 percent
passing the first year to 55 percent the second year. The equation we
need to solve is as follows:

> $2\text{.}\text{15} = \sqrt{P1 \ast Q\frac{1}{N} + P2 \ast Q\frac{2}{N}}$,
> or
>
> $$2\text{.}\text{15} = \sqrt{\frac{\text{50} \ast \text{50}}{N} + \frac{\text{55} \ast \text{45}}{N}}$$

Solving for *N* produces a result of 1076.25. Rounding up means that an
N of 1,077 students per year is required to have a 99 percent
probability that a school's observed scores will increase from one year
to the next if its true percentage of proficient students increases from
50 to 55.

The results above assume that the two samples are independent (as would
be the case, for example, if testing were done at just one grade and the
same grade was tested two consecutive years). If the results of the two
years are not independent but are positively correlated, the required N
drops. This would be the case if, for example, we followed the progress
of a group of students from one year to the next. In that case, the
standard error of the difference scores is computed as follows:

$\sigma_{P1 - P2} = \sqrt{\sigma_{P1}^{2} + \sigma_{P2}^{2} - 2{r\sigma}_{P1}\sigma_{P2}}$

Now, suppose we continued our example from above (determine the standard
error of difference scores when a school's true score changes from 50
percent passing to 55 percent), but followed the same cohort of students
from one year to the next. Suppose further that all the students tested
in one year are tested the next, and suppose the correlation between
scores from one year to the next is .7 (a typical intra-school
student-level correlation of scores across one year). We still need a
standard error of the difference of 2.15, but now the equation is:

> $$2\text{.}\text{15} = \sqrt{P1 \ast Q\frac{1}{N} + P2 \ast Q\frac{2}{N} - 2 \ast \text{.}7\frac{\sqrt{P1 \ast Q1 \ast P2 \ast Q2}}{N}}$$

Solving for *N* produces a result of 245. So, even if the same students
are tracked from one year to the next, it takes a very large number to
be 99 percent certain that the observed results from one year to the
next will increase if the percent proficient goes from 50 to 55.

# Specifics of North Dakota's Approach

Schools or subgroups will be identified as not making AYP if their
status score is insufficiently high, and failing that, if their
improvement is insufficient. This section will describe in more detail
how each of those judgments will be made.

## 

## Status

North Dakota will establish a required statewide status score equal to
the percentage of students proficient or higher in the 20^th^ percentile
school in the state, as required by NCLB. Call that value π~0~. Once
that "starting point" has been established, each subgroup will pass the
status test if the null hypothesis that the proportion of students for
that school is equal to π~0~ cannot be rejected at the .01 level.

[Exact probabilities vs. normal approximation]{.underline}. The exact
probability that the null hypothesis can be rejected, given *X* students
proficient out of *N* tested and a population proportion of π~0~, is:

> $$P(X \leq X_{0}|\pi_{0},N) = \sum_{i = 0}^{X_{0}}C_{i}^{N}\pi^{i}(1 - \pi)^{N - i}$$

For example, if *N* = 3 and π~0~ = .5, the probability that *X* = 0 is
.125 and the probability that *X* = 1 is .375. The probability that *X*
≤ 1 is .5.

To further illustrate, suppose the starting point for North Dakota is 40
percent proficient, and suppose a certain subgroup of 10 students has 2
proficient students. The observed percentage of students passing in the
subgroup is 20, which is less than the required value of 40. But would
one reject the null hypothesis that the true population percentage for
that subgroup is 40? The test for the subgroup would proceed as follows:

The probability of having 0 students proficient out of 10 if π~0~ = .40
is .0001.

The probability of having 1 student proficient out of 10 if π~0~ = .40
is .0016.

The probability of having 2 student proficient out of 10 if π~0~ = .40
is .0106.

Therefore, the probability of 2 or fewer students proficient out of 10
if π~0~ = .40 is .0123. Since this value is greater than .01, this
subgroup would *not* be identified as not having met the AYP status
standard. If, on the other hand, only 1 student had been proficient, the
subgroup *would* be identified as not having met the AYP status
standard, and therefore would have to meet the improvement standard to
avoid having the school identified as failing to make AYP.

Computing these exact probabilities is computationally intense. Before
today's super-fast computers, the amount of computation required was so
extreme that often these exact probabilities were estimated through
normal approximation. With that method, one first computes the standard
error of the mean as $\sqrt{\pi_{0}(1 - \pi_{0}\frac{)}{N}}$, computes a
z-score, and then determines the probability of a z-score that extreme
or more in a table of normal probabilities.

Taking our second example of 1 student proficient out of 10 with π~0~ =
.40, we would compute the standard error as .1549. In that case, the
observed proportion of proficient students (.10) would yield a z-score
of (.10 - .40) / .1549, or --1.94. The probability of observing a
z-score of that value or lower is .026. Thus, in this case, the normal
approximation is not a very good approximation of the exact (correct)
probability. We would have not rejected the null hypothesis in this
case, but as was shown above, we should have. A general rule of thumb is
that the normal approximation works well if *pN* \> 5. In this case, it
equals 1, so the normal approximation does not approximate well.

It would be reasonable to compute exact probabilities only for the most
extreme cases and use the normal approximation for the remainder of the
calculations. In fact, until recently, that was fairly common practice.
However, since computers can make the complex calculations for the exact
probability quickly and that using one method for all calculations leads
to easier programming than using multiple methods, North Dakota proposes
to make the exact calculations for all schools and subgroups. Given that
there will be schools and many subgroups in North Dakota for which *pN*
\< 5 (and therefore many cases in which the exact calculations would
need to be done anyway), this is by far the most practical approach for
assessing status. However, the calculations for safe harbor are much
more complex, and therefore the normal approximation will be used for
those tests.

For the purposes of calculating AYP, the State will reference a school's
and LEA's current year's achievement results in addition to two previous
years' achievement results. Additionally, the State will reference the
combined effect across all grades tested within the school and LEA. The
addition of a total of three years data and the combined effect of all
grades will increase the N value and ensure greater reliability.

[Choosing an *alpha* level]{.underline}. North Dakota has elected to use
an *alpha* =0.01 level to conduct these tests of statistical
significance. This level of confidence will be applied to each subgroup
tested for achievement, participation rate, graduation rate, and
attendance rate within each school, each district, and the State.

The selection of an alpha=0.01 assures a reasonably high level of
confidence given the multiple tests conducted within the process of
determining AYP. However, these tests applied on the various subgroups
are not independent. Reading and math are well correlated, and some of
the subgroups are so highly inter-correlated as to be assessing
virtually the same students because of their cross-over into other
subgroups. Thus, for most schools, the probability of an error across
all the tests done is likely to be below .05, which is the standard
often used in educational research.

To balance all factors and mitigate the combined effect of different
tests, the State has adopted an alpha=0.01 as the means to ensure highly
reliable identifications.

## Safe Harbor: A Provisional Proposal and Study

*Provisionally Adopting the ESEA Safe Harbor Definition.* The State
proposes to conduct a study of the effects of the binomial distribution
on the determination of safe harbor. Until this study of the binomial
distribution's application on safe harbor is completed, the State of
North Dakota will adopt the definition of safe harbor established within
ESEA section 1111. In the event that the study of the binomial
distribution improvement model produces findings that support the use of
the binomial distribution, the State reserves the right to propose the
replacement of the ESEA definition of safe harbor with another that
applies the binomial distribution. The State Superintendent will
communicate with the U.S. Department of Education regarding any proposed
changes affecting the State's AYP safe harbor definition.

*Conducting a Study of Statistical Reliability on Safe Harbor.* The
approach described in the sections above will work well for assessing
status. In contrast to selecting a fixed N, where many subgroups would
pass AYP regardless of performance, only the very smallest subgroups
will receive this automatic pass in North Dakota. Subgroups of even
modest size will need to have at least some reasonable portion of their
students proficient in order to pass the status test. On the other hand,
by selecting an *alpha*-level of 0.01, North Dakota assures that those
subgroups identified as not having met AYP would be very likely to have
a value lower than the state-required amount even if another sample of
students were drawn. This approach provides an excellent balance between
validity (accountability for all subgroups) and reliability (assuring
that those subgroups identified have not been so identified simply on
the basis of random fluctuation).

Assuring this same appropriate balance for measuring safe harbor will
not be as easy. The amount of improvement required each year is small
relative to the standard error for most groups. As a result, it is
possible to reliably detect the required amount of annual improvement
only for very large groups, as was shown in an earlier section.
Measuring improvement reliably will require a considered study. Because
the statistical test for safe harbor is conducted on each element of
performance independently and because there is no cumulative effect, it
is proper to maintain an alpha=0.01 within the safe harbor test for
performance.

*Conducting a Study of the Safe Harbor Model.* The proper application of
the binomial distribution to measure the annual improvement of schools
and districts for safe harbor requires a careful analysis in order to
ensure the vital interests of validity and reliability are honored. A
statistical reliability model for safe harbor remains largely
theoretical; it has not been tested or validated. The application of the
binomial distribution within the status test can be demonstrated as
effective, practical, and balanced; as such, the application of the
binomial distribution within the status measurement has been used and
validated successfully. In the absence of any demonstrable data or
validation and in order to sufficiently analyze the effect of the
binomial distribution on the number and types of identifications and
protections it allows, the Department of Public Instruction will conduct
a study of the binomial distribution within safe harbor prior to its
adoption and implementation. The State will proceed to use the binomial
distribution for determining status. The Department of Public
Instruction will access the technical expertise of the National Center
for the Improvement of Educational Assessments and the North Dakota
Technical Advisory Committee to conduct this study. This study, which is
expected to take several months, will provide evidence on the effects of
adopting the binomial distribution for determining safe harbor within
AYP. The State Superintendent will review any recommendations and
communicate these findings to the U.S. Department of Education.

# Supporting Research Study

The ESEA places a high standard upon States to administer an
accountability system that requires, on the one hand, a consistent
pressure to identify low performing schools, and on the other hand, a
counter force that limits any identifications of schools or districts
where insufficient confidence exists. States must provide for a
meaningful and just accountability system. Any fair accountability
system must adhere to foundational principles, rest upon objective and
universally applied rules, evidence balance between validity and
reliability, evoke a sense of confidence among schools and the public,
and demonstrate administrative practicality.

The North Dakota accountability model is as a viable system that
supports these principles. After careful consideration of the various
options for implementing a valid and reliable accountability system, the
Department of Public Instruction asserts that the binomial distribution
model offers a comprehensive and balanced approach to accomplish these
aims. Its principles are sound. The State's supporting impact data
demonstrate that this statistical model constitutes a viable
accountability system.

The Department of Public Instruction will conduct an ongoing study of
the effects of the binomial distribution model in ensuring the
administration of a valid and reliable accountability system. This
study, to be conducted based on 2001-2004 data for final release by
September 2004, will assess the impact of the statistical model's
vitality, including the selected alpha=0.01 level, in balancing the
dynamic factors of identification and confidence. This study will assess
the impact of the binomial distribution on the identification patterns
for achievement, safe harbor, participation rates, graduation rates, and
attendance rates for subgroups, schools, and districts. This study will
assess impact data related to Type I errors and Type II errors.
Additionally, this study will analyze the level of confidence placed in
the accountability system by schools, parents, policymakers, and the
public. The Department of Public Instruction will submit any findings to
the U.S. Department of Education for further review. It is in the best
interest of the State and the advancement of quality accountability
systems that the North Dakota Accountability System be peer reviewed for
the purposes of improvement and possible wider dissemination.

+----------------------+-----------------------+-----------------------+
| ## CRITICAL ELEMENT  | **EXAMPLES FOR\       | **EXAMPLES OF**       |
|                      | MEETING               |                       |
|                      | REQUIREMENTS**        | ***NOT* MEETING       |
|                      |                       | REQUIREMENTS**        |
+======================+=======================+=======================+
| 2.  What is the      | State has established | State does not have a |
|     State\'s process | a process for public  | system for handling   |
|     for making valid | schools and LEAs to   | appeals of            |
|     AYP              | appeal an             | accountability        |
|     determinations?  | accountability        | decisions.            |
|                      | decision.             |                       |
+----------------------+-----------------------+-----------------------+
| **STATE RESPONSE AND |                       |                       |
| STATE ACTIVITIES FOR |                       |                       |
| MEETING              |                       |                       |
| REQUIREMENTS**       |                       |                       |
+----------------------+-----------------------+-----------------------+

The State of North Dakota has established its accountability system upon
assessments that are documented as valid and reliable measures of
student achievement. Validity denotes the appropriateness,
meaningfulness, and usefulness of any inferences made from an assessment
tool. As such, validity addresses whether an assessment truly assesses
what it purports to assess and whether it will lead any user to an
appropriate understanding and application of results. The State's
Assessment System imbeds the elements of content validity (alignment to
State content standards), item design validity, related assessment
validity, and consequential validity.

\(a\) Content validity (alignment to State content standards).

The activities conducted by the State to assure that all test items are
aligned to the State's content standards. This review of content
coverage, conducted by North Dakota teachers, offers assurance that the
State Assessment indeed does assess student achievement in terms of the
State standards in breadth. North Dakota teachers affirm that the State
Assessment does assess the breadth of the standards and that each
standard is covered sufficiently to generate meaningful results. Each
standard is identified and is supported by a sufficient number of items
to offer enough data to reach a valid indication of a student's
performance.

\(b\) Test design validity.

The construction of individual test items and the test as a whole are
critical elements of validity. Additionally, the effects of any test
item or the test as a whole on subgroups of students similarly becomes
an element of validity. The Department of Public Instruction has
contracted with CTB/McGraw-Hill to develop and administer an augmented,
multiple measures assessment at each respective grade level. These
assessments meet high technical specifications to assure validity,
reliability, and comparability, thereby offering confidence in the
application of any information gained through the use of the
assessments.

Refer to Appendix V: North Dakota State Assessment, Test Specifications
for a summary review of the technical specifications incorporated within
the State Assessment. This summary identifies a variety of factors that
impact test validity and the appropriate use of acquired information.
Refer to pages 1-11 of Appendix W: North Dakota State Assessment, 2002
Preliminary Technical Report, for actual impact data supporting the
overall validity of the North Dakota State Assessment.

\(c\) Related assessment validity.

An inherent consideration confirming the validity of any assessment is
how well it correlates with other assessment tools of comparable
quality. To quantify comparability among differing assessment tools
requires an ability to directly link individual student achievement
among different assessment tools. To do so requires a data analysis and
reporting tool capable of managing such linkages among different
databases.

The State of North Dakota has never possessed the ability to track the
performance of individual student or system performance levels in a
meaningful manner based on quality disaggregated data analysis. The
State has never owned, developed, or accessed a single, statewide
student data system. This absence of a statewide data system has
resulted in an inability to access accurate, meaningful information
regarding student demographics, student achievement levels, school
performance, teacher quality indicators, systemic improvements, or
statewide systems monitoring. In the area of assessment, this absence of
a statewide data system has resulted in an inability to sufficiently
study correlations of student achievement among assessment tools or
instructional methods of varying quality.

To eliminate these deficiencies and to advance meaningful school
improvement measures, the Department of Public Instruction has
contracted with TetraData Corporation, in October 2002, to develop and
administer a statewide data analysis and reporting system. This data
analysis and reporting system will allow for the linkage of various
databases in order to track individual student, staff, and institutional
achievement levels, including the correlation of student achievement
across different assessment tools.

Included in this capability, is the capacity of the State to compile,
compare, and validate student achievement on all grade-level State
assessments and to compare these with other assessment tools or
classroom grading. Official student files will be linked to State
Assessment files that will, in turn, be linked to other assessment tools
and classroom grading in order to conduct correlation studies. This will
offer the State an auditing capability that will approach 100% accuracy,
thereby ensuring a high degree of confidence in any correlation study.
Refer to **Appendix R: TetraData Data Analysis and Reporting System
Summary** for an overview of the project. The system will be functional
statewide by March 2003.

With the development of this statewide data analysis and reporting
system, the State will be able to monitor and confirm the contextual
validity of its State Assessment.

\(d\) Consequential validity.

The fundamental purpose for the administration of any assessment is to
learn how well individual students and populations of students perform
against a standard. The intended consequence of such learning is to
apply this knowledge to the improvement of instruction for each student
individually and for all students collectively and by subgroup.
Consequential validity means that the State Assessment is designed in
such a manner as to accomplish this aim with end users. Does the
assessment lend itself to reaching correct conclusions from the data?

North Dakota has never conducted follow up studies to record the
application of its assessments to enhance instruction. This analysis has
never been attempted because of the difficulty in accurately measuring
the effort of schools to integrate assessment data into school
improvement or, more importantly, measuring the effect of such
improvement efforts on students' achievement. Beginning with the 2001-02
baseline data generated through the first administration of the
standards-based North Dakota State Assessment, the State of North Dakota
will begin a process of confirming the contextual validity of its State
Assessment.

The Department of Public Instruction has contracted with TetraData
Corporation, in October 2002, to develop and administer a statewide data
analysis and reporting system. This data analysis and reporting system
will allow for the linkage of various databases in order to track
individual student, staff, and institutional achievement levels,
including the correlation of student achievement across different
assessment tools. Additionally, the Department of Public Instruction
will contract with an independent, outside contractor to conduct a study
of how well schools use the data from the State Assessment to improve
standards-based instruction.

This study will use survey tools with teachers and administrators to
assess the degree that data from the state assessment are used for
overall school improvement, especially instruction. This study will also
survey the efforts of school personnel to reform instructional
practices. Finally, this study will use the data linkage and analysis
functions within the TetraData application to measure actual student
achievement. Because student cohorts can be linked to teachers and
schools that engage in reform activities, meaningful measurements can be
derived on the effects of these efforts. Specific attention can be made
to track the broad effects of using State Assessment data to improve
instructional areas identified as deficient in the data.

The State seeks to implement a valid assessment and accountability
system. Evidence of such an effort will be marked by the State's ability
to monitor the alignment of its assessment to State content standards,
to assure high technical specification in the development of its State
Assessment, to correlate the State Assessment with other outside
assessments and classroom grading, and to assure the meaningful
application of the assessment for school reform.

e\) AYP Identification Method.

Principle 9.1 identifies the State's method of identifying schools and
LEAs for program improvement. This Principle carefully balances the need
to protect the interests of schools and LEAs from misidentification with
the public interest of knowing the overall performance of their schools.
Refer to Principle 9.1 for a thorough analysis of this issue.

f\) Automatic Appeals.

Any school or district that has been identified through the AYP
determination process will automatically receive an appeal to clarify
and correct information within the determination process and to present
extenuating information that may have bearing on the validity or
reliability of the foundational information or the determination process
itself. In the case of AYP decisions regarding schools, the school
district must consider the appeal, with the assistance of the State, and
render a final decision within 30 days, after the submission date of the
appeal. In the event of a district identification, if a district appeals
a decision regarding AYP, the Department of Public Instruction must make
a final determination within 30 days of the date of the appeal. The
State Superintendent will determine all appeals regarding AYP
identification.

The Department of Public Instruction will provide ongoing technical
assistance to districts and schools regarding the AYP determination
process, all program improvement and corrective action activities,
including appeals regarding AYP decisions.

+----------------------+-----------------------+-----------------------+
| ## CRITICAL ELEMENT  | **EXAMPLES FOR\       | **EXAMPLES OF**       |
|                      | MEETING               |                       |
|                      | REQUIREMENTS**        | ***NOT* MEETING       |
|                      |                       | REQUIREMENTS**        |
+======================+=======================+=======================+
| 3.  How has the      | State has a plan to   | State's transition    |
|     State planned    | maintain continuity   | plan interrupts       |
|     for              | in AYP decisions      | annual determination  |
|     incorporating    | necessary for         | of AYP.               |
|     into its         | validity through      |                       |
|     definition of    | planned assessment    | State does not have a |
|     AYP anticipated  | changes, and other    | plan for handling     |
|     changes in       | changes necessary to  | changes: e.g., to its |
|     assessments?     | comply fully with     | assessment system, or |
|                      | NCLB.[^11]            | the addition of new   |
|                      |                       | public schools.       |
|                      | State has a plan for  |                       |
|                      | including new public  |                       |
|                      | schools in the State  |                       |
|                      | Accountability        |                       |
|                      | System.               |                       |
|                      |                       |                       |
|                      | State has a plan for  |                       |
|                      | periodically          |                       |
|                      | reviewing its State   |                       |
|                      | Accountability        |                       |
|                      | System, so that       |                       |
|                      | unforeseen changes    |                       |
|                      | can be quickly        |                       |
|                      | addressed.            |                       |
+----------------------+-----------------------+-----------------------+
| **STATE RESPONSE AND |                       |                       |
| STATE ACTIVITIES FOR |                       |                       |
| MEETING              |                       |                       |
| REQUIREMENTS**       |                       |                       |
+----------------------+-----------------------+-----------------------+

The State of North Dakota has developed a long-term plan to advance
assessment system improvements, assessment system expansion, and
enhancements to the State's accountability system.

I. Assessment System Improvements

\(a\) Assessment development procedural improvements.

The State Superintendent instituted a state-level advisory committee
consisting of LEA and SEA representatives, titled the *Standards,
Assessment, Learning and Teaching (SALT) Team*, and authorized this
committee to advise the Department of Public Instruction on standards
and assessment development committee work. North Dakota's assessment
development protocols currently are being revised by the Department of
Public Instruction with the advise of the SALT Team to incorporate
improvements into the assessment development process and to accommodate
the expansion of current assessments (grades 4, 8, and 12) into grades
3, 5, 6, and 7 and in science in grades 3-8 and 12. Refer to **Appendix
Y: North Dakota Standards and Assessment Development Protocols**
regarding the procedures to be followed for the development and
improvement of state assessments. Completion of the revised State
Assessment Protocols is expected by May 2003.

Department of Public Instruction has adopted certain improvements in the
development of assessment RFPs. These improvements are identified within
the white paper, *Model Contractor Standards and State Responsibilities
for State Testing Programs,* Education Leaders Council, 2002.
Additionally, the Department is considering for adoption several
innovations identified by other States and developed within the white
paper, *State Innovation Priorities for Testing,* Education Leaders
Council. The Department is expected to submit a comprehensive list of
assessment procedure improvements and potential innovations to the State
Superintendent by May 2003.

The State Superintendent, by State law, is responsible for the oversight
of all assessment development and administration duties (refer to
**Appendix B: North Dakota Century Code, Assessment Statutes** regarding
the delineation of State Assessment oversight responsibilities). The
State Superintendent has commissioned the SALT Team as the primary
advisory committee. The State will contract with an outside consultant
to conduct an independent, systematic review of the State Assessment
system and to issue recommendations to the State Superintendent on the
improvement of the system. The Department contracts with CTB/McGraw-Hill
to conduct the development and improvement of the State's Assessment.

\(b\) Ongoing assessment refinement.

The Department of Public Instruction is developing a long-term plan for
the ongoing replacement of test items with additional selective- and
constructive-response test items. This replacement plan will be written
into the next generation of RFP documents that are scheduled for release
in May 2003. The Department has identified, as a high priority, (1) the
administration of an independent audit of the current State Assessment's
breadth and depth of standards coverage, (2) the expansion of
high-quality constructive-response test items, and (3) the advancement
of discussions with other States to collaborate in the development of
high quality test items and other assessment strategies.

\(1\) *Test item rigor analysis.* The Department of Public Instruction
will conduct a thorough analysis of the current North Dakota State
Assessment regarding its rigor of higher order thinking skills and
understanding. The Department will contract with an independent, outside
facilitator to conduct this analysis. The RFP has not yet been drafted.
The depth and breadth analysis will be conducted in early 2003 as a
baseline evaluation of the current State Assessment in anticipation of
its enhancement with future replacement items and the future development
of other grade-level assessments. It is anticipated that the project
will convene educators from across the State, including classroom
teachers, administrators, content specialists, and university
professors, to conduct an audit of the current State Assessment in terms
of an agreed upon evaluation criteria. This effort would evaluate the
State Assessment against five levels of increasing difficulty: (1)
identity and recall; (2) use of concepts; (3) explanation and reasoning;
(4) evaluation and extension; and (5) integration and performance.

\(2\) *Item replacement policy.* It is the long-term commitment of the
Department of Public Instruction to employ an item-replacement model
that steadily increases the number and quality of constructive-response
test items, including greater use of extended constructive-response
items. Future RFPs for the North Dakota State Assessment will include a
schedule for the improved quality of constructive-response test items.
The Department has adopted a policy requiring future RFPs to incorporate
the recommendations of the Education Leaders Council, *Model Contractor
Standards & State Responsibilities for State Testing Programs, 2002*
(refer to page 19 within **Appendix Y: North Dakota Standards and
Assessment Development Protocols** or reference the following web site,
<http://www.dpi.state.nd.us/standard/content/toc.pdf>).

\(3\) *State consortium efforts.* Following the selection of the State's
next assessment vendor following an RFP selection process, the
Department of Public Instruction will explore with the State's next
approved vendor the prospects of initiating a series of discussions with
other States who contract with the approved vendor to establish a
consortium of States committed to assessment improvement. This
consortium of States would share the costs and advance the development
of high quality test items, specifically constructive-response and
extended-response test items. By convening States that share a common
vendor, there are greater opportunities to achieve successes by unifying
efforts, maximizing gains, and minimizing copyright impediments. The
State will begin discussions with interested States beginning in July
2003.

\(c\) Assessment innovations.

The Department is drafting a series of recommendations regarding the
possible adoption of several innovations identified by other States and
developed within the white paper, *State Innovation Priorities for
Testing,* Education Leaders Council. The Department is expected to
submit a comprehensive list of assessment procedural improvements and
potential innovations to the State Superintendent by July 2003.

In addition to this anticipated list of improvements and innovations,
the Department is considering the integration of two established
products into the current State Assessment: (1) test item task banks
developed by previous development work, and (2) a web-based scoring
application for extended-response test items.

II\. Expansion of the State Assessment System

North Dakota, through an agreement with the U.S. Department of
Education, has established an assessment waiver plan to bring the State
into full compliance with ESEA, Section 1111(b)(1) requirements. . This
waiver plan, approved through August 2003, is enclosed as Appendix A:
North Dakota State Assessment Waiver Agreement Plan and can be accessed
at the following web site:
[**http://www.dpi.state.nd.us/testing/assess/plan.pdf**](http://www.dpi.state.nd.us/testing/assess/plan.pdf).
During the 2001-02 school year, North Dakota administered its state
assessment and is on schedule to meet fully all provisions set forth
within the waiver plan.

State assessments have been developed and adopted thus far in
mathematics and reading at grades 4, 8, and 12 in accordance with North
Dakota's approved assessment waiver agreement and the *North Dakota
Standards and Assessment Development Protocols*
(<http://www.dpi.state.nd.us/standard/content/toc.pdf>). North Dakota
will proceed to develop state assessments in mathematics and reading at
additional grades (grades 3, 5, 6, and 7) by 2005-2006 in accordance
with State protocols and section 1111(b)(1) requirements. North Dakota
will proceed to develop state assessments in science at grades 4, 8, and
12 by 2007-2008 in accordance with State protocols and section
1111(b)(1) requirements. Additionally, North Dakota will expand its
science assessment, voluntarily, at grades 3, 5, 6, and 7 by 2007-2008
in accordance with State protocols and section 1111(b)(1) standards.

North Dakota has submitted its plan to expand the development of grade
specific assessments to meet the requirements of NCLBA. This submission
was an element of the State's Consolidated Application for ESEA funding,
dated June 2002. Refer to **Appendix H: North Dakota State ESEA
Consolidated Application**, pages 7-10, or refer to the following web
site: <http://www.dpi.state.nd.us/grants/DOEapp.pdf>). The North Dakota
State Consolidated Application has since been approved by the U.S.
Department of Education.

The North Dakota Assessment System provides for a single, unified,
statewide assessment that measures the performance of *all* students in
terms of the State's challenging content and achievement standards.

III\. Enhancement to the State's Accountability System

The Department of Public Instruction will develop a state-level advisory
committee that will advise the State Superintendent on the development
and review of all State AYP policies and submit recommendations to the
State Superintendent. The State Superintendent will review and approve
the disposition of all recommendations. The Department of Public
Instruction anticipates the development of this advisory committee by
July 2003.

**PRINCIPLE 10. In order for a public school or LEA to make AYP, the
State ensures that it assessed at least 95% of the students enrolled in
each subgroup.**

+----------------------+-----------------------+-----------------------+
| ## CRITICAL ELEMENT  | **EXAMPLES FOR\       | **EXAMPLES OF\        |
|                      | MEETING               | *NOT* MEETING         |
|                      | REQUIREMENTS**        | REQUIREMENTS**        |
+======================+=======================+=======================+
| 1.  What is the      | State has a procedure | The state does not    |
|     State\'s method  | to determine the      | have a procedure for  |
|     for calculating  | number of absent or   | determining the rate  |
|     participation    | untested students (by | of students           |
|     rates in the     | subgroup and          | participating in      |
|     State            | aggregate).           | statewide             |
|     assessments for  |                       | assessments.          |
|     use in AYP       | State has a procedure |                       |
|     determinations?  | to determine the      | Public schools and    |
|                      | denominator (total    | LEAs are not held     |
|                      | enrollment) for the   | accountable for       |
|                      | 95% calculation (by   | testing at least 95%  |
|                      | subgroup and          | of their students.    |
|                      | aggregate).           |                       |
|                      |                       |                       |
|                      | Public schools and    |                       |
|                      | LEAs are held         |                       |
|                      | accountable for       |                       |
|                      | reaching the 95%      |                       |
|                      | assessed goal.        |                       |
+----------------------+-----------------------+-----------------------+
| **STATE RESPONSE AND |                       |                       |
| STATE ACTIVITIES FOR |                       |                       |
| MEETING              |                       |                       |
| REQUIREMENTS**       |                       |                       |
+----------------------+-----------------------+-----------------------+

The State of North Dakota requires all students enrolled in public
schools within North Dakota to participate in the State Assessment
system. Refer to **Appendix B: North Dakota Century Code citations** or
reference the North Dakota Century Code at the following web site,
[http://www.state.nd.us/lr/cencode/T151C21.pdf](http://www.state.nd.us/lr/assembly/57-2001/cencode/CCT15x1.pdf)
). All students, regardless of their enrollment status, participate in
the State Assessment. This total inclusion policy includes those
students who may have enrolled in a district or school after the
beginning of a school year. Any student who may have been enrolled in a
school or district after the beginning of a school year is identified on
their assessment demographic sheet. Students or school personnel mark a
special code on the assessment demographic sheet that identifies their
late enrollment status. This code is used to identify the student and to
remove them from the school's student roll for AYP identification
purposes. Refer to page 29 for codes "R" and "S" of **Appendix Q: North
Dakota State Assessment, Test Coordinator's Manual 2002-03** for the
enrollment code identification fields. A student who has not been
enrolled in a school for the entire year but has been enrolled in the
district for the entire year will not be included into AYP consideration
for the school but will be included into AYP consideration for the
district.

All students must be accounted for regarding their enrollment status.
This is a required entry on the demographic sheet of all students.
Student participation rates will be compared to the school's and
district's Average Daily Membership student count used to reimburse
school's and district's for their State foundation aid. Therefore, the
State references reimbursement census data to confirm student
participation rates. Student participation rates may be identified
within the aggregate and disaggregated by subgroup.

Refer to page 29 for codes "R" and "S" of **Appendix Q: North Dakota
State Assessment, Test Coordinator's Manual 2002-03** for the enrollment
code identification fields. Student participation rates will be compared
to the school's and district's Average Daily Membership student count
used to reimburse school's and district's for their State foundation
aid. Therefore, the State references reimbursement census data to
confirm student participation rates.

Participation rates on the North Dakota State Assessment are calculated
as follows:

[\# of students with test results]{.underline}

\# of students enrolled at the time of test administration

Students participating in the North Dakota Alternate Assessment are
included in the numerator and denominator.

The State is developing a statewide student data analysis and reporting
system to aid the Sate in monitoring the enrollment patterns and
participation rates of students. The TetraData application will allow
the State to link district enrollment files with the State's assessment
participation files in order to assure that all enrolled students are
accounted for in the State Assessment system files. Refer to **Appendix
R: TetraData Data Analysis and Reporting System Summary** for an
overview of the TetraData system's purpose and design.

The State currently is reviewing its ESEA and accreditation monitoring
policies. The State is pursuing an amendment to its monitoring
requirements that would mandate districts to produce evidence regarding
the enrollment dates of all students. Monitors would check for any
students who had enrolled after the beginning of a school year and
cross-check their participation status in the State Assessment data
file. Any failures to include such students would be identified as a
compliance violation of the school's and district's ESEA compliance
agreement. Refer to **Appendix S: Consolidated Application Certification
and Assurances** for the State assessment requirement for receipt of
federal ESEA funding. A school or district may be sanctioned for any
compliance violation of their ESEA assurances agreement.

It is the expressed policy of the State of North Dakota to include all
students within the North Dakota State Assessment.

+----------------------+-----------------------+-----------------------+
| ## CRITICAL ELEMENT  | **EXAMPLES FOR\       | **EXAMPLES OF\        |
|                      | MEETING               | *NOT* MEETING         |
|                      | REQUIREMENTS**        | REQUIREMENTS**        |
+======================+=======================+=======================+
| 2.  What is the      | State has a policy    | State does not have a |
|     State\'s policy  | that implements the   | procedure for making  |
|     for determining  | regulation regarding  | this determination.   |
|     when the 95%     | the use of 95%        |                       |
|     assessed         | allowance when the    |                       |
|     requirement      | group is              |                       |
|     should be        | statistically         |                       |
|     applied?         | significant according |                       |
|                      | to State rules.       |                       |
+----------------------+-----------------------+-----------------------+
| **STATE RESPONSE AND |                       |                       |
| STATE ACTIVITIES FOR |                       |                       |
| MEETING              |                       |                       |
| REQUIREMENTS**       |                       |                       |
+----------------------+-----------------------+-----------------------+

The State of North Dakota has established a definition for the minimum
number of students in a subgroup for both reporting and accountability
purposes. The definition is consistent with the minimum number
identified within Principle 9. The State has established a test of
statistical significance for the method of determining a minimum number
within a given population and referenced to the established measurable
objective, participation rate, graduation rate, or attendance rate.
Refer to Principle 9.1 for a detailed overview of this method of
statistical significance.

The State will apply the binomial distribution on all school and
subgroup participation data to determine the appropriate statistical
participation rate. The binomial distribution is calculated with the
alpha=0.01, the target set at 95%, and the sample size consisting of all
students enrolled at the time of the assessment.

*The incorporation of federal guidance regarding the full participation
of all students within the State assessment and the restricted omission
of certain students with significant medical conditions.*

On March 26, 2004, the U.S. Department of Education released guidance
regarding the calculation of adequate yearly progress based on the
participation of at least 95% of students in a State's annual assessment
of student achievement, measured by a total school population and by
subgroups. Full participation within the Act ensures a valid and
reliable means of determining accountability for all schools, districts,
and the State. The following provisions apply.

a\. *Multi-year averaging*. In calculating adequate yearly progress for
participation rates, the State will access participation data from the
current year, a two-year average, and a three-year average. If any of
these accessed data points meet or exceed 95%, the specified school,
district, or State will be deemed to meet the requirements of adequate
yearly progress.

b\. *Significant medical emergencies*. In calculating adequate yearly
progress for participation rates, the State will omit, within restricted
conditions, any student who has been prohibited from participating in
the State assessment during the entire testing window, including make-up
dates, due to a significant medical emergency. The Department of Public
Instruction will consider only those requests for the omission of a
student with a significant medical emergency that have been documented
by the affected school. The Department reserves the right to monitor the
circumstances surrounding any requests for omission. Students must be
enrolled within the affected school during the testing window, as
evidenced on the pupil membership files retained within the State Online
Reporting System. The State will document the receipt of any omission
requests, remove the student from the calculation of adequate yearly
progress, and record and report aggregate incidence rates of omissions
statewide. The State will not issue student omission reports that might
lead to the identification of an individual student.

# Appendix A

Required Data Elements for State Report Card

**1111(h)(1)(C)**

1\. Information, in the aggregate, on student achievement at each
proficiency level on the State academic assessments (disaggregated by
race, ethnicity, gender, disability status, migrant status, English
proficiency, and status as economically disadvantaged, except that such
disaggregation shall not be required in a case in which the number of
students in a category is insufficient to yield statistically reliable
information or the results would reveal personally identifiable
information about an individual student.

2\. Information that provides a comparison between the actual
achievement levels of each student subgroup and the State's annual
measurable objectives for each such group of students on each of the
academic assessments.

3\. The percentage of students not tested (disaggregated by the student
subgroups), except that such disaggregation shall not be required in a
case in which the number of students in a category is insufficient to
yield statistically reliable information or the results would reveal
personally identifiable information about an individual student.

4\. The most recent 2-year trend in student achievement in each subject
area, and for each grade level, for the required assessments.

5\. Aggregate information on any other indicators used by the State to
determine the adequate yearly progress of students in achieving State
academic achievement standards disaggregated by student subgroups.

6\. Graduation rates for secondary school students disaggregated by
student subgroups.

7\. Information on the performance of local educational agencies in the
State regarding making adequate yearly progress, including the number
and names of each school identified for school improvement under section
1116.

8\. The professional qualifications of teachers in the State, the
percentage of such teachers teaching with emergency or provisional
credentials, and the percentage of classes in the State not taught by
highly qualified teachers, in the aggregate and disaggregated by
high-poverty compared to low-poverty schools which (for this purpose)
means schools in the top quartile of poverty and the bottom quartile of
poverty in the State.

###### 

###### 

###### The State of North Dakota stipulates that it will include all data elements provided within ESEA section 1111 within the State Report Card and Profile. Refer to Appendix O: North Dakota State Report Card and Profile. The State will include attendance rate for elementary and middle schools. 

###### Table of Appendices

####### A. North Dakota State Assessment Waiver Agreement Plan

**B. North Dakota Century Code**

**C. North Dakota Standards and Assessment Development Protocols**

**D. North Dakota Mathematics Content Standards**

**E. North Dakota Mathematics Achievement Standards**

**F. North Dakota English Language Arts Content Standards**

**G. North Dakota Language Arts Achievement Standards**

**H. North Dakota State ESEA Consolidated Application**

**I. Schools Falling Outside Assessment System Grade Span**

**J. North Dakota State Assessment, Bookmark Standards Setting Technical
Report, 2002**

**K. State Superintendent's Approval Notification of North Dakota State
Assessment Cut Scores**

**L. Program Improvement Activities**

**M. Testimony Before the Education Committee by the Department of
Public Instruction, October 10, 2002**

**N. North Dakota State Assessment, Student Achievement Reports**

**O. North Dakota State Report Card and Profile**

**P. State Superintendent's Release of State Assessment Results**

**Q. North Dakota State Assessment, Test Coordinator's Manual, 2002-03**

**R. TetraData Data Analysis and Reporting System Summary**

**S. Consolidated Application Certification and Assurances**

**T. State Adequate Yearly Progress Computation Rules**

**U. Individualized Education Program Planning Process**

**V. North Dakota State Assessment, Test Specifications**

**W. North Dakota State Assessment, 2002 Preliminary Technical Report**

**X. North Dakota Assessment System 2001-02 Baseline Impact Data.**

**Y. North Dakota Graduation 2001-02 Impact Data.**

**Z. North Dakota Attendance 2001-02 Impact Data.**

**AA. North Dakota Participation 2001-02 Impact Data.**

**BB. North Dakota Composite AYP Identification Impact Data**

[^1]: System of State achievement standards will be reviewed by the
    Standards and Assessments Peer Review. The Accountability Peer
    Review will determine that achievement levels are used in
    determining AYP.

[^2]: The state must provide rewards and sanctions for all public
    schools and LEAs for making adequate yearly progress, except that
    the State is not required to hold schools and LEAs not receiving
    Title I funds to the requirements of section 1116 of NCLB
    \[§200.12(b)(40)\].

[^3]: If the state has separate assessments to cover its language arts
    standards (e.g., reading and writing), the State must create a
    method to include scores from all the relevant assessments.

[^4]: Decisions may be based upon several years of data and data may be
    averaged across grades within a public school \[§1111(b)(2)(J)\].

[^5]: The minimum number is not required to be the same for reporting
    and accountability.

[^6]: The Family Education Rights and Privacy Act (FERPA) prohibits an
    LEA that receives Federal funds from releasing, without the prior
    written consent of a student's parents, any personally identifiable
    information contained in a student's education record.

[^7]: State Assessment System will be reviewed by the Standards and
    Assessments Peer Review Team.

[^8]: See USC 6311(b)(2)(I)(i), and 34 C.F.R. 200.20(b)

[^9]: NCLB only lists these indicators as examples.

[^10]: If the state has more than one assessment to cover its language
    arts standards, the State must create a method for including scores
    from all the relevant assessments.

[^11]: Several events may occur which necessitate such a plan. For
    example, (1) the State may need to include additional assessments in
    grades 3-8 by 2005-2006; (2) the State may revise content and/or
    academic achievement standards; (3) the State may need to
    recalculate the starting point with the addition of new assessments;
    or (4) the State may need to incorporate the graduation rate or
    other indicators into its State Accountability System. These events
    may require new calculations of validity and reliability.