November 14, 2006

> **Control Number**
>
> **ED-OIG/A09G0009**

Dr. Terry Bergeson

State Superintendent of Public Instruction

State of Washington

Office of the Superintendent of Public Instruction

Old Capitol Building, P.O. Box 47200

Olympia, WA 98504-7200

Dear Dr. Bergeson:

This **Final Audit Report**, entitled *Data Quality Review of
Washington* *Consolidated State Performance Reports*, presents the
results of our audit. The purpose of the audit was to determine whether
graduation and dropout rates reported to the U.S. Department of
Education (Department) by the State of Washington's Office of the
Superintendent of Public Instruction (WOSPI) in the Consolidated State
Performance Reports for 2003-2004 and 2004-2005 are supported by
reliable data and meet the requirements of Section 1111(b)(2) of the
Elementary and Secondary Education Act of 1965, as amended, and 34 Code
of Federal Regulations

§ 200.19 (a) and (c).

BACKGROUND

The Elementary and Secondary Education Act (ESEA), as amended by the No
Child Left Behind Act of 2001 (NCLB), requires states receiving ESEA,
Title I, Part A funds to implement a statewide accountability system.
The accountability system must include a high school graduation rate
indicator -- a required factor in determining whether local educational
agencies and high schools made adequate yearly progress in students'
achievement of state academic standards.

States report graduation rates for all students and by student subgroups
in the Consolidated State Performance Report (CSPR) -- the required
annual state report to the Secretary. The CSPR instructions require the
state to report graduation rates computed in accordance with the
definition approved as part of the State's accountability plan (i.e.,
approved Consolidated State Application Accountability Workbook). The
CSPR instructions also require states to report the annual dropout rates
for high school students using the National Center for Education
Statistics' definition of a high school dropout. States reported
graduation and dropout rates for school year 2002-2003 in the 2003‑2004
CSPR. Rates for school year 2003-2004 were reported in the 2004‑2005
CSPR. The Department uses information in the CSPR to prepare reports on
progress in meeting strategic plan goals, individual program
performance, and States' progress in implementing NCLB.

In the definition of "adequate yearly progress" at ESEA §
1111(b)(2)(C)(vi), Congress defined graduation rate as "the percentage
of students who graduate from secondary school with a regular diploma in
the standard number of years." The implementing Federal regulation at
34 Code of Federal Regulations (C.F.R.) § 200.19(a)(1)(i) further
defines the graduation rate to mean---

A.  The percentage of students, measured from the beginning of high
    school, who graduate from high school with a regular diploma (not
    including an alternative degree that is not fully aligned with the
    State's academic standards, such as a certificate or a GED) in the
    standard number of years; or

B.  Another definition, developed by the State and approved by the
    Secretary in the State plan, that more accurately measures the rate
    of students who graduate from high school with a regular diploma as
    defined in paragraph (a)(1)(i)(A) of this section.

The cited regulation also states that "\[i\]n defining \[the\]
graduation rate, the State must avoid counting a dropout as a transfer."
The regulation at 34 C.F.R. § 200.19(c) requires states to ensure that
academic indicators, such as the graduation rate, are valid and
reliable; consistent with relevant, nationally recognized professional
and technical standards, if any; and consistent throughout the State
within each grade span.

In September 2005, WOSPI received approval from the Department to use an
extended graduation rate for adequate yearly progress (AYP) purposes
rather than a rate based on students who graduated within a standard
number of years (i.e., on-time graduation rate). The extended graduation
rate includes students who graduated in more than the standard four
years as graduates in the calculation of the rate.

WOSPI received about \$157 million and \$171 million in Title I, Part A
funds in fiscal years 2003 and 2004, respectively. In its 2003-2004
CSPR, WOSPI reported an on-time graduation rate for all students of 65.7
percent and an annual dropout rate of 6.7 percent based on student data
for school year 2002-2003. In its 2004-2005 CSPR, WOSPI reported an
on-time graduation rate for all students of 70.1 percent, an extended
graduation rate for all students of 74.0 percent, and an annual dropout
rate of 5.8 percent based on student data for school year 2003-2004.
Attachment 1 provides the formulas and calculations WOSPI used to
compute the on-time and extended graduation rates reported in the CSPRs.

AUDIT RESULTS

WOSPI complied with the ESEA requirement to provide an annual report to
the Secretary by submitting the 2003-2004 and 2004-2005 CSPRs, which
included the State's graduation and dropout rates. However, student data
used by WOSPI to calculate the reported rates was not reliable and the
required definition of a "high school dropout" was not used to calculate
the reported dropout rates. The OTHER MATTER section of this report
presents our concerns with the presentation in the State of Washington's
Consolidated State Application Accountability Workbook of the State's
formulas for computing graduation rates for accountability purposes.

In its comments to the draft report, WOSPI concurred with our findings
and agreed to implement our recommendations. In response to the other
matter, WOSPI submitted an amendment to the State's Accountability
Workbook, but the amendment did not fully address our concerns about the
clarity of WOSPI's description of the graduation rate used for AYP
purposes. WOSPI's comments are summarized at the end of each finding.
The full text of WOSPI's comments on the draft report are included as
Attachment 2 to the report.

FINDING NO. 1 -- WOSPI Used Unreliable Data to Calculate Graduation and
Dropout Rates

WOSPI did not use reliable data to calculate graduation and dropout
rates reported in the\
2003-2004 and 2004-2005 CSPRs. WOSPI acknowledged that the data used to
calculate rates reported in the 2003-2004 CSPR was of questionable
accuracy and subsequently took steps to improve the data. Our review of
the student records at three school districts found that data used in
calculating the rates reported in the 2004-2005 CSPR was also
unreliable. WOSPI needs to take additional actions to improve the
accuracy of the data used in calculating the graduation and dropout
rates.

The ESEA § 1111(b)(2)(D) and the implementing regulations at 34 C.F.R. §
200.19(c) require states to ensure high school graduation rates are
reliable and consistent throughout the state within each grade span. The
Government Accountability Office's *Assessing the Reliability of
Computer Processed Data* states data are reliable when they are (1)
complete (they contain all of the data elements and records needed for
the engagement) and (2) accurate (they reflect the data entered at the
source or, if available, in the source documents). A subcategory of
accuracy is consistency. Consistency refers to the need to obtain and
use data that are clear and well-defined enough to yield similar results
in similar analyses. For example, if data are entered at multiple sites,
inconsistent interpretation of data rules can lead to data that, taken
as a whole, are unreliable.

WOSPI used information from a statewide student-level data file to
compute graduation and dropout rates reported in the CSPR. The statewide
data file for each school year was created from data files submitted in
the fall of each year by the State's 249 school districts with high
schools. The individual district data files (referred to as P210
Reports) contained the enrollment status for students enrolled in the
previous school year. To create the P210 report, the districts extracted
student data from the district's student information system and
converted the district's data codes to the State's data codes. WOSPI
staff performed various reasonableness, completeness, and accuracy
checks of data in the P210 reports.

WOSPI categorizes students in the 9^th^ through 12^th^ grade into the
following enrollment categories based on the student's enrollment status
code in the statewide data file:

> [On-Time Graduate]{.underline}: Students who receive a diploma in the
> expected year of graduation.[^1]
>
> [Late Graduate]{.underline}: Students who receive a diploma after
> their expected year of graduation.
>
> [Dropout]{.underline}: Students who intentionally leave school for any
> reason, except death, before receiving a diploma and who do not
> transfer to another school or educational program. Students whose
> status is unknown because they are no longer enrolled, but who are not
> a confirmed transfer or dropout. For purposes of graduation and
> dropout rate calculations, WOSPI includes students who receive General
> Education Development (GED) certificates in the dropout category.
>
> [Transfer (confirmed)]{.underline}: Students who depart school and
> whose transcripts/student records have been requested by a gaining
> school, or students who file an "Intent to Receive Home-Based
> Instruction." Students who move outside the United States before
> completing high school graduation requirements are also considered
> confirmed transfers. For purposes of graduation and dropout rate
> calculations, WOSPI includes deceased students in the transfer
> category.[^2]
>
> [Continuing Student]{.underline}: Students who do not have an
> enrollment status of graduate or other type of completer (e.g., GED
> certificate), transfer, dropout, unknown, or deceased.
>
> A subgroup of this category is the "extended students," who are 12^th^
> grade students who are enrolled beyond their expected graduation year.

The number of students enrolled in each grade (less transfers), the
number of students in each grade in the dropout category, the number of
students in the extended student subgroup, and the number of students in
the on-time graduate and late graduate categories are used to calculate
the State's graduation and dropout rates and, for accountability
purposes, the graduation rates for each school and district in the State
of Washington. WOSPI issues an annual report on the State's graduation
and dropout statistics that provides schools and districts with the
actual student data used to compute individual school and district
graduation and dropout rates. The report for school year 2003‑2004 was
issued in September 2005.

**WOSPI Had Concerns Regarding the**

**Reliability of Data Used to Calculate Rates**

**Reported in the 2003-2004 CSPR**

WOSPI officials advised us that the data used to calculate the rates for
school year 2002-2003 were not accurate, or consistent throughout the
state, as required by the ESEA. In its report titled *Graduation and
Dropout Statistics for Washington's Counties, Districts, and Schools:
School Year 2002-03*, dated September 2004, WOSPI stated that "some
districts have found errors in the information provided for some of
their students" and that district and school "\[r\]ates that are
extremely high or low may also reflect inaccurate reporting." Due to
concerns about data reliability, WOSPI took the following actions to
improve the quality of data for school year 2003-2004.

-   WOSPI published the *P210 Collection Data Manual for the 2003-2004
    School Year* in August 2004. The manual used a new format to present
    data element definitions that identified the various parameters
    (e.g., data type, valid codes, code descriptions, business rules,
    and input examples) for each data field.

-   WOSPI required districts to report an enrollment status code for
    every student. If the enrollment status code field was blank for any
    student, the district's P210 file was returned for correction.
    Previously, the enrollment status code field defaulted to a
    continuing student status when the field was left blank.

-   WOSPI provided additional training via video teleconference to
    district staff on the P210 reports. Documentation showed WOSPI held
    four P210 training sessions (September and October 2004) for the
    school year 2003-2004 reporting period.

Since WOSPI acknowledged that the data for school year 2002-2003 were
not accurate and had subsequently taken steps to improve data, we
focused our review on the reliability of the data used to calculate the
graduation and dropout rates for school year 2003-2004, which were
reported in the 2004-2005 CSPR.

**WOSPI Also Used Unreliable Data to Calculate**

**Rates Reported in the 2004-2005 CSPR**

Our review of student enrollment data in the statewide student-level
data file for three school districts (Seattle Public Schools, Spokane
Public Schools, and Edmonds School District) found that a group of
students at Spokane Public Schools was not included in the statewide
file and student enrollment statuses in the statewide file were not
accurate for the three districts.

[Students on the Spokane Public Schools' P210 Report Were Not Included
in the Statewide Student-level Data File]{.underline}. Spokane Public
Schools' staff provided us with its final P210 report for school year
2003‑2004. The P210 report included 94 students in the 12^th^ grade who
were in the dropout category (i.e., enrollment statuses of dropouts,
unknown, or GED completers). The statewide file contained no
12^th^ grade students in the dropout category for Spokane Public Schools
for school year 2003-2004. With the inclusion of the 12^th^ grade
dropouts, Spokane Public Schools' on-time graduation rate for school
year 2003-2004 would decrease from 85.1 percent to 81.0 percent and its
extended graduation rate would decrease from 86.4 percent to
82.3 percent.

[Student Enrollment Statuses in the Statewide Student-level Data File
Were Not Accurate]{.underline}. Our review of school year 2003-2004
student data extracted from the statewide file for selected high schools
in three school districts identified inaccurate or unsupported
enrollment codes for 141 of the 760 students in our sample (18.6
percent).

+----------------------+-------+------+------+------+------+---+-----+
| ##### Table          |       |      |      |      |      |   |     |
| 1: Results of Review |       |      |      |      |      |   |     |
|  of Student Enrollme |       |      |      |      |      |   |     |
| nt Status for the Sc |       |      |      |      |      |   |     |
| hool Year 2003-2004  |       |      |      |      |      |   |     |
|                      |       |      |      |      |      |   |     |
| ##### for            |       |      |      |      |      |   |     |
| Selected High School |       |      |      |      |      |   |     |
| s in Three Districts |       |      |      |      |      |   |     |
+----------------------+-------+------+------+------+------+---+-----+
| ## Enr               | **On  | **   | *    | **Dr | > ** |   | #   |
| ollment Category Sho | -time | Late | *Tra | opou | Exte |   |  To |
| wn in Statewide File | G     | Gra  | nsfe | ts** | nded |   | tal |
|                      | radua | duat | rs** |      | > St |   |  St |
|                      | tes** | es** |      |      | uden |   | ude |
|                      |       |      |      |      | ts** |   | nts |
|                      |       |      |      |      | >    |   |     |
|                      |       |      |      |      |  (a) |   |     |
+----------------------+-------+------+------+------+------+---+-----+
| ### Number of        | 2,979 | 60   | 1    | 430  | 225  |   | 5,  |
| Students in Category |       |      | ,462 |      |      |   | 156 |
|                      |       |      |      |      |      |   | (b) |
+----------------------+-------+------+------+------+------+---+-----+
| ### Number           | 303   | 60   | 208  | 90   | 99   |   | 760 |
| of Students Reviewed |       |      | (c)  |      |      |   |     |
+----------------------+-------+------+------+------+------+---+-----+
| ### School           | 289   | 50   | 126  | 80   | 74   |   | 619 |
| Documentation Confir |       |      |      |      |      |   |     |
| med Enrollment Codes |       |      |      |      |      |   |     |
+----------------------+-------+------+------+------+------+---+-----+
| ### Inaccur          | 14    | 10   | 82   | 10   | 25   |   | 141 |
| ate Enrollment Codes |       |      |      |      |      |   |     |
+----------------------+-------+------+------+------+------+---+-----+
| > Error Rate         | 4.6%  | 1    | 3    | 1    | 2    |   | 18  |
|                      |       | 6.7% | 9.4% | 1.1% | 5.3% |   | .6% |
+----------------------+-------+------+------+------+------+---+-----+
|                      |       |      |      |      |      |   |     |
+----------------------+-------+------+------+------+------+---+-----+
| ######## Descr       |       |      |      |      |      |   |     |
| iption of Error Type |       |      |      |      |      |   |     |
+----------------------+-------+------+------+------+------+---+-----+
| > Enrollment Should  |       |      |      |      |      |   |     |
| > Have Been Coded    |       |      |      |      |      |   |     |
| > as:                |       |      |      |      |      |   |     |
+----------------------+-------+------+------+------+------+---+-----+
| > Transfer           | \-\-- | \    | \    | 10   | \    | 1 |     |
|                      |       | -\-- | -\-- |      | -\-- | 0 |     |
+----------------------+-------+------+------+------+------+---+-----+
| > Graduate           | \-\-- | \    | \    | \    | 19   | 1 |     |
|                      |       | -\-- | -\-- | -\-- |      | 9 |     |
+----------------------+-------+------+------+------+------+---+-----+
| > Dropout            | \-\-- | \    | 82   | \    | 6    | 8 |     |
|                      |       | -\-- |      | -\-- | (d)  | 8 |     |
+----------------------+-------+------+------+------+------+---+-----+
| > Total              | \-\-- | \    | 82   | 10   | 25   | 1 |     |
|                      |       | -\-- |      |      |      | 1 |     |
|                      |       |      |      |      |      | 7 |     |
+----------------------+-------+------+------+------+------+---+-----+
| > Student Had Less   | 14    | 10   | \    | \    | \    | 2 |     |
| > than Minimum       |       |      | -\-- | -\-- | -\-- | 4 |     |
| > Credit Hours to    |       |      |      |      |      |   |     |
| > Graduate (e)       |       |      |      |      |      |   |     |
+----------------------+-------+------+------+------+------+---+-----+
| > Total Errors       | 14    | 10   | 82   | 10   | 25   | 1 |     |
|                      |       |      |      |      |      | 4 |     |
|                      |       |      |      |      |      | 1 |     |
+----------------------+-------+------+------+------+------+---+-----+
| (a) Continuing       |       |      |      |      |      |   |     |
|     students that    |       |      |      |      |      |   |     |
|     were enrolled    |       |      |      |      |      |   |     |
|     beyond their     |       |      |      |      |      |   |     |
|     expected         |       |      |      |      |      |   |     |
|     graduation year. |       |      |      |      |      |   |     |
|                      |       |      |      |      |      |   |     |
| ```{=html}           |       |      |      |      |      |   |     |
| <!-- -->             |       |      |      |      |      |   |     |
| ```                  |       |      |      |      |      |   |     |
| (a) Total does not   |       |      |      |      |      |   |     |
|     include          |       |      |      |      |      |   |     |
|     continuing       |       |      |      |      |      |   |     |
|     students who     |       |      |      |      |      |   |     |
|     were enrolled in |       |      |      |      |      |   |     |
|     9^th^, 10^th^,   |       |      |      |      |      |   |     |
|     and 11^th^       |       |      |      |      |      |   |     |
|     grade. This      |       |      |      |      |      |   |     |
|     group was not    |       |      |      |      |      |   |     |
|     included in our  |       |      |      |      |      |   |     |
|     sampling         |       |      |      |      |      |   |     |
|     universe. We     |       |      |      |      |      |   |     |
|     considered this  |       |      |      |      |      |   |     |
|     group to be at   |       |      |      |      |      |   |     |
|     lowest risk for  |       |      |      |      |      |   |     |
|     inaccurate       |       |      |      |      |      |   |     |
|     enrollment       |       |      |      |      |      |   |     |
|     statuses.        |       |      |      |      |      |   |     |
|                      |       |      |      |      |      |   |     |
| (b) Our computerized |       |      |      |      |      |   |     |
|     analysis of      |       |      |      |      |      |   |     |
|     student data for |       |      |      |      |      |   |     |
|     the 1,462        |       |      |      |      |      |   |     |
|     transferred      |       |      |      |      |      |   |     |
|     students         |       |      |      |      |      |   |     |
|     confirmed that   |       |      |      |      |      |   |     |
|     840 of the       |       |      |      |      |      |   |     |
|     students had     |       |      |      |      |      |   |     |
|     been reported as |       |      |      |      |      |   |     |
|     enrolled by      |       |      |      |      |      |   |     |
|     another school.  |       |      |      |      |      |   |     |
|     We selected the  |       |      |      |      |      |   |     |
|     sampled students |       |      |      |      |      |   |     |
|     from the         |       |      |      |      |      |   |     |
|     remaining 622    |       |      |      |      |      |   |     |
|     students.        |       |      |      |      |      |   |     |
|                      |       |      |      |      |      |   |     |
| (c) The six errors   |       |      |      |      |      |   |     |
|     did not impact   |       |      |      |      |      |   |     |
|     graduation rates |       |      |      |      |      |   |     |
|     since extended   |       |      |      |      |      |   |     |
|     students are     |       |      |      |      |      |   |     |
|     included with    |       |      |      |      |      |   |     |
|     dropouts in the  |       |      |      |      |      |   |     |
|     calculation of   |       |      |      |      |      |   |     |
|     the grade-level  |       |      |      |      |      |   |     |
|     dropout rates    |       |      |      |      |      |   |     |
|     used to derive   |       |      |      |      |      |   |     |
|     the on-time and  |       |      |      |      |      |   |     |
|     extended         |       |      |      |      |      |   |     |
|     graduation       |       |      |      |      |      |   |     |
|     rates.           |       |      |      |      |      |   |     |
|                      |       |      |      |      |      |   |     |
| (d) Each district    |       |      |      |      |      |   |     |
|     established the  |       |      |      |      |      |   |     |
|     minimum number   |       |      |      |      |      |   |     |
|     of credits that  |       |      |      |      |      |   |     |
|     a student needed |       |      |      |      |      |   |     |
|     for graduation.  |       |      |      |      |      |   |     |
|     For students in  |       |      |      |      |      |   |     |
|     the on‑time and  |       |      |      |      |      |   |     |
|     late graduate    |       |      |      |      |      |   |     |
|     groups, we       |       |      |      |      |      |   |     |
|     reviewed student |       |      |      |      |      |   |     |
|     records to       |       |      |      |      |      |   |     |
|     confirm that     |       |      |      |      |      |   |     |
|     students met the |       |      |      |      |      |   |     |
|     district's       |       |      |      |      |      |   |     |
|     minimum credits  |       |      |      |      |      |   |     |
|     for graduation.  |       |      |      |      |      |   |     |
|     We were unable   |       |      |      |      |      |   |     |
|     to determine     |       |      |      |      |      |   |     |
|     from the records |       |      |      |      |      |   |     |
|     whether the      |       |      |      |      |      |   |     |
|     24 students      |       |      |      |      |      |   |     |
|     cited as errors  |       |      |      |      |      |   |     |
|     received         |       |      |      |      |      |   |     |
|     graduation       |       |      |      |      |      |   |     |
|     diplomas.        |       |      |      |      |      |   |     |
+----------------------+-------+------+------+------+------+---+-----+

While we were not able to identify the specific cause of each enrollment
status error, the following conditions and weaknesses in District and
WOSPI processes may have caused or contributed to the occurrence of the
errors:

-   The *P210 Collection Data Manual for the 2003-2004 School Year*
    (P210 Manual) was issued in August 2004 and included new enrollment
    status coding requirements applicable to the school year that ended
    in August 2004.

```{=html}
<!-- -->
```
-   Information technology and administrative staff at the three
    districts described the extraction and conversion process for
    creating the P210 Report as difficult.

-   A reporting module in the Edmonds School District's student
    information system, which was used to create the District's P210
    Report, did not show the proper enrollment status for students who
    took classes at more than one school in the district. Specifically,
    we noted that Edmonds' student information system (WESPaC)
    identified 12^th^ grade students enrolled full-time at one high
    school (home high school), who took at least one class at another
    high school, as continuing students at the end of 12^th^ grade even
    if the student graduated from the home high school. In the State of
    Washington, 214 school districts use the WESPaC student information
    system, which was developed by the Washington School Information
    Processing Cooperative (WSIPC).

-   Based on our interviews with school staff, we concluded that staff
    at seven of the nine high schools did not fully understand
    enrollment status definitions and documentation requirements. For
    example, staff entered a transfer status for students pursuing a GED
    and relied on statements from a responsible adult as evidence for a
    transfer. The P210 Manual states that transfer codes are only used
    for confirmed transfers to another school (i.e., transcripts/records
    have been requested by another school or student filed an Intent to
    Receive Home-Based Instruction). The P210 Manual instructs districts
    to enter an unknown enrollment status code when the sole evidence is
    hearsay reports and unsubstantiated statements of intent to
    transfer. We also found that, for students with an unknown status in
    the district's student information system, school staff did not
    change the enrollment status to transfer after the district received
    a records request from another school (i.e., confirmed transfer).

-   School staff in Spokane Public Schools altered the graduation year
    of students that were not promoted to the next grade. WOPSI uses the
    graduation year, which is entered at the time the student enters
    high school, to identify students who graduated in the standard
    number of years. As a result of altering the graduation year,
    Spokane did not report any extended students in school year
    2003-2004. On March 30, 2006, Spokane sent an email instructing
    school staff to not change the graduation year in the student record
    and to check student records and make changes, as necessary, to
    reflect the student's original graduation year.

-   Spokane Public Schools information technology staff inadvertently
    changed student records in the district's student information system
    to show an enrollment status of graduate for students with less than
    21 credits and a graduation year of 2004.

-   In the districts we reviewed, a school counselor determines whether
    a student has met the minimum credits for graduation based on
    information in the student's school transcript. There was no
    independent review by a second person or edit check in the
    district's student information system to validate the counselor's
    determination.

-   Reviews of student enrollment statuses recorded in districts'
    student information systems were limited. WOSPI performed various
    reasonableness, completeness, and accuracy checks of data submitted
    in the P210 reports. The Washington State Auditor's Office (WSAO)
    analyzed trends in graduation rates for districts subject to the
    audit requirements of the Single Audit Act. When conducting school
    district audits, WSAO confirmed that the district submitted the
    required P210 reports and reviewed school records for selected
    students with an enrollment status of graduate in the district's
    student information system. However, WSAO did not review support for
    other enrollment statuses and conducted no reviews of school records
    supporting the student's enrollment status for districts that were
    not subject to the audit requirements of the Single Audit Act. WOSPI
    did not review student enrollment statuses as part of its on-site
    monitoring reviews nor did the three districts perform such reviews.

Our tests of the reliability of student-level data was limited to the
760 sampled students who attended selected high schools in three
districts. Thus, the results of our review of the sampled students may
not necessarily be representative of data errors statewide. However, if
the results of the 760 sampled students are representative of the
reliability of data in other districts in the statewide file, the
State's on-time graduation rate for school year 2003-2004 would decline
from 70.1 percent to 64.7 percent and its extended graduation rate would
decline from to 74.0 percent to 68.0 percent.[^3] (The impact of
enrollment status errors on the reported dropout rate is presented in
Finding No.2.) Thus, the Department may have relied on inaccurate rates
to assess the progress of the State of Washington's Title I, Part A
program. Additionally, WOSPI used unreliable data to make AYP
determinations for districts and high schools.

For school year 2004-2005, WOSPI collected data from districts using its
new Core Student Record System (CSRS). WOSPI issued the *Core Student
Record System Monthly Collection Data Manual for the 2004-2005 School
Year* in May 2004. Districts continue to maintain their own student
information systems, but provide updated student data to CSRS on a
monthly basis. WOSPI is using student data in CSRS to calculate the
graduation and dropout rates that will be reported in the 2005-2006
CSPR. While the conversion to CSRS may address some of the conditions
found in our review, others will still need to be addressed since CSRS
continues to rely on data extracted from the districts' student
information systems. During our on-site visits at school districts,
district staff indicated that the procedures for preparing the monthly
CSRS data submissions were difficult and time consuming to perform.

**Recommendations**

We recommend that the Assistant Secretary for Elementary and Secondary
Education require WOSPI to---

1.  Identify the specific difficulties that district information
    technology staff encountered when extracting student data from
    district student information systems for transmittal to WOSPI and
    take action to mitigate the difficulties.

2.  Advise the WSIPC of the deficiency noted in the WESPaC reporting
    module and take the necessary actions to ensure that districts using
    the WESPaC system properly report the graduation status of students
    who took classes at more than one school.

3.  Continue to provide annual training to districts and take steps to
    target the training to district and school personnel who enter
    enrollment codes to ensure consistent understanding of enrollment
    status definitions and required supporting documentation.\
    If feasible, the training should be provided online to enable new
    hires and others to easily access the training.

4.  Assess whether the standard data checks performed by WOSPI staff to
    identify incomplete or unusual student numbers in district data
    should be expanded to include additional data analyses, such as
    confirming that the P210 reports and the statewide student-level
    data file include students in each enrollment category for each
    district and school.

5.  Issue a notice advising districts of the importance of adequate
    internal controls for ensuring that enrollment statuses are
    accurately recorded and that students meet the minimum credits for
    graduation.

6.  Develop and implement a review process for ensuring that school
    staff accurately record student enrollment statuses and retain
    required documentation. WOSPI could consider reviewing enrollment
    statuses of selected students during state on-site monitoring
    reviews and having the WSAO expand its review of graduate status
    during single audits to include review of support for other
    enrollment statuses.

**WOSPI Comments**

WOSPI concurred with the finding and described the action taken or
planned for the recommendations. WOSPI agreed that errors were present
in the student data for the three districts, but expressed concern that
generalization from these districts to the entire state may overestimate
the extent of data errors statewide. WOSPI also noted that the review
focused on student data for a time period (school year 2003‑2004) that
occurred shortly after passage of NCLB, which increased the need for
ensuring that accurate data was available for AYP determinations. WOSPI
disagreed with information presented in the report in the paragraph
titled "Students on the Spokane Public Schools' P210 Report Were Not
Included in the Statewide Student-level Data File." Specifically, WOSPI
asserted that dropouts from Spokane Public Schools were included in the
statewide student-level data file as extended students.

##   {#section .RptSubcaption}

**OIG Response**

To address WOSPI's concern regarding the generalization of the audit
results, we added language in the final report to emphasize the limited
nature of our review and that the results may not necessarily represent
the extent of errors statewide. We made no other changes to the
information in the report. The WOSPI report supporting the student data
used to calculate the graduation rate for Spokane Public Schools for
school year 2003-2004 showed no students in either the dropout or
extended student categories.

FINDING NO. 2 -- WOSPI Reported Dropout Rates That Were Not Computed
Using the Required Definition of a Dropout

In the 2003-2004 and 2004-2005 CSPRs, WOSPI reported dropout rates that
were not computed in adherence with the National Center for Education
Statistics (NCES) definition of a dropout. The CSPR instructions
state---

. . . States should use the annual event school dropout rate for
students leaving a school in a single year determined in accordance with
the NCES' Common Core of Data.

> Consistent with this requirement, States must use NCES' definition of
> "high school dropout," An individual who: 1) was enrolled in school at
> some time during the previous school year; and 2) was not enrolled at
> the beginning of the current school year; and 3) **has not** graduated
> from high school or **completed a state- or district-approved
> educational program**; and 4) does not meet any of the following
> exclusionary conditions: a) transfer to another public school
> district, private school, or state- or district-approved educational
> program ..., b) temporary absence due to suspension or school-excused
> illness; or c) death. **\[Emphasis added.\]**

The NCES definition of a dropout does not include students who completed
a state- or district-approved education program, such as a GED.

WOSPI used the following state definition of a dropout rather than the
NCES definition when calculating the dropout rate reported in the CSPRs.

Dropouts are students who drop out of school for any reason, **finish
their schooling without a regular diploma**, or whose status is
"unknown" because they are no longer enrolled but are not confirmed
transfers or dropouts. **\[Emphasis added.\]**

The state definition of a dropout includes students who received a GED
(GED completers).

By using the state definition, WOSPI overstated the state's dropout rate
in the CSPRs. WOSPI's calculation of the dropout rate for school year
2003-2004 included 18,365 dropouts of which 1,441 were GED completers.
The exclusion of the GED completers in the calculation would decrease
WOSPI's reported dropout rate from 5.8 percent to 5.4 percent. However,
the dropout rate increases when the rate is also adjusted for the
enrollment status errors noted in Finding\
No. 1. If the GED completers were excluded and the results of the
sampled students reviewed in our audit are representative of the
reliability of data in other districts in the statewide file, WOSPI's
reported dropout rate would increase from 5.8 percent to 7.7
percent.[^4]

WOSPI staff considered the state definition of a dropout as the more
appropriate definition for reporting purposes to maintain consistency
with the graduation rate defined at\
ESEA § 1111(b)(2)(C), which does not include GED completers with
graduates.

**Recommendation**

2.1 We recommend that the Assistant Secretary for Elementary and
Secondary Education require WOSPI to adhere to the CSPR instructions for
reporting dropout rates.

**WOSPI Comments**

WOSPI concurred with the finding and agreed to implement the
recommendation.

OTHER MATTER

WOSPI's graduation rate formulas were not accurately reflected in the
State's Consolidated State Application Accountability Workbook.[^5]
Thus, the Department and other readers of the Accountability Workbook
were unaware of the actual formula used for AYP purposes. We identified
two reports that mistakenly concluded, based on the Accountability
Workbook, that the State of Washington used a graduation rate derived
from data accumulated over a four-year period.[^6] In fact, the State's
actual formula used student data exclusively from the most recently
completed school year to derive the rates for each of the four grade
levels. The Accountability Workbook needs to clearly disclose the
formulas used to calculate high school graduation rates used for AYP
determinations and reported in the CSPRs.

In its comments on the draft report, WOSPI referred to an amendment to
its Accountability Workbook that was submitted in August 2006 for
Departmental approval. The amendment includes the actual formulas used
by WOSPI to compute graduation rates. However, further clarification is
needed. The amendment does not state that WOSPI uses data solely from
one school year to compute the rate and use of the term "cohort group"
could lead a reader to incorrectly conclude that the State uses student
data obtained over multiple school years.

In addition, the amendment incorrectly states that the extended
graduation rate "may be used for AYP purposes," when, in fact, the
extended graduation is used for such purposes. The Department had not
approved WOSPI's amendment request as of the issuance of this report.

OBJECTIVES, SCOPE, AND METHODOLOGY

The objective of our audit was to determine whether graduation and
dropout rates reported to the Department by WOSPI in the 2003-2004 and
2004-2005 CSPRs are supported by reliable data and meet the requirements
of the applicable provisions of the ESEA and Federal regulations.
Specifically, we determined whether the---

-   Graduation and dropout rates reported in the CSPRs for the "All
    Students" category were computed in accordance with the requirements
    of ESEA § 1111(b)(2) and\
    34 C.F.R. §200.19 (a) and (c).

-   Data supporting the graduation and dropout rates were sufficiently
    valid and reliable for the Department to use the rates to complete
    reports on its progress in meeting its strategic plan goals,
    individual program performance, and State's progress in implementing
    NCLB.

To accomplish our objective, we---

-   Reviewed applicable laws, regulations, and other guidance;

-   Reviewed pertinent reports dealing with graduation and dropout rates
    published by Federal, state, and private organizations;

-   Identified, assessed, and tested internal controls employed by WOSPI
    to improve the reliability of data supporting graduation and dropout
    rates;

-   Interviewed officials and reviewed written policies and procedures
    related to graduation rates and dropout rates at WOSPI, three school
    districts, and nine high schools;

```{=html}
<!-- -->
```
-   Reviewed student files at nine high schools.

From the state's eleven districts with more than 20,000 enrolled
students in school year 2003‑2004, we selected three districts for
review using a risk-based approach. We selected Edmonds School District
and Seattle Public Schools because each met WOSPI's high school
graduation rate goal of 66 percent using the extended graduation rate,
but failed to meet the goal using the on-time graduation rate employed
in school year 2002-2003.[^7] We selected Spokane Public Schools for
review because of the district's 19.4 percent increase in on-time
graduation rates from 2002-2003 to 2003-2004 (the largest increase for
districts with more than 20,000 students). In each district, we selected
for review the three high schools having at least 1,300 students that
met WOSPI's extended graduation rate goal of 66 percent by the smallest
margin.

To select the student sample at each high school, we used the statewide
student-level data file that WOSPI utilized to compute the graduation
and dropout rates in school year 2003-2004.

To assess the reliability of the data files, we compared the total
records in the statewide file for 9^th^ through 12^th^ grades to
enrollment counts reported by WOSPI. We confirmed that the totals by
category for each of nine high schools matched the totals used to
compute graduation and dropout rates for those schools. We also
determined whether the school had records for sampled students that
confirmed that the students were enrolled during the school year. Based
on these

tests, we concluded that the data were sufficiently reliable for use in
selecting the sample. However, as noted in Finding No. 1 of the Audit
Results section, we found that the student enrollment status information
in the statewide file was not reliable.

For the nine high schools selected for review, we extracted
student-level data from the statewide student universe of 341,996
students in grades 9 through 12. Then, we segregated the students at
each school into six groups: graduates, late graduates, transfers,
dropouts, extended students, and continuing students (other than
extended students). For the transfer group, we separated students into
two subgroups: (1) students who had a gaining school enrollment code to
support the losing school's transfer code, and (2) students who did not
have a gaining school enrollment code to support the losing school's
transfer code. We attributed the greatest risk of error to the second
transfer subgroup and selected the sampled transfer students solely from
that subgroup.

We randomly selected students from the graduate, transfer subgroup,
dropout, and extended student categories. For the selected students and
all students in the late graduate category, we compared the student's
enrollment code to the school's documentation supporting the code. We
did not include students from the continuing student group in our
sampling universe as we considered this group to be at lowest risk for
inaccurate enrollment statuses.

<table style="width:100%;">
<colgroup>
<col style="width: 11%" />
<col style="width: 20%" />
<col style="width: 7%" />
<col style="width: 5%" />
<col style="width: 5%" />
<col style="width: 7%" />
<col style="width: 6%" />
<col style="width: 5%" />
<col style="width: 6%" />
<col style="width: 5%" />
<col style="width: 5%" />
<col style="width: 4%" />
<col style="width: 1%" />
<col style="width: 6%" />
</colgroup>
<tbody>
<tr class="odd">
<td colspan="14"><h4
id="table-2-school-year-2003-2004-student-group-and-sample-sizes-by-district-and-school">Table
2: School Year 2003-2004 Student Group and Sample Sizes by District and
School</h4></td>
</tr>
<tr class="even">
<td><strong>District</strong></td>
<td><strong>High School</strong></td>
<td colspan="2"><strong>On-time Graduates</strong></td>
<td colspan="2"><strong>Late Graduates</strong></td>
<td colspan="3"><strong>Transfers</strong></td>
<td colspan="2"><strong>Dropouts</strong></td>
<td colspan="3"><strong>Extended Students</strong></td>
</tr>
<tr class="odd">
<td></td>
<td></td>
<td><blockquote>
<p>Group Size</p>
</blockquote></td>
<td><blockquote>
<p>Tested</p>
</blockquote></td>
<td><blockquote>
<p>Group Size</p>
</blockquote></td>
<td><blockquote>
<p>Tested</p>
</blockquote></td>
<td><blockquote>
<p>Group Size</p>
</blockquote></td>
<td><blockquote>
<p>Unmatched Group Size</p>
</blockquote></td>
<td><blockquote>
<p>Tested</p>
</blockquote></td>
<td><blockquote>
<p>Group Size</p>
</blockquote></td>
<td><blockquote>
<p>Tested</p>
</blockquote></td>
<td colspan="2"><blockquote>
<p>Group Size</p>
</blockquote></td>
<td><blockquote>
<p>Tested</p>
</blockquote></td>
</tr>
<tr class="even">
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td colspan="2"></td>
<td></td>
</tr>
<tr class="odd">
<td>Edmonds</td>
<td>Edmonds-Woodway</td>
<td>309</td>
<td>31</td>
<td>8</td>
<td>8</td>
<td>114</td>
<td>54</td>
<td>12</td>
<td>98</td>
<td>10</td>
<td colspan="2">27</td>
<td>10</td>
</tr>
<tr class="even">
<td></td>
<td>Lynnwood</td>
<td>241</td>
<td>25</td>
<td>17</td>
<td>17</td>
<td>247</td>
<td>127</td>
<td>25</td>
<td>62</td>
<td>10</td>
<td colspan="2">47</td>
<td>10</td>
</tr>
<tr class="odd">
<td></td>
<td>Meadowdale</td>
<td>304</td>
<td>31</td>
<td>7</td>
<td>7</td>
<td>134</td>
<td>71</td>
<td>(a) 71</td>
<td>45</td>
<td>10</td>
<td colspan="2">49</td>
<td>(a) 49</td>
</tr>
<tr class="even">
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td colspan="2"></td>
<td></td>
</tr>
<tr class="odd">
<td>Seattle</td>
<td>Ballard</td>
<td>360</td>
<td>38</td>
<td>13</td>
<td>13</td>
<td>111</td>
<td>28</td>
<td>12</td>
<td>15</td>
<td>10</td>
<td colspan="2">34</td>
<td>10</td>
</tr>
<tr class="even">
<td></td>
<td>Franklin</td>
<td>328</td>
<td>33</td>
<td>0</td>
<td>0</td>
<td>162</td>
<td>29</td>
<td>17</td>
<td>63</td>
<td>10</td>
<td colspan="2">25</td>
<td>10</td>
</tr>
<tr class="odd">
<td></td>
<td>Roosevelt</td>
<td>380</td>
<td>38</td>
<td>14</td>
<td>14</td>
<td>92</td>
<td>46</td>
<td>10</td>
<td>13</td>
<td>10</td>
<td colspan="2">43</td>
<td>10</td>
</tr>
<tr class="even">
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td colspan="2"></td>
<td></td>
</tr>
<tr class="odd">
<td>Spokane</td>
<td>Lewis &amp; Clark</td>
<td>426</td>
<td>43</td>
<td>0</td>
<td>0</td>
<td>169</td>
<td>68</td>
<td>17</td>
<td>31</td>
<td>10</td>
<td colspan="2">0</td>
<td>0</td>
</tr>
<tr class="even">
<td></td>
<td>North Central</td>
<td>292</td>
<td>30</td>
<td>0</td>
<td>0</td>
<td>194</td>
<td>73</td>
<td>20</td>
<td>28</td>
<td>10</td>
<td colspan="2">0</td>
<td>0</td>
</tr>
<tr class="odd">
<td></td>
<td>Rogers</td>
<td>339</td>
<td>34</td>
<td>1</td>
<td>1</td>
<td>239</td>
<td>126</td>
<td>24</td>
<td>75</td>
<td>10</td>
<td colspan="2">0</td>
<td>0</td>
</tr>
<tr class="even">
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td colspan="2"></td>
<td></td>
</tr>
<tr class="odd">
<td>Totals</td>
<td></td>
<td>2979</td>
<td>303</td>
<td>60</td>
<td>60</td>
<td>1462</td>
<td>622</td>
<td>208</td>
<td>430</td>
<td>90</td>
<td colspan="2">225</td>
<td>99</td>
</tr>
<tr class="even">
<td colspan="9" rowspan="3"></td>
<td colspan="5"></td>
</tr>
<tr class="odd">
<td colspan="3">Total Students</td>
<td colspan="2">5,156</td>
</tr>
<tr class="even">
<td colspan="3">Total Tested</td>
<td colspan="2">760</td>
</tr>
<tr class="odd">
<td colspan="14">(a) Due to the high error rates at Meadowdale High
School, we reviewed 100 percent of students in the transfer and extended
groups. </td>
</tr>
</tbody>
</table>

We performed our fieldwork at WOSPI offices in Olympia, Washington and
administrative offices of the three districts and nine schools. We held
an exit briefing with WOSPI officials on July 12, 2006. Our audit was
performed in accordance with generally accepted government auditing
standards appropriate to the scope of the review described above.

ADMINISTRATIVE MATTERS

Statements that managerial practices need improvements, as well as other
conclusions and recommendations in this report, represent the opinions
of the Office of Inspector General. Determinations of corrective action
to be taken will be made by the appropriate Department of Education
officials.

If you have any additional comments or information that you believe may
have a bearing on the resolution of this audit, you should send them
directly to the following Education Department official, who will
consider them before taking final Departmental action on this audit:

> Henry Johnson

Assistant Secretary

Office of Elementary and Secondary Education

U.S. Department of Education

400 Maryland Ave., SW

Washington, DC 20202

It is the policy of the U. S. Department of Education to expedite the
resolution of audits by initiating timely action on the findings and
recommendations contained therein. Therefore, receipt of your comments
within 30 days would be appreciated.

In accordance with the Freedom of Information Act (5 U.S.C. §552),
reports issued by the Office of Inspector General are available to
members of the press and general public to the extent information
contained therein is not subject to exemptions in the Act.

Sincerely,

/s/

> Gloria Pilotti

Regional Inspector General

for Audit

Attachments

**Attachment 1: WOSPI Graduation** **Rate Formulas and Calculations**

WOSPI uses the below formulas to calculate the on-time graduation rate
and extended graduate rate reported in the CSPR. The student numbers
used in the formulas are based on the student's enrollment status at the
end of the school year. The on-time graduation rate formula uses the
dropout rate in each grade level for that school year as a proxy for the
rates in previous years.

+-----------------------------------------------------------------------+
| **WOSPI's On-time Graduation Rate Formula**                           |
|                                                                       |
| (asterisk denotes multiplication)                                     |
+-----------------------------------------------------------------------+
| *On-time Graduation Rate:*                                            |
|                                                                       |
| 100\*(1 minus grade 9 dropout rate)\*(1 minus grade 10 dropout        |
| rate)\*                                                               |
|                                                                       |
| (1 minus grade 11 dropout rate)\*                                     |
|                                                                       |
| > (1 minus grade 12 dropout rate minus grade 12 continuing rate)      |
|                                                                       |
| *Formula used to derive grade-level dropout rates for the above:*     |
|                                                                       |
| Number of students with a dropout, unknown, GED completer code        |
|                                                                       |
| Total number of students served (minus transfers out)                 |
+-----------------------------------------------------------------------+

+-----------------------------------------------------------------------+
| **WOSPI's Extended Graduation Rate Formula**                          |
+-----------------------------------------------------------------------+
| *Extended Graduation Rate:* Number of on-time and late graduates      |
|                                                                       |
| Number of on-time graduates divided by on-time graduation rate        |
+-----------------------------------------------------------------------+

The OIG prepared the following charts to assist the reader in
understanding the computation of graduation rates using the actual
figures used by WOSPI.

![](media/image3.jpeg){width="6.499305555555556in"
height="4.768055555555556in"}

![](media/image4.jpeg){width="6.49375in" height="3.957638888888889in"}

**Attachment 2: WOSPI Comments to Draft Report**

![](media/image5.jpeg){width="6.493055555555555in"
height="8.389583333333333in"}

![](media/image6.jpeg){width="6.493055555555555in"
height="8.389583333333333in"}

![](media/image7.jpeg){width="6.493055555555555in"
height="8.389583333333333in"}

![](media/image8.jpeg){width="6.423611111111111in"
height="7.209722222222222in"}

[^1]: "Graduate" includes students who receive a regular high school
    diploma, students who receive an adult diploma from a community
    college program, and students with disabilities who complete their
    Individualized Education Plan (IEP). For school year 2003-2004, the
    statewide data file included 244 adult diploma recipients and 120
    IEP completers.

[^2]: For school year 2003-2004, the statewide data file included
    101deceased students.

[^3]: The revised rates are a non-statistical estimate of the impact of
    our audit results on the State's graduation rates. The revised rates
    were derived by adjusting the statewide total of students in each
    enrollment category using the error rates from our sample and
    recalculating the rates using the State's formulas and the adjusted
    statewide totals.

[^4]: The revised rates are a non-statistical estimate of the impact of
    our audit results on the State's reported dropout rates.

[^5]: The Washington Accountability Workbook was initially submitted to
    the Department in January 2003 and, after revision, approved by the
    Department in June 2004. The Department approved subsequent
    amendments to the Washington Accountability Workbook on September 1,
    2005 and July 19, 2006.

[^6]: The National Institute of Statistical Sciences/Education
    Statistics Services Institute Task Force's *Final Report on
    Graduation, Completion, and Dropout Indicators* (NCES 2005-105) and
    the Urban Institute's, *Keeping Count and Losing Count: Calculating
    Graduation Rates for All Students Under NCLB Accountability*.

[^7]: For school year 2003-2004, high schools and districts in the State
    of Washington must have an extended graduation rate of 66 percent or
    increase their rate by at least two percent from the prior school
    year to meet AYP.