![](media/image1.png){width="1.40625in" height="0.7291666666666666in"}

**ADDENDUM TO THE**

**PROPOSAL TO THE**

**UNITED STATES DEPARTMENT OF EDUCATION**

**FOR EMPLOYING A GROWTH MODEL FOR**

***NO CHILD LEFT BEHIND***

**ACCOUNTABILITY PURPOSES**

(Response to the request for additional information contained in the
December 15, 2006,

letter from Assistant Secretary Henry L. Johnson to

Ohio Superintendent of Public Instruction Susan Tave Zelmen)

**Submitted January 19, 2007**

**Ohio Contact:**

**Mitchell D. Chester**

**Senior Associate Superintendent for Policy and Accountability**

**Ohio Department of Education**

**25 South Front Street -- Mail Stop 708**

**Columbus, Ohio 43215**

**Phone: (614) 728-4510**

**Fax: (614) 728-2627**

***Ohio Growth Model Addendum***

This addendum is in response to the request for additional information
contained in the December 15, 2006, letter from Assistant Secretary
Henry L. Johnson to Ohio Superintendent of Public Instruction Susan Tave
Zelmen.

**Principle 1: Universal proficiency**

-   *Does the State's proposed growth model include the relationship
    between consequences and rate of student growth consistent with
    Section 1116 of the Elementary and Secondary Education Act of 1965
    (ESEA)?? (Principle 1.4)*

    -   *Has the State clearly described consequences the State/local
        educational agency (LEA) will apply to schools? Do the
        consequences meaningfully reflect the results of student growth?
        (Principle 1.4.1)*

        -   *Please clarify the interventions facing a school or LEA
            that does not meet adequate yearly progress (AYP) under the
            growth model and whether they are consistent with Section
            1116.*

> The interventions facing a school or LEA that does not meet adequate
> yearly progress (AYP) under the growth model are identical to those
> for schools and districts that miss AYP under the status model. In
> both cases, the interventions are consistent with Section 1116. These
> consequences include parental notification, public school choice,
> supplementary educational services, and other provisions that comply
> with Section 1116.
>
> Submitted with this addendum, the document *School
> Improvement/District Improvement Questions and Answers* outlines the
> interventions (this document may be found at:
> <http://www.ode.state.oh.us/GD/Templates/Pages/ODE/ODEDetail.aspx?page=3&TopicRelationID=132&ContentID=2151&Content=17826>).
> Under Ohio law, all interventions apply to all schools and districts,
> regardless of whether they receive Title I funds, with the exception
> of public school choice, supplemental educational services, and the 10
> percent set asides for professional development. These three
> interventions are not required of schools that do not receive Title I
> funds.
>
> Ohio believes that the addition of the projection methodology to the
> AYP calculation will help the state better identify the schools and
> districts about which it should be most concerned. With the data from
> the projection methodology, Ohio believes the Regional School
> Improvement Teams will have an increased ability to identify root
> causes of low achievement and provide intensive support to those
> schools and districts. This intensive support then can be aimed more
> precisely at areas where not only is achievement low but, in addition,
> where gains are not strong.

**Principle 2: Establishing appropriate growth targets at the student
level**

-   *Has the State proposed a technically and educationally sound method
    of depicting annual student growth in relation to growth targets?
    (Principle 2.1)*

    -   *Has the State adequately described a sound method of
        determining student growth over time? (Principle 2.1.1)*

        -   *Please clarify whether the growth model projections will be
            recalculated each year for each student.*

> The growth model projections being proposed for use in Ohio will be
> recalculated each year, and for each student. The time to reach
> proficiency will not be extended, however. Annual recalculations allow
> Ohio to update growth trajectories (based on the recalculated scores)
> to more precisely identify whether students are on track to reach
> proficiency within the initially-identified timeframe.

-   *Please provide further detail regarding the various grade
    configurations in Ohio. The department is most interested in the
    number of schools with kindergarten through grade 8 and whether
    students in these schools will have 6 years to reach grade-level
    proficiency.*

> In Ohio, there are 192 schools with a grade configuration ending in
> grade eight and beginning either in grade K, 1, 2 or 3.
>
> A student will be considered proficient if the student is projected to
> score at or above the Ohio's proficiency standard by the grade beyond
> the configuration of the school in which the student first was tested,
> or within four years (whichever occurs first) of the first test
> administration. This means, for example, that a third grade student in
> a K-8 school must be on track to meet the proficiency targets by the
> end of 7^th^ grade.

-   *Please clarify the methodology for the growth model projections,
    specifically the information that will be taken into account.*

    -   *Does the model create any situations where two students with
        the same reading score in year 1 will have different growth
        expectations in year 2?*

> The growth model projection proposed for use in Ohio does not create
> any situations where two students with the same achievement score in
> year 1 could have a different growth expectation in year 2. The
> projection model will incorporate all prior test scores available for
> a student. In a case where two students have the same set of past test
> scores, these students would have the same projected scores. This is
> due to the projected scores being determined entirely by the set of
> identical past scores.

-   *Please clarify whether the model will be based upon a student's
    prior test scores, or whether it will be based upon a trajectory of
    similar students.*

> Unlike traditional uses of value-added models where the standard for
> how much growth is good enough is conditioned on the past performance
> of similar students, the projection model Ohio will employ predicts
> individual student growth trajectories based solely on a student's
> past performance on state tests. The reviewer may have noted that, on
> page 14 of the proposal, the response indicated: "anticipated growth
> is based on the past performance of other low achieving students..."
> The response was intended to convey the notion that anticipated growth
> for students with the same test history would be the same. Again, the
> growth trajectory will be based upon a student's prior test scores,
> and not upon a trajectory of similar students.

-   *Please clarify whether different growth curves will be generated
    for students from different classrooms or different schools.*

> The growth model projection proposed for use in Ohio uses a "pooled
> within schools" variance-covariance matrix to produce the "pooled
> within schools" regression coefficients use to compute the
> projections. Since the growth curves will be generated entirely from a
> student's prior achievement scores, different growth curves will not
> be generated for students from different classrooms or different
> schools.

-   *Please clarify what variables will be used to calculate the
    regression for the growth model.*

> An individual student's prior achievement scores will be used in the
> growth model projection calculations. Other information, such as
> school-level, classroom-level, or demographic data will not be used in
> the projection calculations.

-   *Please clarify how instances of missing data will be resolved.*

> The growth model projection methodology proposed for use in Ohio is
> not based on the assumption that every student must have the same set,
> or number of predictors. In those instances where data are missing,
> the missing data will not be imputed. The methodology handles missing
> data by finding the "pooled within schools" covariance matrix of all
> prior test scores and the predicted score. From this matrix,
> regression coefficients are obtained and used to obtain the projection
> parameters. This flexible way of handling missing data ensures that
> highly mobile populations are not excluded from the growth model
> projections.

**Principle 3: Accountability for reading/language arts and mathematics
separately**

-   *Has the State proposed a technically and educationally sound method
    of holding schools accountable for student growth separately in
    reading/language arts and mathematics? (Principle 3.1)*

    -   *Are there any considerations in addition to the evidence
        presented for Core Principle 1? (Principle 3.1.1)*

        -   *Please clarify how the model accounts for student
            performance on each subject assessment (mathematics,
            reading, science, social studies, and any additional
            assessments), including the weighting that will be used.*

> Ohio's projection model will compute projected scores separately for
> reading/language arts and mathematics. The projected scores for each
> subject will be computed from the test data that are available. Thus,
> when determining whether a school, district, or subgroup has met the
> annual proficiency target in reading/language arts, students'
> projected scores in reading/language arts will be evaluated in
> conjunction with students' reading/language arts scores which are
> proficient or above. Likewise, when determining whether a school,
> district or subgroup has met the annual proficiency target in
> mathematics, students' projected scores in mathematics will be
> evaluated in conjunction with students' mathematics scores which are
> proficient or above.
>
> In an effort to dampen measurement error and thus increase
> reliability, the projection methodology will incorporate all available
> prior testing data in the computation of projected future performance.
> The Ohio projection methodology will utilize prior test scores in
> reading, mathematics, science and social studies. The projected score
> in reading will computed from prior reading, mathematics, science and
> social studies scores. The projected score in mathematics will be
> computed from prior mathematics, reading, science and social studies
> scores. Since the correlation between scores within each content area
> is larger than the correlation between scores of differing content
> areas, the prior test scores from the same content area will have the
> greatest predictive power in the statistical model. Restated,
> projected scores in a given content area largely are determined by the
> student's prior achievement in the same content area. This makes it
> very unlikely for a student's prior achievement in another content
> area to overshadow poor performance in the content area for which the
> projected score is computed.

-   *Please clarify that the achievement on other assessments does not
    compensate for failure to achieve proficiency in reading/language
    arts and mathematics. Please clarify how this will be demonstrated.*

> As explained above, the projected scores in a given content area
> largely are determined by the student's prior achievement in the same
> content area. While the projected scores for reading/language arts
> will utilize prior achievement in reading/language arts, mathematics,
> science and social studies, prior achievement in reading will have
> greater predictive power in the prediction model. Thus, if
> reading/language arts performance declined while the performance in
> the other content areas remained steady or improved during the same
> period, the predicted reading/language arts score likely would
> decline.

**Principle 4: Inclusion of all students**

-   *Does the State's growth model proposal address the inclusion of all
    students, subgroups, and schools appropriately? (Principle 4.1)*

    -   *Does the State's growth model proposal address the inclusion of
        all students appropriately? (Principle 4.1.1)*

        -   *Please clarify whether the growth model will be applied to
            all students in every school in the State, including
            students taking an alternate assessment.*

> Because the projection model exploits all available data and accounts
> for missing records, the projection model used in Ohio will be applied
> to students in all subgroups for schools and districts with tested
> grades. Additionally, measures of percent projected proficiency can be
> produced for any school whose students have at least one year of prior
> test records regardless of previous building location ensuring that
> new and reconfigured schools will be included in the accountability
> model. For students for whom insufficient prior test data are
> available for projections, their current proficiency status will be
> used in the projected percent proficiency calculation. This ensures
> that all students will be included in the calculation. Additionally,
> the projected percent proficient calculation will include alternative
> assessment status results for those students for whom alternative
> assessments have been deemed appropriate.

**Principle 5: State assessment system and methodology**

-   *How will the State report individual student growth to parents?
    (Principle 5.2)*

    -   *How will an individual student's academic status be reported to
        his or her parents in any given year? What information will be
        provided about academic growth to parents? Will the student's
        status compared to the State's academic achievement standard
        also be reported? (Principle 5.2.1)*

        -   *Please clarify whether a student's growth and trajectory
            will be reported to parents and what, if any, additional
            information will be provided.*

> At the present time, Ohio plans to produce student-level reports that
> may be shared with parents in accordance with sections
> \<rule_text\>3301.0711, 3301.0714, and 3319.321 of the Revised Code
> and the Family and Educational Rights and Privacy Act (FERPA), 20
> U.S.C. 1232g. Individual student test scores and individual student
> reports shall be made available only to a student\'s classroom
> teacher, other appropriate educational personnel, and the student\'s
> parent or guardian.\</para_first\>­\</level1\>­
>
> Additionally, in accordance with paragraph (C) of rule 3301-13-06 of
> the Administrative Code, Ohio requires that each school district and
> participating school adopt a policy and establish procedures for
> releasing or permitting access to a student\'s proficiency,
> achievement, or diagnostic test results. The policy and procedures
> shall comply with paragraphs (G) and (H) of rule 3301-13-01 of the
> Administrative Code and with section 3319.321 of the Revised Code.

**Principle 6: Tracking student progress**

-   *Has the State designed and implemented a technically and
    educationally sound system for accurately matching student data from
    one year to the next? (Principle 6.1)*

    -   *Is the system proposed by the State capable of keeping track of
        students as they move between schools or school districts over
        time? (Principle 6.1.2)*

        -   *Please provide any studies or evidence available regarding
            the student tracking system and the capability of Ohio to
            track students over time and across schools.*

> A key step in securing Ohio's individual, longitudinal student-data
> reporting system is the Statewide Student Identifier system (SSID).
> Through an independent vendor, this system assigns a unique student
> identification number to all 1.8 million K-12 public school children
> in the state. This mandatory identifier is associated with each
> student as s/he moves between schools and districts within Ohio. To
> facilitate the process of tracking students over time, the SSID
> application uses matching logic to search for records that match
> identifying student attributes before creating a new SID (or to search
> for an existing SID record a user wishes to modify) In addition, the
> system uses SoundEX techniques where appropriate to search for
> existing records. The SoundEX function searches for similar-sounding
> words or names in the database to identify records that match the
> criteria input. Several factors weigh in the student match process.
> The system first searches on the following nine mandatory elements:

-   Legal First Name

-   Legal Middle Name

-   Legal Last Name

-   Gender

-   Date of Birth

-   Birth Place City

-   Race/Ethnicity

-   Native Language

-   Admission Reason

> If multiple records are located, the matching logic then searches
> based on additional student attributes to narrow the search. Based on
> the combination of these attributes, the logic determines the outcome
> of the Matching Entries Process. To assist with tracking students over
> time, the independent vendor maintains a crosswalk that contains the
> information necessary to match SSIDs to student records, and thus
> increase the number of student records tracked over time.

-   *What studies have been conducted to demonstrate the percentage of
    students who can be "matched" between two academic years? Three
    years or more years? (Principle 6.1.4)*

    -   *Please provide additional information on the match rates for
        three years for the whole population and by subgroup.*

> As stated in Principle 2.1.1 (above), the growth model projection
> methodology that will be used in Ohio is not based on the assumption
> that every student must have the same set, or number of predictors. In
> those instances where data are missing, the methodology handles
> missing data by finding the "pooled within schools" covariance matrix
> of all prior test scores and the predicted score. From this matrix,
> regression coefficients are obtained and used to obtain the projection
> parameters. This flexible way of handling missing data ensures that
> highly mobile populations are not excluded from the growth model
> projections.
>
> The percentage of students who were "matched" over the previous two
> and three academic years is shown in the table below. The SSID system
> is now in its fifth year of implementation. With each passing year,
> Ohio has developed increasingly sophisticated means of securing the
> reliability of matches. As a result, each year Ohio has secured higher
> match rates for each subgroup than in previous years, and expects even
> higher two and three year match rates when the 2006-07 data are
> included.

  ---------- ---------- ------- -------------- ------- ------- -------- ---------- --------------- ------ ------
             All        Black   Multi-racial   White   Asian   Native   Hispanic   Economically    SWD    LEP
             Students                                          Am.                 Disadvantaged          

  Matches    95.9       93.8    94.1           96.8    89.1    91.7     90.0       95.1            95.9   87.9
  between 2                                                                                               
  years                                                                                                   

  Matches    93.4       90.7    89.9           94.6    81.7    82.2     85.3       92.1            93.5   81.1
  between 3                                                                                               
  years                                                                                                   
  ---------- ---------- ------- -------------- ------- ------- -------- ---------- --------------- ------ ------

-   *Does the State student data system include information indicating
    demographic characteristics? (Principle 6.1.5)*

    -   *Please clarify the information collected as part of the
        Statewide Student Identifier system (SSID).*

> In Ohio's data management system, the following elements are included
> with each individual SSID record:

-   Legal First Name

-   Legal Middle Name

-   Legal Last Name

-   Date of Birth

-   Gender

-   Native Language

-   Race/Ethnicity

-   Birthplace Name

-   Admission Reason

-   Disability Condition

-   Disadvantagement

-   Limited English Proficiency Status

-   Limited English Proficiency Reclassification Date

-   Migrant Status

-   Student Being Served by a 504 Plan

```{=html}
<!-- -->
```
-   *How does the proposed State growth accountability model adjust for
    student data that are missing? (Principle 6.1.6)*

    -   *Please clarify whether the model will impute values and, if so,
        the procedure for doing so.*

> As stated in Principle 2.1.1 (above), the growth projections will not
> use imputed values for missing data.

-   *Does the State data infrastructure have the capacity to implement
    the proposed growth model? (Principle 6.2)*

    -   *How does the proposed growth model take into account or
        otherwise adjust for decreasing student match rates over three
        years or more? How will this affect the school accountability
        criteria? (Principle 6.2.3)*

        -   *Please provide further data for the match rates of the SSID
            and explain the procedures to account for decreasing match
            rates over time.*

> The growth model projection methodology estimates projection scores
> for all students with prior achievement scores- even if the student
> has missing achievement scores. For students for whom insufficient
> prior test data are available for projections, their current
> proficiency status will be used in the projected percent proficiency
> calculation, thus ensuring that all students will be included. With
> the current procedures used to facilitate the process of tracking
> students over time (matching logic system), coupled with the proposed
> projection methodology, decreasing match rates over time is not
> expected to adversely impact Ohio's capacity to implement the proposed
> growth model.