# Presentation: 688903

## Reading First AcademyAssessment Committee

**David Francis, University of Houston**
**Lynn Fuchs, Vanderbilt University**
**Roland Good, University of Oregon**
**Rollanda O’Connor, University of Pittsburgh**
**Deborah Simmons, University of Oregon**
**Gerald Tindal, University of Oregon**
**Joseph Torgesen, Florida State University **

**Team Leader Edward J. Kame’enui, University of Oregon**

- Assessment Committee

- 2002

**Notes:**
Notes:

## Assessment in the Context of State Accountability

**Assessment can provide a vivid, public report of reading outcomes.**
**Assessment can inform instruction to help schools, teachers, and children achieve important reading outcomes.**
**Accountability is crucial to the educational agenda.**
**31 states use norm-referenced tests for accountability, 33 use criterion referenced tests, and 19 use both (Bond, Roeber, & Connealy, 1998).**
**Accountability for reading outcomes is central to *****Reading First Initiative*****.**

- Assessment Committee

- 2002

**Notes:**
Notes:
Only 5 states do not use use criterion-referenced or norm-referenced tests for accountability.
		Good, R. H., Simmons, D. C., & Kame'enui, E. J. (2001). The importance and decision-making utility of a continuum of fluency-based indicators of foundational reading skills for third-grade high-stakes outcomes. Scientific Studies of Reading, 5, 257-288.

## Accountability for Grade Level Reading Outcomes

**Frequently, statewide accountability assessments are given only at selected grades, so accountability for grade level reading outcomes raises two questions:**
***Do students meet state standards on the current state assessment?*****  Students are *****at grade level***** if they meet expectations for reading proficiency on a state accountability assessment.  **
***Are students predicted to meet state standards on the next state assessment?***** Students are *****at grade level***** if they are likely to meet the state standard on a later accountability assessment (assuming effective comprehensive reading instruction). **

- Assessment Committee

- 2002

**Notes:**
Notes:
If a state accountability assessment isn’t given until third grade, for example, then we need some way of finding out if children are on track for that accountability assessment.  
First, it makes no sense to place all of the stress and responsibility on the third grade teachers only when third grade outcomes are the culmination of K through third grade instruction.
Second, waiting until third grade to assess outcomes and accountability limits our ability to do something about it.  At kindergarten, first, and second grade we need accountability for progress and outcomes in time to make adjustments in instruction to change third grade outcomes.  
So, even if the first formal state accountability assessment does not occur until third or fourth grade, we need intermediate accountability assessments to show that children are on track and are likely to meet the state accountability assessment.  

## Example: Oregon Statewide Assessment in Reading and Literature

**The Oregon Statewide Assessment in Reading and Literature (OSA) is a standardized reading achievement test developed by a panel of experts and a research and development company.  **
**Standards are established by the Oregon State Board of Education.  **
**In third grade, a score of *****201 or higher***** is judged to “*****meet expectations*****” and a score *****below 201***** is described as “*****does not meet expectations*****.”  **
**The OSA is repeated in grades 3, 5, 8, and 10.  **
**In third grade, a score of *****212 or higher***** is *****predictive of success***** on the 5th and 8th grade state accountability assessments. **

- Assessment Committee

- 2002

**Notes:**
Notes:
Longitudinal research can examine the predictive validity of earlier grade accountability goals for later grade state accountability assessments.  
Oregon is used here only as an example.  Similar research and conclusions can be expected for other state accountability assessments.  
Two questions are important: Are students meeting state accountability standards now? And, are they on track to meet later state accountability standards.  
For example, what about a student who receives a score of 202 on the OSA in third grade?  That student would be rated as “meets expectations” for third grade, but they may not be rated as “meets expectations” on the fifth grade and eighth grade assessments.  

## First Grade Reading Predictive of Later State Accountability Outcomes

**40 or more: *****predictive of success***** on later state accountability outcomes, 51 out of 58 or 88%.**
**10 to 39:  uncertain of later state accountability outcomes, 37 out of 64 or 58%.**
**Below 10: *****predictive of difficulty***** on later state accountability outcomes, 18 out of 27 or 22%.**

***r***** = .57, *****p***** < .001,**
**33% of variance explained**

- Assessment Committee

- 2002

**Notes:**
Notes:
In first grade, a strong prediction that a student is on track for the third grade state accountability assessment is possible.  
No prediction is ever perfect -- a perfect prediction would mean that second and third grade instruction did not matter in changing outcomes. In first grade, the goal is to reach a level of reading skill where the odds are that the child will meet state accountability standards.  
Reading the picture (optional discussion).  Each dot represents a child.  If you go straight down from the dot, the score represents the child’s first grade performance.  If you go straight left from the dot, the score represents the child’s third grade performance on the state accountability assessment.  
If you cover all the dots for children who scored below 40 in first grade (to the left of the green line) and look only at the children who met the first grade goal, all met the third grade state accountability standard. And, 88 percent were predicted to meet the fifth and eighth grade state accountability standard.
If you cover all the dots for children who score above 10 in first grade (to the right of the red line) and look only at the children who were predicted to experience difficulty on the state accountability standard, only 22 percent were predicted to meet fifth and eighth grade state accountability standard.

## Levels of Reading Outcomes

**For young children in the early grades, the purpose of assessment is to identify difficulties early so as to *****provide additional instructional intervention***** to achieve grade level reading outcomes.  **
_**At Grade Level**_**: Students on track to achieve reading outcomes with an effective comprehensive reading program.**
_**Needs Additional Intervention**_**: Students who need *****some***** additional instructional intervention above and beyond an effective comprehensive reading program to achieve grade level outcomes. **
_**Needs Substantial Intervention**_**: Students who need *****substantial***** additional instructional intervention to achieve grade level outcomes. **

- Assessment Committee

- 2002

**Notes:**
Notes:
Especially in the early grades, one purpose of accountability assessment is to provide intervention to change the prediction.  Predictions of difficulty on later state accountability assessments are only accurate if we don’t do anything to change the outcome.  
A second purpose of accountability assessment at each grade is to inform decisions about curriculum, instruction, and intervention to get more children on track for later state accountability assessments.  

## District End-of Year Reading Outcomes

**28% At Grade Level**
**57% Need Additional Intervention**
**15% Need Substantial Intervention**

**Oral Reading Fluency**

**1998 – 99 First Grade Reading**

**Number of children**

- Assessment Committee

- 2002

**Notes:**
Notes:
This slide and the next three slides tell the story of a district that used first grade outcome assessment to make decisions about their curriculum, instruction, intervention, and allocation of resources.  
In the first year, 1998-1999, they found that only 28% of their children were on track for success on the 3rd, 5th, and 8th grade state assessments.  For 57% a clear prediction was not possible, and for 15% difficulty passing was predicted unless substantial intervention was provided.  

## Outcome Assessment Informs District Actions to Change Reading Outcomes

**Based on the Year 1 reading outcome assessment, the district engaged in a school-wide reform process to accelerate reading outcomes. The schools:**
**1. Decided the obtained reading outcomes were unacceptable and substantial change was needed.**
**2. Selected a research-based comprehensive reading program from a menu of options.**
**3. Implemented a system to identify early children who were at risk of reading difficulty.**
**4. Implemented a system of early intervention and additional instructional intervention to change outcomes.**

- Assessment Committee

- 2002

**Notes:**
Notes:
The first grade accountability assessment was only one piece of the change process, but it provided a clear basis for a decision that a change was needed, and it provided a clear goal for the reform process.  

## Same School District, 1 Year Later

**57% At Grade Level**
**36% Need Additional Intervention**
**6% Need Substantial Intervention**

**Oral Reading Fluency**

**1999 – 00 First Grade Reading**

***After District Changes***

**Number of children**

- Assessment Committee

- 2002

**Notes:**
Notes:
In the second year, 1999-2000, for their next cohort of first graders, the results of the district’s work and effort to reform their curriculum, instruction, intervention, and allocation of resources was clearly and vividly apparent.  They doubled the percent of children at grade level, and they halved the number of children predicted to experience difficulty on the state accountability assessment without substantial intervention.  
Comparing Slide 9 with Slide 7 illustrates the clear difference in outcomes achieved by the district.
For this district, accountability for outcomes provided a basis for celebration -- and renewed effort.  

## Outcome Assessment Informs District Decisions and Actions

**Based on the Year 2 reading outcome assessment, the district decided:**
**They were making dramatic changes in the reading outcomes for children in their district.**
**Renewed effort was still needed, including continued professional development to implement a comprehensive reading program with increasing fidelity.**
**They needed to institute a system of increasing the intensity of instructional interventions for students who require more intensive intervention to be at grade level.**

- Assessment Committee

- 2002

**Notes:**
Notes:
Of course, it is possible that the change was due to a different group of students or to a change in the district demographics.  By including a measure of beginning skills for each group of students and by examining the overall pattern of change over time, other explanations for changes in outcomes can be examined for the district.  

## Reporting Outcome Assessment

**Outcomes in the important beginning reading core areas *****should be***** reported at the beginning and end of the academic year.  **
**Phonemic Awareness**
**Phonics**
**Fluency**
**Comprehension**
**Vocabulary**
**The following assessment framework specifies the important beginning reading skill areas to be assessed at the beginning and end of each grade level.**

- Assessment Committee

- 2002

**Notes:**
Notes:
Accountability assessment should focus on outcomes in the important beginning reading areas.  The important beginning reading areas are the ones where we can make the most difference in reading outcomes for children.
All beginning reading areas are not equally important at different ages.  For example, foundation skills are more important early -- reading comprehension skills are more important later.  

## Kindergarten Outcome Assessment

|  | Suggested Beginning of year | OptionalMiddle ofYear | Suggested 
End ofYear |
| --- | --- | --- | --- |
| Phonemic Awareness | X | X | X |
| Phonics |  | X | X |
| Fluency |  |  |  |
| Comprehension |  |  |  |
| Vocabulary | X | X | X |

- Assessment Committee

- 2002

**Notes:**
Notes:
These charts show a recommended minimum assessment schedule.  Districts may assess more frequently or in more skill areas, but an assessment of phonemic awareness and vocabulary at the beginning and end of kindergarten and a measure of phonics at the end of kindergarten are suggested.  

## First Grade Outcome Assessment

|  | Suggested Beginning of year | OptionalMiddle ofYear | Suggested 
End ofYear |
| --- | --- | --- | --- |
| Phonemic Awareness | X | X | X |
| Phonics | X | X | X |
| Fluency |  | X | X |
| Comprehension |  |  | X |
| Vocabulary | X | X | X |

- Assessment Committee

- 2002

**Notes:**
Notes:
In later grades, once children have foundation reading skills, the focus of assessment shifts to fluency and reading comprehension.  

## Second Grade Outcome Assessment

|  | Suggested Beginning of year | OptionalMiddle ofYear | Suggested 
End ofYear |
| --- | --- | --- | --- |
| Phonemic Awareness |  |  |  |
| Phonics | X |  |  |
| Fluency | X | X | X |
| Comprehension | X | X | X |
| Vocabulary | X | X | X |

- Assessment Committee

- 2002

**Notes:**
Notes:
For most children, assessment of phonemic awareness and phonics skills is less important in later grades.  

## Third Grade Outcome Assessment

|  | Suggested Beginning of year | OptionalMiddle ofYear | Suggested 
End ofYear |
| --- | --- | --- | --- |
| Phonemic Awareness |  |  |  |
| Phonics |  |  |  |
| Fluency | X | X | X |
| Comprehension | X | X | X |
| Vocabulary | X | X | X |

- Assessment Committee

- 2002

**Notes:**
Notes:

## Suggested District-Level Reading Outcome Summary

**Overall, end-of-year, number and percent of children who are:**
**At grade level**
**Need additional intervention**
**Need substantial intervention**
**Percent of children who are at grade level by racial and ethnic status, and by free/reduced lunch status as an indicator of low income.**
**Change in percent of children who are at grade level compared to:**
**Beginning of year percent at grade level**
**Prior year percent at grade level**

- Assessment Committee

- 2002

**Notes:**
Notes:
To be maximally useful, accountability assessment should be focused and clear.  The purpose is not to overwhelm users and consumers with a morass of scores, subtests and numbers.  For the example district illustrated earlier, the first year they had 26% at grade level at the end of first grade.  The next year they had 57% at grade level.  
Providing a clear goal for each grade and providing clear information about progress toward the goal can be an important support for a district in changing reading outcomes and meeting state accountability assessments.  

## Four Kinds of Reading Assessments

***Outcome***** ****- Assessments that provide a bottom-line evaluation of the effectiveness of the reading program.**
***Screening***** - Assessments that are administered to determine which children are at risk for reading difficulty and who will need additional intervention.**
***Diagnosis***** - Assessments that help teachers plan instruction by providing in-depth information about students’ skills and instructional needs.**
***Progress Monitoring***** ****- Assessments that determine if students are making adequate progress or need more intervention to achieve grade level reading outcomes.**

**An effective, comprehensive, reading program includes reading assessments to accomplish four purposes:**

- Assessment Committee

- 2002

**Notes:**
Notes:
Accountability or outcome assessment should be one piece of an assessment system.  All pieces of the system need to be in place and need to work together to support reading outcomes for our children.  Other pieces of a complete assessment system will be discussed in the assessment presentation.