# Presentation: 109293

## Supporting Effective Instruction: Technology and Professional Development 

- Carol McDonald Connor
- Frederick J. Morrison
- Christopher Schatschneider
- Barry Fishman

## Coming to PD through the back door

- Individualizing Student Instruction Study
- RFT to test the impact of child characteristics-by-instruction interactions on student outcomes
- Aptitude by treatment interactions

## Designing the PD Protocol

- Used the research base to develop a “state of the art” professional development protocol,
- delivered to all treatment group teachers
- We assumed that, with our coaching, the teachers would be able to implement ISI in the first experimental year with enough fidelity so that we could test our hypotheses
- most PD research suggests that it takes 2 to 3 years for teachers to fully implement a new practice

## What was the Intervention?

- Created algorithms based on the HLM results in our Beyond the Reading Wars paper
- Assessment to Instruction (A2i) software created to compute recommended amounts and types of instruction for each child
- Embedded in planning software design to scaffold research-based reading practices

## Slide 5

## Multiple Dimensions of Instruction

|  | Teacher Managed | Child Managed |
| --- | --- | --- |
| Code Focused
 | Alphabet activities
Letter Sight-Sound
Phonological Awareness
   Onset-rime, blending and         
   segmenting
   Word Segmentation | Spelling
phonics worksheets, handwriting activities decoding activities
 |
| Meaning Focused | Vocabulary
Teacher Read Aloud
Student Read Aloud, Choral
Group Writing, Writing Instruction, Model Writing
Listening Comprehension
Discussion | Student Buddy Reading
Sustained Silent Reading
Reading Comprehension 
  worksheets
Student Individual Writing
Pair writing |

**Notes:**
HLM to computer

## Slide 7

## Algorithm Results TM-CF

- TMCFa = ((End of Year Target - (.2* LW GE))/(.05 + (.05 * LW)))+13.
- TMCF_Recommended = (TMCFa - (.82 * Month )).

## Algorithm Results for CM-MF

- CMMFa = ((3.76 – End of year target + (1.4 * Vocabulary AE))/(.30)) - 14.
- CMMFsl = 10-(.24*CMMFa).
- CMMF_Recommended= CMMFa + .5*(CMMFsl * Month).

## Slide 13

## PD Protocol

- Focused on using A2i to plan instruction and then on implementing the recommended amounts for each child in the classroom
- Grouping
- small group rather than whole class
- using A2i recommended groupings (homogeneous)
- Classroom organization
- Effective child-managed activities
- Individualizing time, content, and delivery
- no one way to individualize instruction
- Research-based practice

## Mentor or Coaching Model

- “Research Partners”
- classroom based – 2 hours bi-weekly
- School level meetings
- Individual meetings
- Individualized
- Same amount of time
- Content and focus varied

## Did the PD work?

- Evidence from child outcomes
- Evidence from A2i software tracking
- Evidence from changes in teacher knowledge
- Evidence from classroom observations

## Evidence from Child OutcomesHLM - Treatment versus Control Student Reading Comprehension Outcomes

- Mean scores controlling for fall vocabulary, passage comprehension, letter-word reading, curriculum, FARL, and Reading First status. 464 = GE 1.8, 468 = GE 2.0, n = 616 students

## Evidence from Software TrackingA2i Use and Reading Comprehension

- HLM fitted growth curves controlling for fall vocabulary, letter-word reading, curriculum, FARL, and Reading First status. 464 = GE 1.8, 468 = GE 2.0,

- AE = 8.2 years

- AE = 6.0 years

## Evidence from changes in Teacher Knowledge

- Teacher Knowledge Test Descriptives and Reliability
- Assesses teachers’ understanding of English phonology, orthography, and morphology, and concepts of literacy acquisition and instruction
- 34 multiple choice and 11 short answer items
- Administered fall and spring
- Cronbach’s alpha = .87.
- Teachers in the treatment group had significantly higher spring TKS scores compared to control teachers
- Controlling for school SES status, other PD opportunities, teacher credentials and fall TKS scores
- standardize beta = .40
- A2i and TKS correlations
- Fall TKS and A2i total use did not correlate
- Spring TKS and A2i correlation = .58

## Importance of Teacher Knowledge 

- First Graders – end of year grade equivalent score of 1.9 = 428

- Scores on the TKS ranged from 9 to 36 (*M* = 23.45, *SD* = 7.27).

## Evidence from Classroom Observations

- Child-managed
- Pair
- 4.1. Literacy Codes:
- 	4.1.2. Phoneme Awareness
- 	4.1.3. Syllable Awareness
- 	4.1.4. Morpheme Awareness
- 	4.1.5. Onset/Rime Awareness
- 	4.1.6. Word ID/Decoding
- 	4.1.7. Word ID/Encoding
- 	4.1.8. Fluency
- 	4.1.9. Print Concepts
- 	4.1.10. Oral Language
- 	4.1.11. Print Vocabulary
- 	4.1.12. Reading Comprehension
- 	4.1.13. Text Reading
- 	4.1.14. Writing
- 	4.1.15. Library
- 	4.1.16. Assessment

- 4.1.2. Phoneme Awareness

- 4.1.2. Phoneme Awareness
- 	4.1.2.2. Blending
- 	4.1.2.3. Elision/Initial
- 	4.1.2.4. Elision/Final
- 	4.1.2.5. Elision/Vowel
- 	4.1.2.6. Elision/Medial
- 	4.1.2.7. Substitution/Initial
- 	4.1.2.8. Substitution/Final
- 	4.1.2.9. Substitution/Vowel
- 	4.1.2.10 Substitution/Medial
- 	4.1.2.11 Segmenting/Counting

**Notes:**
Here is a screenshot of our coding software, and, to give you a sense of the detail with which we are able to code, here is a list of our instructional categories.  Each category is further broken down into specific activities.  For example,...

## TCM Small-group Code-focused

## Teacher-Managed Instruction

- Whole Class

- Small Group

**Notes:**
Descriptive Statistics

 	treatment	Mean	Std. Deviation	N	
TM-Whole Class-Code Focused	.00	.0000	.00000	227	
 	1.00	.0000	.00012	186	
 	Total	.0000	.00008	413	
TM-Whole Class-Meaning Focused	.00	.2739	1.14832	227	
 	1.00	.5086	1.29146	186	
 	Total	.3796	1.21898	413	
TCM-Whole Class-Code Focused	.00	9.4729	8.54112	227	
 	1.00	14.1571	13.77630	186	
 	Total	11.5825	11.43157	413	
TCM-Whole Class-Meaning Focused	.00	20.3966	19.69005	227	
 	1.00	21.7216	12.53186	186	
 	Total	20.9933	16.84113	413	

## Child Managed Instruction

## Winter Observed – A2i recommended amounts 

- Simple Differences

- Distance From Recommendation Absolute Values

- *

## Effect of Distance from Recommendations

- 464 = 1.8, 470 = 2.1 or about a 3 month difference in GE

- 450 = 1.9, 458 = 2.5  .6 GE or about a 5 and a half month difference in GE

- Winter TM-CF DFR

- Winter CM-MF DFR

**Notes:**
464 = 1.8, 470 = 2.1 about a 3 month difference in GE

## HLM - DFR predicting student outcomes

- Cumulative fall, winter and spring DFR for TM-CF and CM-MF
- DFR Treatment teachers < DFR Control Teachers
- Total Amounts of instruction did not predict student outcome growth (residualized change)
- Cumulative TM-CF and CM-MF DFR negatively predicted both Passage Comprehension and Letter-Word Identification residualized change
- TM-CF DFR amount and change fall to spring
- CM-MF DFR amount

## Discussion

- Finding effective ways to change teacher practice quickly are critical
- For Random Field Trials
- Improving student learning
- Why did our PD work?
- Practice-based PD versus General Knowledge
- Focus on A2i, which was designed to scaffold the kinds of instruction research suggests are more effective in improving students’ reading skills
- Did the technology really make a difference?
- We don’t know – and will need to conduct a RFT to find out