Relating Knowledge about Reading to Teaching Practice: An Exploratory Validity Study of a Teacher Knowledge Assessment
Author: Phelps, Geoffrey
Date: 2009-00-00
Type: Reports - Research
Subjects: Video Technology,Grounded Theory,Teacher Characteristics,Knowledge Level,Program Effectiveness,Teaching Methods,Test Validity,Scores,Test Results,Evaluation Utilization,Teacher Effectiveness,Instructional Effectiveness,Reading Instruction,Teacher Evaluation,Reading Programs,Program Evaluation,Elementary School Teachers,Teacher Surveys,Questionnaires,Observation,Coding,Case Studies,Qualitative Research,Psychometrics

The research reported in this paper is focused directly on assessing the validity of the &quot;Teaching Knowledge about Reading and Reading Practices&quot; (TKRRP) assessment. Following the recommendations of the Standards for Educational and Psychological Testing (APA/AERA, 1999), the authors see validation as a process of constructing an argument that builds evidence for valid inferences that can be drawn regarding the proposed uses of assessment scores. In this study, the authors investigate a central claim about the knowledge they are seeking to measure with the TKRRP to inform their understanding of the inferences that can be validity drawn from the test scores--that is, &quot;teacher's scores on the Teacher Knowledge of Reading and Reading Practices (TKRRP) are associated with the quality of the teacher's reading instruction.&quot; The study was carried out in the context of the Michigan Reading First evaluation. All first through third grade teachers (n=1,635) in schools participating in Reading First were administered a questionnaire containing the &quot;Teaching Knowledge about Reading and Reading Practices&quot; in the spring of 2008. The validation study employs two different approaches to investigate the validity of the TKRRP. First the authors make use of the data from a systematic coding of the quality of teachers' lessons using the Video Assessment of Teaching (VAT-R). They use a two level HLM with lessons nested within classrooms to predict variation in lesson quality. In a parallel analysis using the same sample of 31 teachers, they conducted a qualitative case study comparing high knowledge teachers and low knowledge teachers. In this analysis they use constant comparative techniques to investigate characteristics of the reading instruction of teachers with high levels of knowledge compared to teachers with low levels of knowledge. Preliminary results suggest a number of interesting findings. Unsurprisingly, many of the same features of quality that were prominent in the VAT-R coding were also noted in the qualitative analysis. However, there were also noticeable differences in instruction that were not represented in the VAT-R. For example, preliminary analyses suggest that the instruction of the higher knowledge teachers differed from the lower knowledge teachers. Higher knowledge teachers appeared to provide more elaborate and accurate descriptions of ideas and linked classroom work more directly to literacy concepts. High knowledge teachers asked more demanding questions and probed student responses more deeply. High knowledge teachers tended to use language more precisely to explain concepts and elaborate on classroom discussions. While these findings are preliminary, they do suggest that there were noticeable differences in the reading instruction observed for the high and low knowledge cases. Measures are appended. (Contains 1 figure and 1 table.)