Identification of Student- and Teacher-Level Variables in Modeling Variation of Mathematics Achievement Data
Author: Tarr, James E.
Date: 2010-00-00
Type: Reports - Research
Subjects: Mathematics Education,Teacher Characteristics,Mathematics Achievement,Program Effectiveness,Data Analysis,Models,Longitudinal Studies,Mathematics Instruction,Curriculum Implementation,Secondary School Curriculum,Instructional Effectiveness,Correlation,Predictor Variables,Student Characteristics,Comparative Analysis,Educational Assessment,Item Response Theory,Educational Testing,Evaluation Research,Evaluation Problems,Program Evaluation,Curriculum Evaluation,Curriculum Research,Mathematics Curriculum,Statistical Bias,Textbooks,High School Students,Mathematics Teachers,Factor Analysis

The Comparing Options in Secondary Mathematics: Investigating Curriculum (COSMIC) project is a longitudinal study of student learning from two types of mathematics curricula: integrated and subject-specific. Previous large-scale research studies such as the National Assessment of Educational Progress (NAEP) indicate that numerous variables are associated with mathematics performance; however, consideration of too many variables can result in biased models that inflate the amount of variation explained. This paper addresses key issues in the design of studies of curricular effectiveness including data collection, data reduction, and coherence in modeling student achievement in Year 1. At the student-level, demographic and prior achievement data was collected for 2621 secondary school students in five states. These scores were transformed onto NAEP scales to yield a COSMIC Prior Achievement (CPA) score that provided a control for prior achievement across the entire sample. At the teacher-level, data related to teaching experience, professional development, teacher beliefs, and curriculum implementation was collected. Student learning was measured using the Iowa Test of Educational Development: Mathematics (Level 15) and two project-developed assessments (Test A and Test B) that embodied curricular validity (National Research Council, 2004). Item Response Theory (IRT) scaled student scores were adjusted for CPA and subsequently aggregated by teacher. Seven factors were extracted from 24 teacher-level variables through principle components analysis, four related to curriculum implementation and three to teacher characteristics. Curriculum Type, Percent Free/Reduced Lunch (%FRL), and Lesson Development variables were examined in relation to student outcomes. Full and partial correlations procedures were used to identify key variables associated with student mathematics learning. Significant correlations included Curriculum Type with student scores on Test A and Test B; %FRL correlated (negatively) with scores on Test A and ITED-15; Technology &amp; Collaboration with scores on Test B and ITED-15; Opportunity-to-Learn (OTL) with performance on all three outcome measures; and Knowledge of Standards with scores on Test A and Test B. When controlling for %FRL, Curriculum Type was significantly correlated with all three dependent measures. A similar pattern emerged when controlling for OTL, suggesting a relationship between OTL and %FRL. In contrast, controlling for Technology &amp; Collaboration weakened the correlations between Curriculum Type and all three assessments. Recommendations related to key issues in data collection, data reduction, and coherence that are prerequisite to the productive modeling of student achievement data in comparative curriculum studies are provided. (Contains 13 tables, 5 figures, and 4 footnotes.) [This paper is based on research conducted as part of the Comparing Options in Secondary Mathematics: Investigating Curriculum (COSMIC) project, a research study supported by the National Science Foundation.]