Elementary Mathematics Student Assessment: Measuring the Performance of Grade 1 and 2 Students in Counting, Word Problems, and Computation in Fall 2014. Research Report No. 2016-04
Author: Schoen, Robert C.
Date: 2016-12-00
Type: Reports - Research
Subjects: Elementary School Mathematics,Grade 1,Grade 2,Mathematics Tests,Mathematics Achievement,Mathematics Skills,Computation,Word Problems (Mathematics),Psychometrics,Test Construction,Test Validity,Test Reliability,Test Items,Correlation,Common Core State Standards,Statistical Analysis,Goodness of Fit,Factor Analysis,Regression (Statistics)

The subject of this report is a pair of written, group-administered tests designed to measure the performance of grade 1 and grade 2 students at the beginning of the school year in the domain of number and operations. These tests build on previous versions field-tested in fall 2013 (Schoen, LaVenia, Bauduin, &amp; Farina, 2016). Because the tests are designed to be a measure of student achievement in elementary mathematics, we call them the Elementary Mathematics Student Assessment (EMSA) tests. The EMSA tests were designed to serve as a covariate for students' baseline performance in statistical models estimating the impact of a teacher professional-development program on student achievement in mathematics. This report is written for researchers and evaluators who may be interested in using the tests in the future or who wish to know about the psychometric properties of the tests. The 2014 EMSA tests were administered with 3,080 participating grade 1 and grade 2 students in 22 schools located in two public school districts in Florida during fall 2014. The tests were administered by classroom teachers in a whole-class setting using a paper-pencil format. The school districts were implementing a curriculum based on the Mathematics Florida Standards, which are very similar to the Common Core State Standards for Mathematics. To generate overall test scores, we first regressed three first-order factors (i.e., Counting, Word Problems, Computation) onto a single second-order factor (i.e., Math). The second-order Math factor score is intended to serve as the overall achievement score on the test. Model fit statistics for the grade 1, second-order model are as follows: ?[superscript 2](149) = 1715.379, p &lt; 0.001; RMSEA = 0.081, 90% Confidence Interval (CI) [0.078, 0.085]; CFI = 0.916; and TLI = 0.904. Grade 2 model fit statistics are: ?[superscript 2](167) = 532.780, p &lt; 0.001; RMSEA = 0.038, 90% CI [0.035, 0.042]; CFI = 0.968; and TLI = 0.964. A composite reliability estimate for the Math factor based on ordinal forms of Cronbach's a was used to estimate reliability for the three subscales, resuting in a grade 1 math composite reliability estimate of 0.88 and a grade 2 math composite reliability estimate of 0.91. Regression results suggested that the fall 2014 EMSA Math score was a moderate to strong predictor of students' scores on the ITBS Math Problems test administered in spring 2015, where an R[superscript 2][subscript Adjusted] of 0.45 was found for grade 1 and an R[superscript 2][subscript Adjusted] of 0.55 was found for grade 2. The fall 2014 EMSA Math scores provided more modest predictive power with the spring 2015 ITBS Math Computation test, where an R[superscript 2][subscript Adjusted] of 0.28 was found for grade 1 and an R[superscript 2][subscript Adjusted] of 0.36 was found for grade 2. These relations were statistically significant at p &lt; 0.001. The two test forms were not vertically equated, but vertical equating is a natural next step for test development. To maintain test security, the details of the mathematics problems posed to students are not provided in this report. Interested parties may contact the first author for more information.