21st Century Community Learning Centers 2014: A Quasi-Experimental Investigation of Program Impacts on Student Achievement in Mathematics and Reading/Language Arts
Author: White, Larry
Date: 2014-04-00
Type: Reports - Research
Subjects: Quasiexperimental Design,Investigations,Mathematics Achievement,Language Arts,Reading Achievement,Student Participation,Cohort Analysis,Experimental Groups,Control Groups,Achievement Gains,Predictor Variables,Scores,Statistical Significance,Performance Based Assessment,Group Membership,Time Perspective,Participant Characteristics,Online Surveys,Teacher Surveys,Student Records,Program Effectiveness,Elementary Secondary Education

This report summarizes an evaluation study investigating the effects of participation in the 21st Century Community Learning Centers (CCLC) program on student achievement in mathematics and reading/language arts, for the cohort of students who participated during the 2013-2014 school year. The report is a supplement to the Office of Assessment, Accountability, and Research's annual evaluation of the CCLC program. The authors conducted a quasi-experimental examination of within- and between-group differences in student assessment outcomes in both mathematics and reading/language arts. The study addressed 10 research questions (RQs). The treatment group consisted of students who participated in at least 30 days of CCLC during the 2013-2014 school year. A control group consisting of demographically similar students who did not participate in CCLC was selected using propensity score matching (PSM). RQ1 and RQ2 sought to determine if CCLC and non-CCLC students' year-to-year gains in mathematics and reading/language arts were significantly different. These questions were addressed using a series of independent samples &quot;t&quot; tests. The analyses used group membership as the independent variable and mean WESTEST 2 scale score gains from 2012-2013 to 2013-2014 in mathematics and reading/language arts as the outcome variables. Results showed no significant differences between these groups in either mathematics or reading/language arts scale score gains. RQ3 and RQ4 sought to determine if CCLC and non-CCLC students' 2013-2014 end-of-year scores in mathematics and reading/language arts were significantly different. These questions were addressed using a series of independent samples &quot;t&quot; tests. The analyses used group membership as the independent variable and mean WESTEST 2 scale score from the 2013-2014 school year in mathematics and reading/language arts as the outcome variables. Results showed no significant differences between these groups in either mathematics or reading/language arts scale scores. RQ5 and RQ6 sought to determine if CCLC students experienced statistically significant changes in performance from 2012-2013 to 2013-2014. These questions were addressed using a series of paired &quot;t&quot; tests. The analyses used time as the independent variable and CCLC students' mean 2012-2013 and 2013-2014 WESTEST 2 scale scores in mathematics and reading/language arts as the outcome variables. CCLC students exhibited statistically significant mathematics gains in Grades 4, 5, 6, 7, 9, and 11; the students also exhibited statistically significant gains in reading/language arts in Grades 4, 5, 6, 7, 8, and 10. RQ7 and RQ8 sought to determine if non-CCLC students experienced statistically significant changes in performance from 2012-2013 to 2013-2014. These questions were addressed using a series of paired &quot;t&quot; tests. The analyses used time as the independent variable and non-CCLC students' mean 2012-2013 and 2013-2014 WESTEST 2 scale scores in mathematics and reading/language arts as the outcome variables. Similar to CCLC students, non-CCLC students exhibited statistically significant mathematics gains in several grades (i.e., Grades 4, 5, 6, 7, 8, 9, and 11), and reading/language arts gains in Grades 4, 5, 6, and 7. RQ9 and RQ10 sought to determine if student achievement over time differed significantly between groups. These questions were addressed using repeated measures analysis of variance (RM ANOVA) tests. The analyses used two predictor variables, group membership and time, as independent variables predicting the outcome of WESTEST 2 performance in mathematics and reading/language arts. The authors looked for a significant interaction effect to indicate one group scored differently from the other over time. They found significant main effects for time, but there were no statistically significant interaction effects between groups and time. However, in reading/language arts, the interaction effects approached significance in Grades 8 and 10. In both cases, CCLC students outperformed the non-CCLC comparison group. There were no statistically significant differences between groups. When examining within-group differences, both groups exhibited multiple statistically significant changes in mathematics and reading/language arts performance. However, only in the case of Grades 8 and 10 reading/language arts did the results approach statistical significance in the predicted direction (i.e., with CCLC students outperforming non-CCLC students). The following are appended: (1) Detailed statistical information for all hypotheses tested; and (2) Graphical representations of the differences observed in student achievement over time by group. [For the evaluation study in &quot;21st Century Community Learning Centers: A Descriptive Evaluation for 2013-2014,&quot; see ED569701.]