Lessons in Character Impact Evaluation. Final Report. NCEE 2012-4004
Author: Hanson, Thomas
Date: 2012-03-00
Type: Numerical/Quantitative Data
Subjects: Control Groups,Intervention,Elementary School Students,Education,Language Arts,Academic Achievement,Reliability,Teacher Surveys,Student Surveys,Personality,Values Education,Educational Change,Outcomes of Education,Interpersonal Competence,Behavior Problems,Educational Environment,Grade 4,Grade 5,Scores,Correlation,Evaluation,Program Implementation,Sample Size,Effect Size,English Language Learners

Character education has become one of the fastest growing reform movements in K-12 education today, partially in response to unacceptable levels of student misbehavior and inadequate endorsement of good character values (Williams 2000). Between 1993 and 2009, 36 states passed laws mandating or recommending some aspect of character education in schools. Character education programs also have strong support from parents, teachers, and school administrators (Character Education Partnership 2010; Glanzer and Milzen 2006). But despite the widespread popularity of such programs, relatively few randomized controlled trials have examined the impact of these programs on the character traits, behavior, and academic outcomes of students (What Works Clearinghouse 2007). This study examines the impact of the Lessons in Character (LIC) program--an English language arts-based character education program--on student academic achievement, social competence, and problem behaviors and, secondarily, on the school environment. The program consists of literature-based supplementary material aligned with California English language arts standards and designed to integrate easily into the current English language arts curricula. The LIC curriculum is designed to be easy to implement in the classroom and to involve minimal teacher training, which distinguishes the program from other character education programs. The following confirmatory research questions guide this study: (1) Do students in grades 4 and 5 who attend schools in the LIC intervention group exhibit higher scores on measures of academic achievement after two academic years of potential LIC exposure than their counterparts who attend schools in the control group?; (2) Do students in grades 4 and 5 who attend schools in the LIC intervention group exhibit higher scores on measures of social competence after two academic years of potential LIC exposure than their counterparts who attend schools in the control group?; and (3) Do students in grades 4 and 5 who attend schools in the LIC intervention group exhibit fewer problem behaviors after two academic years of potential LIC exposure than their counterparts who attend schools in the control group? The impact analyses did not find that LIC improved student academic achievement, social competence, or problem behaviors on any of the analyzed measures. Nor were statistically significant impacts on the school environment measures of expectations and student belonging detected. Moreover, analyses of teacher reports of program implementation indicated that 30 percent of teachers reported that, in year 1, they implemented the number of core lessons recommended by the developer and that, in year 2, 23 percent reported having done so. The failure to detect program impacts on student and school environment outcomes could be due to the weak implementation of the program, an intervention design that simply does not work in bringing about changes in student and school environmental outcomes, or methodology limitations of the study. An important limitation of the study was that retention rates differed between teachers in intervention and control schools. Although participation rates in intervention and control schools did not differ and there were no statistically significant differences in the baseline characteristics of teachers or students in the intervention and control schools, it is possible that there were important unobserved differences between intervention and control samples that could have resulted in biases in estimated program impacts. Appended are: (1) Statistical power estimates; (2) Teacher and student recruitment and retention by grade level and data source; (3) Survey items used to measure confirmatory and intermediate outcomes; (4) Internal consistency reliability and intraclass correlations of confirmatory and intermediate outcomes measures; (5) School-level and teacher-level baseline characteristics for retained sample; (6) Covariates included in impact analysis models; (7) Sensitivity analyses--primary impacts on student-level outcomes; (8) Sensitivity analyses--primary impacts on teacher-level outcomes; (9) Sensitivity Analyses--exploratory impacts on student-level outcomes; (10) Sensitivity analyses--exploratory impacts on teacher-level outcomes; (11) Teacher survey; (12) Student survey; and (13) Teacher implementation log. (Contains 65 tables, 6 figures, 2 boxes and 27 footnotes.)