Effects of Problem Based Economics on High School Economics Instruction. Final Report. NCEE 2010-4022rev
Author: Finkelstein, Neal
Date: 2011-03-00
Type: Numerical/Quantitative Data
Subjects: Test Items,Intervention,Economics Education,Problem Based Learning,Teacher Surveys,Problem Solving,Secondary School Teachers,Grade 12,Instructional Effectiveness,Curriculum Implementation,Teaching Methods,Secondary School Curriculum,Satisfaction,High School Students,Academic Achievement,Knowledge Level,Outcomes of Education,Teacher Characteristics,Pretests Posttests,Faculty Development,Statistical Significance,Control Groups,Experimental Groups,Teaching Styles,Educational Strategies,Measures (Individuals)

This study examines whether the Problem Based Economics curriculum developed by the Buck Institute for Education improves grade 12 students' content knowledge as measured by the Test of Economic Literacy, a test refined by the National Council on Economic Education (NCEE) over decades. Students' problem-solving skills in economics were also examined using a performance task assessment. In addition to the primary focus on student achievement outcomes, the study examined changes in teachers' content knowledge in economics and their pedagogical practices, as well as their satisfaction with the curriculum. The study was designed as an experimental trial. It was implemented from summer 2007 through spring 2008 in high schools in Arizona and California. The analysis at the primary (student) level supports the following: (1) A statistically significant finding that students whose teachers had received professional development and support in Problem Based Economics (model-adjusted mean score = 22.61) outscored their control group peers (model-adjusted mean score = 20.01) on the TEL by an average of 2.6 test items (effect size = 0.32); and (2) The outcomes on student measures of problem-solving skills and application to real-world economic dilemmas also showed significant differences in favor of the intervention group (model-adjusted mean score for the intervention group was 6.72 versus 6.18 for the control group; the difference of 0.54 corresponded to an effect size of 0.27). The study also confirmed the following at the secondary (teacher) level: (1) No statistically significant difference between the intervention and control groups on teachers' knowledge of economics (model-adjusted means were 37.15 and 36.86 for the intervention and control group teachers, respectively). As discussed in the conclusions of the report, a ceiling effect on the Test of Economic Literacy instrument may have masked any true content gains for teachers; (2) No statistically significant difference in teachers' pedagogical style with the survey measures used (model-adjusted means were 29.92 and 26.60 for the intervention and control group teachers, respectively); and (3) Statistically significant differences in favor of the intervention group teachers on a measure of satisfaction with the teaching materials and methods (model-adjusted means were 8.35 and 6.88 for the intervention and control group teachers, respectively; the difference of 1.47 corresponded to an effect size of 1.09). Future study of this curriculum might emphasize the classroom observation component to get a clearer understanding of teachers' pedagogical strategies in varying classroom settings. From observations in intervention and control classrooms, it did not appear to the research team that having and using the problem-based learning curriculum automatically enforced a more hands-on, exploratory classroom learning style. Additional study in this area might help to refine the pedagogical strategies and allow for additional support and practice for teachers on implementing the curriculum effectively. Appended are: (1) Study power estimates based on the final analytic samples; (2) Procedure for assigning new strata to the final analytic sample; (3) Scoring procedures for the performance task assessments; (4) Sample test/survey administration guide; (5) Teacher-level baseline equivalence tests; (6) Additional student-level baseline equivalence tests; (7) Estimation methods; (8) Summary statistics of teacher data from teacher surveys; (9) Sensitivity of impact estimates to alternative model specifications; and (10) Explanations for sample attrition. (Contains 7 figures, 35 tables, 2 boxes and 17 footnotes.) [For the What Works Clearinghouse (WWC) review of this report, &quot;WWC Review of the Report 'Effects of Problem Based Economics on High School Economics Instruction,'&quot; see ED533124. This report is an update of &quot;Effects of Problem Based Economics on High School Economics Instruction. Final Report. NCEE 2010-4002&quot; (ED511228).]