Project Based Learning: Evaluation Report and Executive Summary
Author: Menzies, Victoria
Date: 2016-11-00
Type: Reports - Research
Subjects: Active Learning,Student Projects,Questionnaires,Attitude Measures,Intervention,Correlation,Pretests Posttests,Experimental Groups,Comparative Analysis,Randomized Controlled Trials,Foreign Countries,Case Studies,Grade 6,Control Groups,Secondary School Students,Learner Engagement,Eligibility,Teaching Assistants,Pilot Projects,Literacy,Lunch Programs,Factor Analysis,Statistical Analysis,Observation,Focus Groups,Interviews,Teacher Surveys

Project Based Learning (PBL) is a pedagogical approach that seeks to provide Year 7 pupils with independent and group learning skills to meet both the needs of the Year 7 curriculum as well as support their learning in future stages of their education. It particularly aims to improve their engagement in learning as well as practical literacy skills. This trial evaluates a specific type of PBL known as &quot;Learning through REAL Projects&quot;, developed by the Innovation Unit--an independent social enterprise that aims to improve public sector services. A year long pilot project in 2013/2014 with eight schools (763 pupils) established the feasibility of a main trial. It provided the opportunity to develop the intervention and test the research procedures. The results contained in this report relate to a randomised controlled trial that took place between September 2014 and April 2016. Twelve intervention schools (2,101 pupils) and 12 control schools (1,973 pupils) were involved in the trial. REAL projects was delivered by teachers, supported in many instances by teaching assistants (TAs), with further support from senior leadership colleagues. It was delivered for a year in the intervention schools with a relatively large proportion of timetabled teaching (varying between 20% and 50%). In almost all cases it was delivered to mixed-ability Year 7 classes. The intervention used structured cross-subject &quot;REAL Projects&quot; planned by the delivery teachers who were supported by Classroom Coaches from the Innovation Unit. Schools were also supported by Leadership Coaches. REAL Projects are driven by an &quot;essential question&quot; which has significant educational content. The projects encouraged pupils to create an &quot;excellent&quot; product through drafting and redrafting and then to exhibit their work to an &quot;authentic&quot; audience. Visits were made to intervention schools at two time points and consisted of lesson observations, interviews, and focus groups with school leadership, project leads, class teachers, and pupils. Two case studies were also conducted in schools that it was agreed delivered the intervention well. Key conclusions are as follows: (1) Adopting PBL had no clear impact on either literacy (as measured by the Progress in English assessment) or student engagement with school and learning; (2) The impact evaluation indicated that PBL may have had a negative impact on the literacy attainment of pupils entitled to free school meals. However, as no negative impact was found for low-attaining pupils, considerable caution should be applied to this finding; (3) The amount of data lost from the project (schools dropping out and lost to follow-up) particularly from the intervention schools, as well as the adoption of PBL or similar approaches by a number of control group schools, further limits the strength of any impact finding; (4) From our observations and feedback from schools, we found that PBL was considered to be worthwhile and may enhance pupils' skills including oracy, communication, teamwork, and self-directed study skills; and (5) PBL was generally delivered with fidelity but requires substantial management support and organisational change. The Innovation Unit training and support programme for teachers and school leadership was found to be effective in supporting this intervention. Overall, the findings have low security. The trial was designed as a two-armed randomised controlled trial with schools being allocated to intervention or control groups. It was set up as an efficacy trial which aimed to test if the intervention can succeed under ideal conditions. However, 47% of the pupils in the intervention and 16% in the control group were not included in the final analysis. Therefore there were some potentially important differences in characteristics between the intervention and control groups. This undermines the security of the result. The trial results did not find that the PBL programme had an impact either on the pupils' literacy performance (as measured by Progress in English 12 tests), engagement, or attendance. The analysis did find a statistically significant negative impact on students eligible for free school meals (FSM), however no negative impact was found for lower-attaining pupils more generally, which makes it difficult to hypothesise why PBL might negatively impact FSM pupils specifically. This adds to the uncertainty of the finding.