Effectiveness of an evidence-based predictive model for motivating success in freshmen engineering students (2012)

Background: Although several testing mechanisms are available to lecturers to identify students not meeting entry level skills to engage them in bridging courses prior to enrolment in the intended course, implementing these tests is often not an option. Typically, lecturers are obliged to accept all students and subsequently find innovative ways to increase retention/success. We believe that engineering students will be more inclined to improve their behaviour if they are provided with quantitative evidence that their behavioural change can lead to success and improved course grades. Consequently, a year ago an evidence-based tool (MECS: Motivation, Encouragement, Completion, Success) to predict course outcome was developed based on its quantified relationships with student attendance, early engagement, performance in interim assessments using a data mining technique applied to information on a cohort of first year students enrolled in Fluid Mechanics in 2010. The results showed that good attendance in lectures and higher performance in interim summative assessments led to higher final examination results (Fernando and Mellalieu, 2011). The tool also helped lecturer detect at-risk students who need significant intervention by the lecturer, and/or change in behaviour by the student to succeed. The strength of these correlations between the final course outcome and other attributes was so strong that it signalled promising results from an extended study. Purpose: The purpose of our on-going efforts was to find how best to use the MECS- tool to motivate and engage engineering students to achieve better course outcomes. The analysis sought answers to the question of what intervention mechanisms most effectively improved the course completion rates. Design/Method: The validity of the MECS-tool developed for the 2010 cohort of students was tested with data for the 2011 batch. Both groups did not use the tool and therefore did not benefit from its predictive capabilities enabling change of behaviour; as such the validation results help confirm if the tool is applicable in general. In 2012, the tool was given to the students enabling them to be in control of the desired outcomes. Intervention measures were also implemented. Comparison was then made between 2010 and 2011 performances with those in 2012. Feedback was gathered from the 2012 students on the usefulness of the tool. Results: Success in 2012 was superior to those in 2010 and 2011 implying the effectiveness of the MECS-tool in initiating behavioural changes and intervention techniques. Adoption of the tool for students was not widespread but an overwhelming majority recommended it be made available to future cohorts. Conclusions: The tool is very effective in identifying at-risk students. It helps students adapt to achieve desired course outcomes. A combination of methods varying from encouragement to use the tool, intervention mechanisms such as persistent follow ups, catch-up lessons, and extra tutorials offered to the detected at-risk students were the effective ways that enhanced success and retention in 2012.