Learning Gain and Confidence Gain as Metrics for Pedagogical Effectiveness

Introduction This paper details the most recent results of a research project aimed at investigating the concept of ‘learning gain’ as a measure of pedagogical effectiveness. In the first part of the paper we describe the principles of a teaching approach that combines self-assessment and Peer-Instruction (Mazur, 1997) to facilitate the creation of active learning environment for a large-class undergraduate module in Economics. In the second part of the paper we conceptualise our definitions of learning gain and confidence gain, and we operationalise them to develop an evidence-based evaluation of our teaching approach. In recent times, the international pedagogical debate has registered increasing interest in defining, measuring, and evaluating the impact of different pedagogical approaches on student learning. Whilst the Organisation for Economic and Social Development embraced the broader approach of ‘assessment of learning outcomes’ (OECD, 2014 and 2011), the American and British literature seems to focus more tightly on the concept of learning gain. Within this context, the debate tackles the validity of metrics based on student performance taken at distinct points in time over a student’s educational journey (McGrath et al., 2015). However, while the debate on learning gain focusses predominantly on measures of student attainment, little attention has been devoted so far to other important dimensions of learning, such as student selfassessment skills (Taras, 2015; Henderson and Harper, 2009) and student self-efficacy (Bandura, 1977 and 1997; Pajares, 1996). We argue that students’ ability to reflect on their performance and form positive beliefs on their educational experience represents an important catalyst to learning, and should also constitute a learning objective on its own right (Ritchie, 2016). For these reasons, we propose a teaching approach that aims at enhancing students’ performance, as well as students’ awareness about their own skills. Our teaching strategy relies on an active-learning approach, which combines formative quizzes, Peer-Instruction, and self-assessment components (Aricò and Watson, 2015). In practice, we devise an algorithm that alternates formative assessment questions, selfassessment questions, and Peer-instruction moments, in a sequence of learning cycles iterated over the duration of each session. Thus, using data collected in the classroom, we proceed to evaluate the impact of our teaching approach constructing two measures of learning: one related to attainment, and one related to self-efficacy and self-assessment skills. In previous research (Aricò and Watson, 2015) it was demonstrated that within our learning environment: (i) students develop good selfassessment skills, (ii) Peer-Instruction effectiveness is independent of student self-efficacy, and (iii) learning gains generated by Peer-Instruction are higher when students’ preparedness is initially poorer. Building on these results, we enhance the pedagogical design devised by Aricò and Watson (2015) to investigate: (i) the patterns of correlation between learning gain and confidence gain, (ii) the synergy between Peer-Instruction and self-assessment, and (iii) the characteristics of the distribution of learning gain and confidence gain. Our first research objective aims to uncover whether higher learning gain and are associated to better self-assessment skills, and positive selfefficacy beliefs. Our second research objective assesses whether the development of good selfassessment skills affects Peer-Instruction outcomes.