Measuring Engineering Students’ In-class Cognitive Engagement: Survey Development informed by Contemporary Educational Theories

Ben Lutz is a Postdoctoral Scholar in Engineering Education at Oregon State University. His research interests include innovative pedagogies in engineering design, conceptual change and development, school-to-work transitions for new engineers, and efforts for inclusion and diversity within engineering. His current work explores how students describe their own learning in engineering design and how that learning supports transfer of learning from school into professional practice as well as exploring students’ conceptions of diversity and its importance within engineering ﬁelds. Civil Engineering Engineering teaching best in engineering and the personal epistemology development students. afﬁliated electronic and computer engineering a Masters in Project management University of Sunderland, and a PhD in Educational Psychology from Washington State University. His research interests include learning and cognition, students’ engagement, and the assessment of learning and students engagements, in engineering classrooms. His expertise also include the development and validation of measurement inventories, systematic reviews and meta-analysis, and quantitative research designs. of computer-based resources; meta-analysis of of Denise R. Simmons, Ph.D., PE, LEED-AP, is an assistant professor in the Myers-Lawson School of Construction and in the Department of Civil & Environmental Engineering, and an afﬁliate faculty of the Department of Engineering Education at Virginia Tech. She holds a B.S., M.S., and Ph.D. in civil engineering and a graduate certiﬁcate in engineering education – all from Clemson University. She has over 10 years of experience working for energy companies and as a project management consultant. Her research contributes to the advancement of labor and personnel issues in engineering broadly and speciﬁcally in the construction industry through two research areas: untangling the complex relationship between activities people become involved in — operationalized as engagement — and the technical and professional outcomes gained — operationalized as competencies. The broader impact of this work lies in achieving and sustaining productive, diverse and inclusive project organizations composed of engaged, competent people. Dr. Simmons’ research is supported by awards from NSF, including a CAREER award. She oversees the Simmons Research Lab (www.denisersimmons.com), which is home to a dynamic, interdisciplinary mix of undergraduate and graduate students and a post-doctoral researcher from various colleges and de-partments at Virginia Tech who work together to explore engineering and construction human centered issues with an emphasis on understanding difference and disparity. Abstract In this research paper, we examine the development of an instrument designed to measure students’ in-class cognitive engagement. Cognitive engagement is a critical aspect of success in an engineering program; students who are more engaged are more likely to succeed. However, understanding how and in what ways engagement takes place remains a challenge. In this study, we describe the development of a survey informed by an existing theoretical framework that links overt, behavioral indicators to cognitive states of engagement. In this description, we present the results of an initial factor analysis, articulate the differences between anticipated factor structure and actual structure as a result of that analysis, and identify the improvements made from subsequent iterations of the survey. Results from the factor analyses suggest that in-class engagement should be defined and conceptualized as a combination of behavioral and cognitive scales, rather than either scale alone. However, further iterations are needed to arrive at a final composite measure of engagement at the student- and course-levels.