Exploring self-reported survey data in higher education as an artifact of socio-environmentally influenced behavior

Much of the research about college student engagement is based upon self-reported surveys, but little is known about how students formulate responses to these instruments. The purpose of this study was to specifically address this dearth of knowledge by deepening our understanding of how students’ perceptions of their environments and demographic characteristics influenced their response patterns on self-reported surveys. Bronfenbrenner’s (2005) human ecology model of development, Bourdieu and Passeron’s (1990) theory of social reproduction, and Tourangeau, Rips, and Rasinski’s (2000) four phase survey response process were used, as the theoretical framework to better understand this phenomenon. This was an explanatory sequential mixed methods study, and the participants were first-year undergraduate students at a four-year, private institution in New England. Students completed the College Student Report (CSR) as well as a series of time-use diaries, and the results of the instruments were compared using descriptive and multivariate analyses. Finally, semi-structured individual interviews were conducted, which included aspects of retrospective cognitive interviewing, with twenty-seven (27) students to understand how their experiences and response processes were shaped by their individual campus experiences and identities. Findings from this study suggest the construct validity of selfreported survey data measuring behavioral frequency patterns is questionable, as students statistically significantly under reported time spent preparing for class, engaging in co-curricular activities, commuting to campus, and relaxing and socializing. Furthermore, student characteristics such as racial/ethnic identity and satisfaction with college choice statistically significantly explained some of the variance in the reporting behaviors of students after controlling for other factors. This information coupled with the data gleaned from the semistructured individual interviews indicate factors related to how students differentially experience EXPLORING SELF-REPORTED DATA iv the campus environment based upon their unique ecological niches affects how they respond on self-reported surveys, which means the data provided by such instrumentation is likely providing substantively different information than how it is most commonly interpreted and applied. EXPLORING SELF-REPORTED DATA v ACKNOWLEDGEMENTS This dissertation is the culmination of the dedication and hard work of many, many people. First, I want to thank Dr. Heather Rowan-Kenyon who has supported and challenged me throughout my entire doctoral journey, especially as the chair of my dissertation committee. She pushed me to think deeper and invest more of myself into this project when I was being complacent; she pulled me back from the brink of impending madness when I spent a month pursuing the wrong path in data analyses; and she always provided me with pathways and opportunities to explore my scholarship. I also want to thank Dr. Ana Martinez-Aleman and Dr. Lauren Saenz for their guidance, feedback, and insight throughout my dissertation process, which helped to shape, define, and improve not only the outcome of this document, but also my own thinking. I want to thank my cohort, Dr. Scott Radimer and Dr. Danny Zepp. Your friendship, conversation, and laughter throughout graduate school shaped my experiences and challenged me to be a better scholar. Thank you also to my other fellow graduate students who shared many a drink and a laugh with me throughout my time in school as well as the many friends I made during my time in Boston. Thank you all for helping me maintain a semblance of balance between the personal and the professional. Thank you to all of the people in my professional and personal life who have shaped my beliefs, thoughts, and actions. From my colleagues at Western Kentucky University to Seattle University to the Office of the Dean of Students at Boston College to Virginia Commonwealth University, thank you all for your encouragement, friendship, and mentorship. I want to especially thank all of the students, faculty, and staff of Spectrum Weekend. Your courage, generosity of spirit, and faith in the inherent goodness of humanity fed my spirit for three years. EXPLORING SELF-REPORTED DATA vi Thank you to my family who has always supported me and provided me with examples of courage, diligence, and humility. I am extremely privileged to have a family who loved me, cheered me, and supported me my entire life, even when I made choices they would not necessarily have made for themselves. I am the person I am today because of you. Finally, I want to thank and acknowledge my wife and partner, Megan. You have made my life more than I ever would have dared dreamed it could be. You have given me the courage to move to new places, the ambition to strive to be the best version of myself, the humility to honestly reflect upon my own failings and weaknesses, and the love and encouragement to boldly take risks. You are the partner of my life. Without you, this dissertation would not be possible, but in truth, without you, my life, as I have lived it, would not be possible. So, thank you, and I love you! EXPLORING SELF-REPORTED DATA vii TABLE OF CONTENTS ABSTRACT . iii ACKNOWLEDGEMENTS . v LIST OF TABLES . x FIGURES . xii CHAPTER 1: INTRODUCTION .