Applying lessons from a critical analysis of machine learning in psychological science : two examples predicting psychological traits and states using voice data
Creators: name: Kafer Marrero, Zachariah Nelson
name: 0000-0003-3681-6434
Publisher: name: The University of Texas at Austin
Year: 2023
Subjects: Machine learning
Smartphones
Emotion
Personality
Voice samples
LIWC
Remote sensing
EMA
Ecological momentary assessment

description: In recent years, psychological science has increasingly turned to Machine Learning (ML) and Big Data to address new and longstanding questions in the field. These technological advancements have the potential to transform how researchers approach research design, analysis of complex psychological data, and interpretation of findings. However, the effective application of ML in psychological research requires adopting a new approach to data analysis that diverges from and sometimes contradicts many of the assumptions underlying traditional methods. The dissertation critically examines the current use of ML methods in psychological research. It was inspired by the failure to replicate past findings that depression can be predicted from behaviors passively sensed via smartphone using ML. In particular, the analysis presented here identifies six common pitfalls that arise from the incompatibility between the assumptions underlying traditional analytic methods and those underlying new ML methods. Using intuitive arguments, the analysis compares and contrasts ML and traditional methods within a common “function-space perspective”, which casts each approach as a method for navigating a topological surface wherein the locations on the surface are the models derived from either approach. This perspective reconciles apparent tensions between explanatory and predictive modeling, ultimately providing a basis for offering suggestions for navigating the pitfalls, and a broad foundation for understanding ML that researchers can adopt. The dissertation then demonstrates the application of ML, informed by the lessons from the critical analysis, in two studies. The first study uses a fully automatable ML design to predict psychological traits (i.e., personality scores) from linguistic and vocal characteristics extracted from naturalistic voice samples. The study finds that such predictions are possible from everyday language and that psychological traits may be expressed through a large number of vocal characteristics beyond fundamental frequency and loudness. In the second study, the approach from the first study is expanded to predict psychological states (e.g., momentary emotions) collected concurrently with the voice samples, from vocal characteristics and language obtained from the voice samples. The study finds worse overall predictive accuracy. However, a closer examination of the sample data indicates that the poor predictive performance is likely due to heterogeneity in voice samples and emotional experiences. Consequently, it is likely that better predictions will be possible with more extensive analysis than is typical for psychological investigations. Finally, the dissertation discusses and interprets the results, highlighting methodological issues, providing specific recommendations for researchers applying ML methods in psychological science, and suggesting fruitful directions for future research