The Quality of Popular Press Reporting on Scientific Findings
Creators: name: Smiley, Adam Herschel
name: Shoda, Yuichi
name: Naik, Jared
name: Zhu, Jasmine
name: Yifan Liu
Publisher: name: OSF Registries
Year: 2022
Subjects: subject: Medicine and Health Sciences
subject: Social and Behavioral Sciences

description: Culture tends to view psychology as interesting but medicine as consequential. These two perceptions, while not mutually exclusive, may differentially influence journalistic practices around scientific reporting and create disparities in the fidelity and nuance with which news outlets convey scientific information on these topics to the general public. The capacity for even the most well-intentioned news media to sensationalize, misrepresent, or oversimplify information makes it important to characterize how the popular press talks about research findings, especially those pertaining to mental and physical health. The present study aims to build this understanding. Using a web-scraping algorithm, we collected articles reporting psychological and/or biomedical findings in popular press outlets worldwide. A team of trained raters reviewed these e-articles and systematically evaluated the extent to which each article (1) describes the findings' limitations; (2) identifies the specific samples (of people, methods, and contexts) involved in the cited study and how they potentially constrain the findings' generalizability; and (3) avoids making unsubstantiated claims without providing caveats. We expect that the popular press frequently neglects to call attention to scientific findings' constraints on generality (COGs) and other limitations, and frequently makes unsubstantiated claims without caveats when reporting on such findings. In particular, we hypothesize that these problems are more prevalent when findings are relevant to psychology and/or mental health compared to when they are relevant to biology and/or physical health. This investigation provides important information to researchers, journalists, healthcare providers, and consumers about the current state of scientific reporting, how to constructively navigate it, and how to improve it.
 
There is a growing recognition in recent years (in psychology as well as medicine) that research samples continually overrepresent Western, educated, industrialized, rich, and democratic (WEIRD) groups and underrepresent women, racial and ethnic minorities, low-SES populations, older adults, and more. Dissemination and translation of findings commonly neglect these issues, portraying the findings as absolute and broadly applicable. Improving media’s transparency surrounding research COGs is a critical step to inform underrepresented populations of whether or not such information is relevant for them.
Constraints on Generality (Simons, Shoda, &amp; Lindsay, 2017) refers to multiple ways in which a study (or a set of studies) differs from other studies or real-life situations to which the findings are proposed to be relevant. This includes differences in the populations of people who are studied, differences in the research materials used as stimuli, as well as differences in social and physical contests and historical moments. 

In light of this, we hope to determine the kinds of COGs that are most underreported. This has the potential to help identify social determinants of health that may be under-acknowledged in the popular press. Web articles are especially important to examine given their availability to populations across the globe. Everyone deserves health information that is dependable and valid for their social, cultural, economic, and environmental contexts, and the popular press holds a responsibility to communicate the extent of health findings' applicability to an audience whose differences matter.