Insights into accuracy of social scientists' forecasts of societal change
Authors: Grossmann Igor , Rotella Amanda , Hutcherson Cendri , Sharpinskyi Constantine , Varnum Michael E. W. , Achter Sebastian , Dhami Mandeep K , Guo Xinqi , Kara-Yakoubian Mane , Mandel David R. , Raes Louis , Tay Louis , Vie Aymeric , Wagner Lisa , Adamkovic Matus , Arami Arash , Arriaga Patricia , Baník Gabriel , Bartoš František , Baskin Ernest , Bergmeir Christoph , Bialek Michal , Børsting Caroline Kjær , Browne Dillon , Caruso Eugene Matthew , Chen Rong , Chie Bin-Tzong , Chopik William J. , Collins Robert N. , Cong Chin Wen , Conway Lucian Gideon , Davis Matthew E , Day Martin V. , Dhaliwal Nathan , Durham Justin , Dziekan Martyna , Elbaek Christian T. , Shuman Eric , Fabrykant Marharyta , FIRAT Mustafa Altuğ , Fong Geoffrey T. , Frimer Jeremy Abraham , Gallegos Jonathan , Goldberg Simon , Gollwitzer Anton , Goyal Julia , Graf-Vlachy Lorenz , Gronlund Scott D. , Hafenbrädl Sebastian , Hartanto Andree , Hirshberg Matthew James , Hornsey Matthew , Howe Piers , Izadi Anoosha , Jaeger Bastian , Kačmár Pavol , Kim Dean Y.J. , Krenzler Ruslan , Lannin Daniel G , Lin Hung-Wen , Lou Nigel Mantou , Lua Verity Y. Q. , Lukaszewski Aaron , Ly Albert Le , Madan Christopher R , Maier Maximilian , Majeed Nadyanna M. , March David Scott , Marsh Abigail , Misiak Michał , Myrseth Kristian Ove R. , Napan Jaime , Nicholas Jonathan , Nikolopoulos Konstantinos , O Jiaqing , Otterbring Tobias , Paruzel-Czachura Mariola , Pauer Shiva , Protzko John , Raffaelli Quentin , Ropovik Ivan , Ross Robert M , Roth Yefim , Røysamb Espen , Schnabel Landon , Schütz Astrid , Seifert Matthias , Sevincer A. Timur , Sherman Garrick , Simonsson Otto , Sung Ming-Chien , Tai Chung-Ching , Talhelm Thomas , Teachman Bethany , Tetlock Phil , Thomakos Dimitrios , Tse Dwight , Twardus Oliver , Tybur Joshua M. , Ungar Lyle H , Vandermeulen Daan , Williams Leighton Vaughan , Vosgerichian Hrag A. , Wang Qi , Wang Ke , Whiting Mark E , Wollbrant Conny Ernst-Peter , Yang Tao , Yogeeswaran Kumar , Yoon Sangsuk , Alves Ventura R , Andrews-Hanna Jessica R. , Bloom Paul Alexander , Boyles Anthony , Loo Charis , Choi Mingyeong , Darling-Hammond Sean , Ferguson Z E , kaiser cheryl , Karg Simon Tobias , Ortega Alberto López , Mahoney Lori , Marsh Melvin , Martinie Marcellin F. R. C. , Michaels Eli , Millroth Philip , Naqvi Jeanean , Ng Weiting , Rutledge Robb , Slattery Peter , Smiley Adam Herschel , Strijbis Oliver , Sznycer Daniel , Tsukayama Eli , van Loon Austin , Voelkel Jan G. , Wienk Margaux Nicole Agnes , Wilkening Tom 
Published: September 2022

How well can social scientists predict societal change, and what processes underlie their predictions? To answer these questions, we ran two forecasting tournaments testing accuracy of predictions of societal change in domains commonly studied in the social sciences: ideological preferences, political polarization, life satisfaction, sentiment on social media, and gender-career and racial bias. Following provision of historical trend data on the domain, social scientists submitted pre-registered monthly forecasts for a year (Tournament 1; N=86 teams/359 forecasts), with an opportunity to update forecasts based on new data six months later (Tournament 2; N=120 teams/546 forecasts). Benchmarking forecasting accuracy revealed that social scientists’ forecasts were on average no more accurate than simple statistical models (historical means, random walk, or linear regressions) or the aggregate forecasts of a sample from the general public (N=802). However, scientists were more accurate if they had scientific expertise in a prediction domain, were interdisciplinary, used simpler models, and based predictions on prior data.