Predicting the Past: Imputation of Historical Data
Creators: name: Spicer, Rachel
Publisher: name: Open Science Framework
Year: 2022
Subjects: subject: History of Religion
subject: Statistical Methodology
subject: History
subject: Physical Sciences and Mathematics
subject: Arts and Humanities
subject: Statistics and Probability
subject: Religion
subject: Cross-cultural Data
subject: Historical Data
subject: Imputation

description: There are many gaps in the historical record, for which we have no evidence. These can arise because no records were originally kept or because evidence is accidentally or deliberately destroyed. Gaps such as these pose a challenge for anyone wanting to systematically analyze historical data, as they can only be filled if new evidence is uncovered, which may never occur. These missing values introduce challenges to many common statistical and machine learning techniques. The gaps in evidence however, provide an opportunity to test methods of imputation on historical and other cultural data. A large body of work suggests that culture clusters (Harton &amp; Bullock, 2007; Muthukrishna et al., 2020) - certain features tend to co-occur either through a package of adaptations to the ecology (Barsbai et al., 2021) or through social learning processes such as the prestige bias (Henrich et al., 2015) or conformist bias (Harton &amp; Bullock, 2007; Henrich &amp; Boyd, 1998). Moreover, cultures have path dependence, persisting over time (Muthukrishna et al., 2021). Thus one potential solution to the problem of missing data is to use the existing past and present data to predict gaps in the past (Blanke, 2018; Jeffares, 2008). In recent years, machine learning techniques, which were originally developed for prediction, have also been effective for imputing missing data (Bießmann et al., 2019; Miranda &amp; Freeman, 2020). Rubin (1976) proposed that there are three categories of missing data: missing completely at random (MCAR), missing at random (MAR) and missing not at random (MNAR). If all data has the same likelihood of being missing, the data are MCAR. For data to be MCAR, the probability of a data point being missing must be independent from all other values in the dataset, both observed or unobserved (missing). If the probability of data being missing is related to observed values, but not unobserved missing values, it is said to be MAR. If data is neither MCAR or MAR, then it is MNAR⸺the probability of data being missing is systematically related to the unobserved values and the reason for the data being missing is unknown. In cross-cultural research, most commonly only complete cases of data are used for analysis (Dow &amp; Eff, 2009a). Any instances that contain missing values are removed using listwise deletion. This can be problematic as cross-cultural data is often not MCAR, leading to biased estimates and lower statistical power (Dow &amp; Eff, 2009a). Dow and Eff (2009) proposed multiple imputation as the most practical approach for handling missing data in cross-cultural studies, whilst limiting bias. Subsequently, this approach has been applied in several studies (Thompson et al., 2020; Minocher et al., 2019; Turchin et al., 2018), but listwise deletion remains the most common approach for handling missing data. More recently, Miranda and Freeman (2020) applied datawig (Bießmann et al., 2019), a deep learning approach, to more accurately impute cross-cultural data. However, to the best of our knowledge, no studies have compared the accuracy of more than 2 different imputation methods for missing cross-cultural data. In this research, we aim to address this gap in the literature and to compare the accuracy of different imputation methods for predicting missing historical data, using data from the Database of Religious History (DRH, https://religiondatabase.org; Slingerland and Sullivan, 2017; Sullivan et al., 2017). To do this, we deliberately remove known data to explore the degree to which we can predict this missing data based on other variables. Although culture clusters, this clustering may be specific to a particular time, place, and society. This research will also uncover the degree to which there are broad patterns of cultural clustering that can be exploited to predict missing data from the past.