# **Appendix B: Performance of Classification Algorithms in the Hegre and Sambanis (2006) Dataset**

In this annex we briefly describe how the four classifications algorithms described in the main text perform in the conflict dataset in Hegre and Sambanis (2006), (HS). In their seminal contribution, HS were the first to systematically document the robustness of the relationship between civil conflict and many of the correlates of conflict that had been proposed in the previous literature. HS considered a set of 88 such variables, and used a variant of extreme bounds analysis to isolate a subset of 18 variables that were robustly associated with conflict, in the sense of having a weighted average p-value smaller than 0.05. We take these 18 variables, listed in Table 3 of HS, together with two of the three “core” variables included in all of their specifications (logarithm of population and the logarithm of GDP per capita, Table 2 in HS) as our set of candidate explanatory variables.<sup>24</sup> We use the HS primary definition of civil war as our measure of conflict. As in the original HS contribution, conflict episodes begin in the first year that conflict is observed. HS treat all country-years in which conflict is not observed as non-conflict observations, and then logit regressions to estimate the relationship between conflict and contemporaneous explanatory variables in a country-year panel. Here we use the HS conflict data to create a set of conflict episodes and complementary non-conflict episodes in the same way as in the main text. Conflict episodes begin in a year where conflict is observed, and no conflict is observed in the three previous years. Non-conflict episodes begin in the first of nonoverlapping five-year periods in which conflict is not observed, and which are preceded by three years of no conflict as well. As in the main text, and consistent with our emphasis on predicting future conflict based on contemporaneously-available data, we measure the explanatory variables as annual averages in the three years prior to the start of the episode. Transformed in this way, the dataset consists of XX conflict episodes and YY non-conflict episodes.

Annex Table 1 summarizes the predictive performance of the four classification algorithms in the HS data. Model 1 in the left panel considers a model with the top 10 most significant variables from HS, while Model 2 in the right panel considers a model with all 20 variables. The first block of the table considers the in-sample fit of the two models, and is analogous to Table 4 in the main text. Here the threshold classifier performs best in Model 1 with 10 explanatory variables, but the random forest classifier performs best in Model 2. The second and third panels of the table divide the sample at 1980 and at 1990 respectively, and report the in-sample predictive power for the four classifiers estimated in the first half of the sample, followed by the out-of-sample predictive power in the second half of the sample. In Model 2 with 20 explanatory variables, the threshold classifier has the best out-of-sample predictive performance for both sample splits, although extremely close to that of the linear classifier in the sample split at 1980. In the more parsimonious Model 1, the linear classifier edges out the threshold classifier by a tiny margin when the sample is split at 1980, and both perform only marginally better than the random forest classifier. Somewhat surprisingly, the probit classifier has the best out-ofsample performance when the sample is split at 1990. Despite this aberration, overall the results that emerge from Annex Table 1 are broadly similar to those in the main text: the threshold classifier performs relatively well as an out-of-sample prediction rule, and there is little evidence of a penalty in

> 24 We do not include the third core variable (number of years at peace) since it was not consistently significant in the HS specifications.  We obtained the dataset from the replication materials generously provided by Muchlinsky et. al. (2016) who also work with this dataset.

34