Importantly, the error of model increases only slightly as we move to the 8-month forecast horizon, while errors made with binarized outlooks nearly double when moving from the near-term to the medium-term horizon. This means that the model-based predictions are better suited for forecasting further into the future, which may help increase lead time for preventive action.

# **4.3 Model interpretability**

The previous sections showed that the Random Forest model provides good predictive capabilities. While predictive performance is important, the modeling structure of the transformed classifier may be difficult to interpret. Out-of-the-box diagnostics may be used to help determine which variables are important in the forest, but do not inform about the signs and magnitude of (local) relationships. Parametric models are more transparent in that regard, but while it can be argued that these models are more interpretable, they might not be comprehensible given the number of predictors involved. Moreover, standard methods do not account for effects induced by the transformation of probabilities. To overcome these issues, this section develops a prediction decomposition strategy that relies on assessing the change in prediction values when the model is confronted with observed values compared to reference input data. This provides comprehensible interpretation at an observational level and takes the following algorithmic strategy.

1. Partition the spatio-temporally covariates into a fully exhaustive set of non-overlapping groups. We use the groups defined in section 2.2 but split rainfall variables from the other environmental factors.<sup>29</sup>

2. Construct a country data set by leaving all time-invariant district-level data at local values and setting all spatio-temporally varying covariates at a reference value. We use country means.<sup>30</sup> Make reference predictions with this data set.

3. For each group of covariates defined in step 1, set the levels of the data to the observed values while keeping all the other covariates at reference values as defined in the previous step, then generate a new set of predictions.

4. For each set of predictions generated in the previous step, subtract the reference predictions generated in step 2. Then, from this prediction difference, calculate the lowest value in each district across all time periods and subtract these.

5. Standardize the prediction differences generated in the previous step to [0,1] by dividing them by their sum, then multiply by probabilities predicted when all data are set to observed levels.

> only 8 assessment cycles are available for cross-validation and errors are heavily driven by the first two periods. Again, good performance was established in a holdout, binarized outlooks scoring respectively 9 _._ 3 and 16 _._ 3 points while the model-based predictions missed the outcome by only 1 _._ 6 and 3 _._ 2 points. Finally, few assessments are available for the Republic of Yemen which has stayed near 100% crisis in all but the first period, no difference was ever recorded between near-term preliminary assessments and final outcomes in Uganda, and Zambia never experienced crisis. When focusing on the remaining 16 countries for validation, RMSE in the near-term was identical for model-based predictions and binarized outlooks, while on the 8-month target the model reduced RMSE by 48% increasing only 0 _._ 6 points compared to the binarized near-term outlook.

> 29“Markets” includes all features calculated from food prices and “Conflict” all features derived from the conflict data. The Environmental Factors are divided in two groups: “Rainfall” and “Agricultural Stress”, the latter combining all the features derived from NDVI and evapo-transpiration.

> 30It would also be possible to construct specific alternatives with which to compare observed values such as conflict counts of 0 rather than a non-zero average.

18