by the chosen value of w, the scale of the prediction result may differ from the scale of the actual outcomes. To adjust for this simple bias, we regress the country-level result on the available country-level historical outcomes and use the fitted values as final forecasts.<sup>26</sup> We validate the forecasts by calculating Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE), reported in Table 6.<sup>27</sup> Table 6: Country-specific validation results of crisis-affected population forecasts from binarized FEWS NET outlooks and predictions from the wide Random Forest.

||**B**<br>**Near-**|**inarized **<br>**term**|**Outlooks**<br>**Medium**|**-term**|**h=**|**Random **<br>**4**|**Forest**<br>**h=**|**8**|
|---|---|---|---|---|---|---|---|---|
|**Country**|**RMSE**|**MAE**|**RMSE**|**MAE**|**RMSE**|**MAE**|**RMSE**|**MAE**|
|**Afghanistan**|7.9|3.9|15.5|9.8|5.4|3.2|5.9|3.9|
|**Burkina Faso**|1.9|0.4|2.7|0.8|0.8|0.3|1.1|0.5|
|**Chad**|5.2|2.2|13.2|9.0|6.0|4.5|6.4|4.8|
|**Congo, Dem. Rep.**|5.6|2.5|3.2|1.5|16.9|12.5|18.9|14.6|
|**Ethiopia**|3.8|2.4|16.5|12.8|8.8|7.4|9.5|7.9|
|**Guatemala**|1.1|0.4|6.3|3.5|3.9|3.0|4.0|3.3|
|**Haiti**|3.4|2.1|6.6|4.7|3.8|3.3|4.8|4.4|
|**Kenya**|5.0|2.7|4.2|2.5|6.2|5.2|7.2|6.6|
|**Malawi**|11.6|4.9|31.5|20.7|10.0|7.4|11.1|9.4|
|**Mali**|3.4|1.6|5.4|2.8|1.9|1.1|2.1|1.2|
|**Mauritania**|9.0|3.7|15.5|7.2|10.5|7.7|11.9|9.7|
|**Mozambique**|2.1|0.4|8.0|2.8|4.3|3.5|5.3|4.3|
|**Niger**|10.2|4.1|15.0|6.5|3.8|2.7|4.3|3.4|
|**Nigeria**|0.2|0.0|1.9|1.1|1.1|1.0|1.0|0.9|
|**Somalia**|18.7|8.6|26.5|16.2|21.5|18.2|22.2|18.8|
|**South Sudan**|14.4|8.6|17.4|14.5|11.8|10.7|11.9|10.8|
|**Sudan**|4.8|2.1|9.3|6.5|4.1|3.5|4.3|3.7|
|**Uganda**|0.0|0.0|1.0|0.5|0.9|0.7|1.2|1.0|
|**Yemen, Rep.**|12.4|4.8|16.1|7.1|14.5|8.5|14.3|8.4|
|**Zambia**|0.5|0.1|0.5|0.1|0.0|0.0|0.0|0.0|
|**Zimbabwe**|6.4|3.1|20.6|11.3|7.9|6.2|9.3|7.9|
|**Balanced average**|6.4|2.9|11.8|7.1|7.2|5.5|7.8|6.3|
|**Balanced average**<sup>_∗_</sup>|5.6|2.7|11.9|7.3|5.6|4.4|6.2|5.2|

The first four columns report Root Mean Squared Error and Mean Absolute Error for forecasts of country-level populations in crisis-affected districts, constructed from near-term and medium-term binarized FEWS NET outlooks. Both the binary outcomes and outlooks are netted of humanitarian assistance effects. The second set of four columns report cross-validated statistics of for the wide Random Forest, for 4- and 8-month-ahead forecast horizons.<sup>_∗_</sup> Zambia, Uganda, the Republic of Yemen, Somalia and the Democratic Republic of Congo are excluded for reasons described in the text. The validation sample spans from July 2009 - February 2019.

The results suggest that aggregated district-level predictions are reasonable predictors of country-level outcomes. At both 4- and 8-month forecast horizons, the model produces error rates that are broadly comparable to those obtained with binarized near-term outlooks.<sup>28</sup>

> 26A simple equation that involves only one scalar, e.g. of the form _Y_ = _βX_ + _ε_ . The district-level predictions used here, _P_<sup>ˆ</sup> _ijt_ , are generated setting w=<sup><u>1</u></sup> 3<sup>inthepredictionlossfunctionandareoptimizedas</sup> before to minimize loss at the district level.

> 27Specifically, we generate holdout predictions for all observations in the data set under optimized hyperparameters, and using 5 repetitions of 10-fold cross validation, as described in the previous section. It is important to note that the model hyper-parameters have been optimized to forecast crisis outbreaks, not total crisis-populations. Future efforts could test possible forecast gains when designing a prediction model specifically for the latter purpose.

> 28The balanced average of the model is heavily impacted by Somalia and the Democratic Republic of Congo. Inspection of the predictions in Somalia revealed that a considerable share of the errors was due to over-predicting in the period between 2014-2017 when outcomes briefly hit historical lows of 0%, and under-predicting when the outcomes hit 100%. Generally, we found that the modeled results were actually good leading indicators. For example, in a holdout (June and October 2019), the binarized near- and medium-term outlooks from FEWS NET had an absolute error of 13 _._ 4 and 47 _._ 0 points respectively, while the model errors were 1 _._ 2 and 2 _._ 0 points on the 4 and 8 month horizons. In Democratic Republic of Congo,

17