learned to predict the preferred average risk level but that, on average, the probabilities also trend in the right direction. The _LB_ values thus indicate that the models predict good probabilities.

Note that standard tuning parameters do not explicitly re-distribute predictions toward a high-cost outcome. The ability to tune predictions for the differential in costs instead relies on the transformation of the predicted probabilities controlled by tuning parameters ( _α, β_ ).<sup>23</sup> For example, the untransformed “Vanilla” model at _h_ = 4, tuning only over default parameters – and not ( _α, β_ ) – benefits from up-sampling but has no explicit flexibility to deal with the unbalanced prediction problem and reaches an _LB_ �w = ( 3<sup><u>1</u></sup><sup>_,_</sup><sup><u>1</u></sup> 2<sup>_,_</sup><sup><u>2</u></sup> 3<sup>)</sup> � of ( _._ 31 _, ._ 41 _, ._ 51). The associated values of _LA_ are (15 _._ 3% _,_ 21 _._ 9% _,_ 28 _._ 5%). Finally, it is also possible to tune ( _α, β_ ) to obtain an extremely conservative model. For example, when minimizing FNR under the constraint that FPR may not increase over the 1 _._ 4% set by the performance baseline, the model produces an FNR of 49% and an _LA_ of 17 _._ 3% at w=<sup><u>1</u></sup> 3<sup>,downfromthe</sup> baseline value of 26%.

Table 5: Country-specific validation results of binarized FEWS NET outlooks and predictions from the wide Random Forest.

||**Bin**|**arized **|**Outlo**|**oks**|**R**|**andom**|**Fores**|**t**|
|---|---|---|---|---|---|---|---|---|
||**Near-**|**term**|**Med.**|**-term**|**h=**|**4**|**h**|**=8**|
|**w**|**1/3**|**2/3**|**1/3**|**2/3**|**1/3**|**2/3**|**1/3**|**2/3**|
|**Afghanistan**|27%|53%|24%|46%|17%|28%|17%|27%|
|**Chad**|30%|59%|33%|63%|7%|9%|7%|8%|
|**Ethiopia**|30%|57%|29%|55%|20%|19%|19%|18%|
|**Haiti**|25%|50%|31%|59%|27%|49%|28%|49%|
|**Kenya**|25%|47%|26%|50%|16%|26%|20%|35%|
|**Malawi**|27%|52%|30%|58%|11%|14%|11%|13%|
|**Mali**|24%|47%|28%|55%|11%|16%|8%|14%|
|**Niger**|22%|41%|32%|60%|9%|14%|10%|16%|
|**Nigeria**|31%|62%|31%|60%|5%|4%|5%|4%|
|**Somalia**|25%|47%|33%|59%|30%|21%|33%|21%|
|**South Sudan**|24%|41%|28%|50%|17%|13%|17%|13%|
|**Sudan**|25%|50%|30%|60%|16%|22%|15%|21%|
|**Zimbabwe**|27%|53%|23%|44%|11%|11%|10%|10%|
|**Balanced average**|26%|51%|29%|55%|15%|19%|15%|19%|
|**Burkina Faso**|0%|0%|33%|67%|28%|45%|33%|67%|
|**Congo, Dem. Rep.**|34%|67%|33%|67%|19%|28%|17%|21%|
|**Guatemala**|30%|59%|30%|59%|23%|41%|23%|41%|
|**Mauritania**|35%|67%|35%|67%|22%|40%|23%|40%|
|**Mozambique**|29%|57%|24%|48%|21%|31%|26%|44%|
|**Uganda**|33%|67%|34%|67%|21%|37%|21%|36%|
|**Yemen, Rep.**|6%|9%|24%|16%|36%|20%|28%|15%|

The first four columns report weighted averages of false negative rates and false positive rates ( _LA_ ) for binarized FEWS NET outlooks in the near-term (3-4 months, validated against outcomes one reporting cycle later) and mediumterm (6-8 months, validated against outcomes two reporting cycles later), for the indicated value of **w** . Both the binary outcomes and outlooks are netted of humanitarian assistance effects. The next four columns report crossvalidated statistics for the wide Random Forest optimized for the indicated value of **w** , for **h** =4- and 8-month ahead forecasts of crisis outbreaks. The validation was performed at the admin2 level, and only for observations that were preceded by non-crisis levels of food insecurity. This specifically describes prediction performance for new outbreaks of food crisis before they occur. Statistics for countries with approximately fewer than 50 historical transitions into crisis are in the bottom part. Zambia is not included in the validation sample due to an absence of positive class values. The validation sample spans from July 2009 - February 2019.

23 <u>1</u> For w= 2<sup>andw=</sup><sup><u>2</u></sup> 3<sup>wefind</sup><sup>_α_=2,ourmaximumconsideredvalue,and</sup><sup>_β_closeto0,indicatingthat</sup> nearly the entire range of predicted probabilities are skewed upward. Recall that the natural frequency of the data favors a w value around 0 _._ 05 and so our predictions need a strong transformation to do well on a loss function that penalizes FNR heavily. Future work with more validation samples could consider higher values for _α_ . For now, we simply note that the results are already extremely competitive for high values of w.

15