in advance opens important opportunities to mitigate and avoid the worst. In this paper, we have explored the benefits of using formal statistical models to forecast food crises, using sub-national data provided by FEWS NET for 21 countries since 2009.

As a baseline, it is instructive to compare model-based forecasts with existing early warning systems. In our validation framework, these historical outlooks turn out as quite conservative, with extremely low false positive rates and relatively few warnings when compared with the actual incidence of food crisis outbreaks. As a consequence, existing outlooks have high false negative rates, i.e. many food crisis outbreaks were not anticipated. In addition, the overall forecast performance of existing outlooks deteriorates with longer forecast horizons.

Using standard cross- and temporal holdout validation methods, we have shown that relatively simple statistical models provide good predictive performance up to 12 months in advance. We used verifiable monthly data capturing environmental factors, conflict, and food price shocks as key predictors for this. We find that the model-based forecasts can deliver much lower false negative rates at the cost of only modest increases in false positive rates. Moreover, our metrics of forecast performance are fairly stable as the forecast horizon increases from 4 to 12 months, suggesting good predictive performance over longer forecast horizons.

A key benefit of statistical forecasts is that they can be optimized relative to an explicit trade-off between false positives and false negatives. This trade-off may be different for different policy purposes. When the costs of full-blown food crises are large and preventive measures have high returns, a forecasting framework with a high tolerance for false positives might be appropriate. Conversely, when resources for prevention are scarce, a greater tolerance for false negatives is more suitable. Our validation framework made this trade-off explicit, by tuning the parameters of our models to minimize prediction loss functions that balance false positives and false negatives. Depending on the use case, calibrated predictions can be generated that strike the appropriate balance between false positives and false negatives dictated by the application at hand. We acknowledge, however, that putting this principle into practice is a difficult task. While specifying a loss function that trades off false positives and false negatives is mathematically straightforward, setting the appropriate weights requires careful deliberation of the human and economic implications of competing errors, both of which are subject to considerable uncertainty.

Combined, the results show that statistical models can provide new and attractive forecasting capabilities in various ways, including an improved ability to detect new crisis outbreaks for a specified tolerance for raising (false) alarms; calibrating error types to take the costs and benefits of specific interventions into account; increasing lead time for preventive action by forecasting further into the future, and updating predictions continuously using readily-available, verifiable and open high-frequency data. Realizing these benefits will require close coordination between statistical modeling work and policy dialog, to ensure that the design of warning systems is tailored to the policies they are intended to inform.

20