**Table 5.1: Testing the models with 2005 and 2011 as training and “holdout” data**

||Baseline|Model 1|Model 2|Model 3|Model 4|
|---|---|---|---|---|---|
|Original R<sup>~~2~~</sup>|0.245|0.246|0.247|0.247|0.251|
|<br>Original Adj.R<sup>2</sup>|0.244|0.245|0.245|0.246|0.249|
|Original AIC|20047.21|20022.04|20072.26|20065.60|20001.73|
|Original BIC|20271.92|20270.00|20335.72|20360.06|20319.43|
|Num.obs.|17134|17134|17134|17134|17134|
|boot.cv|0.170|0.171|0.170|0.170|**0.169**|
|RMSEtest05|**0.0232**|0.0281|0.0338|0.0272|0.0291|
|RMSEtest11|0.0438|0.0431|**0.0427**|0.0429|0.0490|

Notes: cv=cross‐validation, R2=r‐squared; AIC=Aikike information Criterion; BIC= ; boot=bootstrap; RMSE=root mean squared error. HRE=Highlands, Reliable Region; HDR=Highland, drought‐prone Region; LOE=Lowlands Enset growing region; LOR=Lowlands reliable region. 05,11 refer to the datasets collected in 2005, 2011 respectively (and defined above).

We then repeat the exercise, but this time considering the whole dataset for 2005/2011 as the training/testing dataset respectively. In this way one might imagine going back in time to 2005, predicting consumption with that dataset for 2011, and assessing the results. As a though exercise we also repeat the process but with the training and testing roles reversed. When 2005 is the training dataset, Model 2 is the best predictor (lowest MSE). However when 2011 is the training dataset, the baseline model performs best.

Comparing how useful the two datasets are as “training” datasets. For all the models the MSE is smaller when 2005 is the testing dataset (& 2011 is training) than when 2011 is the testing dataset (& 2005 is the training dataset). This means that the 2005 dataset fits the model predicted by the 2011 dataset better than the 2011 dataset fits the model predicted by the 2005 dataset In other words, the model fitted on the 2011 dataset predicts the 2005 data more accurately that the model from 2005 predicts the 2011 data. This is a somewhat unexpected result, given that the 2005 dataset has more variation in the rainfall/drought variable. However, it must also be remembered that the PSNP safety net was introduced in mid‐2005, just after the survey was completed, which may have changed the structural relationship between drought and consumption in ways that are not captured perfectly by the regression model.

In all cases there is actually fairly small difference in the fit of the model, which suggests that the relationship between drought and consumption is actually fairly homogenous, and stable.

**Table 5.2: Testing the models with 2005 and 2011 as training and “holdout” data – squared model**

||Baseline|Model 1|Model 2|Model 3|Model 4|
|---|---|---|---|---|---|
|R<sup>~~2~~</sup><br>|0.245|0.246|0.244|0.245|0.248|
|Adj.R<sup>2</sup>|0.244|0.245|0.243|0.243|0.246|
|AIC|20044.22|20020.87|20068.66|20065.60|20003.63|
|BIC|20276.69|20276.58|20339.87|20360.06|20329.08|
|Num.obs.|17134|17134|17134|17134|17134|
|boot.cv|0.170|0.171|0.170|0.170|**0.169**|
|RMSEtest05|0.0297|0.0386|0.0338|0.0344|0.0348|
|RMSEtest11|0.0412|0.0408|0.0422|0.0424|0.0453|

Notes: cv=cross validation. R2=r‐squared; AIC=Aikike information Criterion; BIC= ; boot=bootstrap; RMSE=root mean squared error.

The results using the squared models appear very similar to those of the linear model. The cross‐ validation results are identical to three decimal places. Using 2005 as training data and 2011 as testing, the parsimonious model has the lowest RMSE, and for the opposite specification, the baseline model has lowest RMSE. Comparing the overall results for the linear and quadratic models, it appears that the linear model has lower RMSE for all specifications when 2005 is the testing dataset. But when the 2011 data is the testing dataset, the quadratic model performs best. The 25