relationship between shock and impact for other households of the same type (type being defined according the list of vulnerability‐determining household characteristics we identify as attenuating factors in the drought‐consumption relationship) that can be replicated in other contexts/ using other datasets. As noted below, one of the datasets used in the analysis has now been updated to a 2‐period panel, which could potentially be an informative extension to the analysis.

# **d. External validity**

External validity is concerned with whether results of any individual piece of analysis are generalizable to other contexts. Whilst in scientific contexts, it may be possible to establish complete external validity for certain experiments, in the context of social science, the question is rather _how_ generalizable are the results to other contexts? In our case, the model is to be estimated on rural Ethiopian (nationally representative) data, so as long as we believe the results to be internally valid, then we could say that they are representative for the population in that particular time and place (Ethiopia, 2011 for example). However, it is more difficult to establish whether the relationships we find between rainfall and consumption in 2011 are generalizable to other years with e.g. more extreme drought.

It is helpful to define three specific sub‐topics within overall external validity that are relevant in our context. These are referred to as EV1) out‐of‐sample shock estimates EV2) time‐stability of estimated relationships and EV3) concerns around over‐fitting to the training data. We define this in more detail below.<sup>13</sup> EV1) Antilla‐Hughes and Sharma (2014, hereafter, AHS) note that the main threat to external validity in the context of specifying the damage function D(.) as outlined above, lies in mis‐specifying the function for values of the hazard (shock) that were unavailable for the analysis‐ in particular, extreme values that occur only very infrequently (e.g. a once‐in‐a‐generation severe drought, typhoon, tsunami etc).

For the initial model estimation Hill and Porter (2014) combined the years 2004‐5 and 2010‐11 in order to allow the broadest (artificially) cross‐sectional range of values for our shock. In some pre‐ analysis, we conducted a careful analysis of the range of data (see LEAP document). The issue in the context of our dataset is the limited number of observations available at some of the high levels of crop loss (e.g. greater than 50 percent crop loss).

Figure 2.1 shows the distribution of the crop loss data for 2011 and 2005 combined, and it can be seen that the distribution is left‐skewed. Where we identify data paucity that challenges the strength of fit of the modelled relationship at this extreme end of the crop loss spectrum, we need to highlight a caveat regarding the sparse data above 60% crop loss. Table 2.3 outlines in more detail the number of observations in each “bin” of data at 5% intervals. 2005 is somewhat worse year than 2011, with a higher mean crop loss, and 300 observations above 50%. However in 2011, just under 100 observations lie between 50 and 60%, and none at all are above 60%. The distribution can then be said to be somewhat different between years, and we therefore examine the difference between the pooled dataset that has a broader distribution, with the use of one round of data as the training data, and another round as the testing data, in order to test the stability of the drought‐consumption relationship over different drought distributions.

> 13 EV1) is discussed as overlapping within the scope of the internal validity checking.

13