Uncertainty about uncertainty within a stakeholder group

The explicit quantification of uncertainty is one of the useful features of Bayesian network models. Identification and quantification of uncertainty is particularly important in complex fields such as environmental management, where there may be great natural variability and data uncertainty, and there may not be sufficient data available to characterise all relevant variables. Bayesian networks also encourage explicit decisions about the subjective choices that can be common in management generally, for example in deciding on the relative importance of non-comparable variables. In this paper, some of the data from a previous study in the Bayesian framework is further examined. The previous project is described in more detail in a previous paper, however, here we focus specifically and in more detail on the expert elicitation process, outcomes and implications than was possible in the previous paper. The original project involved using an Ecological Risk Assessment approach to guide the overall project design, with a participatory approach used to build a Bayesian network model with input from a range of stakeholders in the Kongulai Catchment in the Solomon Islands. This process incidentally produced quantified subjective opinion data from one group of expert stakeholders, those involved professionally in water or catchment management. Specifically, we examine the variation in information elicited from the individual experts for the conditional probability tables used in the Bayesian network model, using Median Absolute Deviation (MAD) as a simple but robust descriptive statistic to quantify differences in opinion between experts. For univariate data X , X , . X ; the MAD = median ( |X - median (X )| ). That is, the median of the set of absolute differences between each value and the median. The quantification of variation in opinion within this group allows us to explore and identify sources of this variation and divergence. As the experts provided estimates of uncertainty regarding catchment relationships, the MAD then provides a measure of their 'uncertainty about uncertainty'. Some primary features illustrated with this representation of variation in expert opinion: • the greatest agreement between experts (lowest MAD) usually occurs where pressures (parent variables) all act on a variable in the same direction, • greater disagreement (higher MADs) are seen where pressures act on a variable in opposing directions, and also where there is one or several variables where little is known, • the highest average MAD can be used to identify what combination of pressures cause the greatest uncertainty between experts, and can narrow identification where there is one of multiple pressures causing the greatest uncertainty. Difficulties encountered in the use of this measure include its low resolution and discarded outliers, inconsistencies in expert opinion, and time constraints limiting the complexity of the elicitation and the number of experts involved. Further work would be valuable to explore the impact of a larger group of experts, including those with non-traditional forms of expertise, such as local knowledge, and alternative elicitation processes. How quantification of uncertainty, and further, our uncertainty about uncertainty, can be used in the wider context of natural resource management remains unresolved.