Efficacy of Bias Awareness in Debiasing Oil and Gas Judgments

Efficacy of Bias Awareness in Debiasing Oil and Gas Judgments Matthew B. Welsh (matthew.welsh@adelaide.edu.au) Australian School of Petroleum, University of Adelaide, North Terrace Adelaide, SA 5005, Australia Steve H. Begg (steve.begg@adelaide.edu.au) Australian School of Petroleum, University of Adelaide, North Terrace Adelaide, SA 5005, Australia Reidar B. Bratvold (reidar.bratvold@uis.no) Department of Petroleum Engineering, University of Stavanger 4036 Stavanger, Norway they are certain to some stated level of confidence that a value will fall within. In an industry such as oil and gas, biased decision-making can be extremely expensive; industry observers argue that such biases are, in part at least, responsible for the large losses in the oil industry – one recent estimate being that unexpected outcomes during exploration cost the industry US$30 billion each year (Goode, 2002). With the cost of drilling a single, off-shore well regularly exceeding US$60 million, the estimates of technical parameters that feed into the models of oil fields used to predict oil reserves can have a huge impact on whether a company chooses to drill or walk away from a prospect. If these estimates are subject to cognitive biases – as most judgments under uncertainty are – the accuracy of these models are also compromised and poor decisions will result. Abstract It is argued that biases such as anchoring and overconfidence contribute to a US$30 billion/year loss in the oil and gas industry (Goode, 2002). The most commonly used debiasing technique, within the industry, is awareness-style training, where participants have the biases and debiasing techniques described to them without specific training. Given such training is delivered by consultants, however, there is little available evidence of its efficacy and concern regarding a lack of up to date debiasing methods. We present a study designed to measure the benefit of such awareness-style training for the well-known anchoring and overconfidence biases, using a sample of petroleum engineering students. Results indicate that course attendance reduced participants’ overconfidence - calibration improving by 21% - but no benefit was observed for anchoring. It is argued that this difference results from the debiasing technique for anchoring requiring a greater degree of domain knowledge than the students possessed. A detailed analysis of the relationship between accuracy in, and susceptibility to, the anchoring question supported this – indicating that participants who simply relied on the anchor actually performed better than those who hazarded their own guesses. Potential benefits of debiasing and, specifically, the incorporation of up to date debiasing techniques, are discussed along with the need for further research. Keywords: decision making; debiasing; overconfidence; oil and gas; judgment. Biases in Oil and Gas Decisions anchoring; The question for the industry, of course, is: which biases are affecting oil and gas decisions and how can these be reduced? This question has not, of course, avoided investigation. Within a few years of Tversky and Kahneman’s (1974) groundbreaking work on biases, Capen (1976) introduced the concept of overconfidence to the industry – demonstrating its effect on oil and gas professionals. In 2002, however, Hawkins, Cunningham and Coopersmith (2002) noted that less than half of observed values fell inside the 80% confidence intervals commonly used for predicting uncertain parameters in the oil industry, which seems clearly indicative of overconfidence’s continued impact. Merkhofer (1987) has generalized the results of these type of studies to the “2/50” rule: When people are asked to provide estimates of quantities lying between their 1% and 99% confidence limits (that is, with only a 2% chance of being wrong), usually about 50% of the answers fall outside this range and, thus, they are overconfident in their predictions. Many oil and gas professionals are also familiar with the anchoring bias, through the work of Rose (2001), but other biases are rarely mentioned (see Pieters, 2004 for a recent exception). This is regarded as particularly damaging due to the use of analogs in oil and gas decision-making. That is, given the scarcity of data on a new oil prospect, analogous Introduction Previous research has demonstrated that most people are susceptible to cognitive biases when making decisions or judgments under conditions of uncertainty (Morgan & Henrion, 1990; Tversky & Kahneman, 1974). That is, people tend to give responses that are systematically biased compared to calculated, optimal solutions as a result of the simplified mechanisms that our brains use to make judgments. Two of the most commonly discussed biases are: anchoring (Tversky & Kahneman, 1974), which describes the tendency people have to base estimates on any number at hand regardless of its relevance to the question at hand (Chapman & Johnson, 2002); and overconfidence (Lichtenstein, Fischhoff, & Phillips, 1982; Tversky & Kahneman, 1974), which describes the tendency of people to give too narrow bounds when asked to set a range that