Sensitivity to statistical regularities: People (largely) follow Benford’s law

Sensitivity to statistical regularities: People (largely) follow Benford’s law Bruce D. Burns (bburns@psych.usyd.edu.au) School of Psychology, Brennan MacCallum Bldg, A18 University of Sydney, NSW 2006, Australia Abstract Recent decision making research has emphasized people’s sensitivity to statistical relationships in the environment. A little-known relationship is Benford’s law, that the first digits of numbers representing many natural and human phenomena have a logarithmic distribution (Benford, 1938). Benford’s law is being used to help detect fraudulent financial data, but this assumes that people will not follow Benford’s law when generating data. In two studies I examined whether people follow Benford’s law. In Study 1 participants were given nine questions (e.g., “Length of the Indus river: km”) chosen to have a flat distribution of first-digits for correct answers. The generated distribution was close to Benford’s law. In Study 2 the results for generated data were replicated with new questions, and a selection task was also given in which participants selected from nine possible answers. Selected answers were a poor fit to Benford’s law. Taken together the results suggest that Benford’s law is a product of the way people generate responses, rather than sensitivity to the relationship itself. Keywords: Benford’s law; randomness; decision making; forensic accountancy. Introduction A common theme in recent research into reasoning and decision making has been that people are influenced by statistical relationships in the environment. This is a key part of adaptive approaches to decision making such as that of Gigerenzer, et al, (1999) and it underpins apparent automaticities in everyday life (Bargh & Ferguson (2000). Key to these approaches is that people follow statistical relationship of which they have little awareness. Rarely though is it possible to test if people are truly acting precisely in accord with an unknown pure statistical relationship, rather than just following heuristics broadly consistent with measurable relationships. There are usually too many issues around conditions and sampling that exactly what the statistical relationship a person may have experienced cannot be stated precisely. Therefore Benford’s law (also called the first-digit law) offers an interesting test case, because it is a precise statistical relationship that is both universal and little known. Table 1: Percentage frequency of each first digit from theory (Newcomb, 1881) and data (Benford, 1938). Newcomb (BND) Benford’s data History. Benford’s law is named after the physicist Frank Benford, who gathered data supporting a proposal by Newcomb (1881). Newcomb noted that the first pages of logarithmic tables seemed to wear out faster, and deduced that the frequency of the first digit in numbers from nature are logarithmic. In its most general form as stated by Hill (1995), the leading digit d (d ∈ {1, …, b − 1} ) in base b (b ≥ 2) occurs with probability P(d)=log b (d + 1) − log b d = log b ((d + 1)/d). For a base 10 number system this gives the distribution Newcomb first proposed and which I will refer to as the Benford / Newcomb distribution (BND). Benford (1938) empirically demonstrated the validity of the law by collecting first-digit distributions for a number of quantities, both natural and human. For 20 different quantities with an average of 1011 data points each he found the mean distribution shown in Table 1, which for each digit was within the margin of error of Newcomb’s (1881) prediction. These quantities included rivers, physical constants, cost data, populations and newspaper circulations. Because logarithms yield the only frequency distribution that is invariant under transformation, it does not matter what units these quantities are expressed in or even if they are expressed as reciprocals. Benford noted that the poorest fitting data was data such as physical data (e.g., weights) and the best was for numbers generated by no known relationship (such as river lengths), so he called it the law of anomalous data. Since Benford’s demonstration this logarithmic relationship has been shown to apply to many types of naturally occurring data such that its validity is not disputed, though its scope is a more open question. Not all data would be expected to follow Benford’s law, for example telephone numbers, lottery numbers, and populations of villages under 1000 would not. Benford's law long remained just a curiosity for mathematicians who argued over how it could be derived from more general mathematical principles (see Raimi, 1976). Hill (1995) seems to have resolved the mathematical issue somewhat by deriving Benford’s law from the assumption of scale invariance. Further, Hill (1998) argued that because a distribution seems to fit better the more it arises from completely unrelated data (e.g, batting averages, areas of rivers); the critical point may be that the data is a combination of different distributions. So he proposed a theorem, “If distributions are selected at random (in any ‘unbiased’ way) and random samples are taken from each of these distributions, then the significant-digit frequencies of the combined sample will converge to Benford's distribution, even though the individual distributions selected may not closely follow the law. ” (p. 6).