Detecting Data Errors in Organizational Settings: Examining the Generalizability of Experimental Findings

Introduction There is a growing recognition of the importance of managing information (Davenport, 1997), and managers are increasingly recognizing the importance of information quality (Huang et al., 1999). Several studies show that data stored in organizational databases have a significant rate of errors (e.g., Laudon, 1986; Madnick and Wang, 1992; Morey, 1982; Redman, 1992). Recent developments in information systems such as enterprise resource systems (Escalle and Cotteleer, 1999; Robinson and Dilts, 1999) and data warehousing (Ferdinandi, 1999) have increased the need to eliminate errors in data. If efforts to eliminate data errors are unsuccessful, the organizational price in terms of poor support of business processes and decision making is likely to be high (Redman, 1995). Two approaches to eliminating errors in data are (1) validating data as they are input to or stored in databases and (2) depending on users to detect and correct errors. Two laboratory experiments examining the efficacy of the second approach have been completed. Experiments show that giving users incentives helps them detect errors in data and that expectations about the base rate of errors in data improve error detection. The findings provide the foundation for organizational initiatives designed to improve error detection. However, experimental findings are often criticized as not being generalizable to organizational settings (Fromkin and Streufert, 1976). This paper discusses the results of a field interview study conducted to link the findings of laboratory experiments on user detection of data errors to practice in organizations. Twenty interviews were conducted with professionals in four business domains in order to increase our understanding of the applicability of the laboratory findings to organizational practices. The remaining sections of this paper present (1) a review of prior research bearing directly on the question of the conditions under which individuals detect errors in data, (2) a discussion of the theoretical framework underlying the study and the propositions tested in the study, (3) the results of the prior laboratory experiments on user detection of data errors (4) the design of the field interview study, (5) the results of the field interview study, and (6) conclusions and suggestions for further research. Background The relevant research on data quality can be summarized with several conclusions. First, there is agreement that data accuracy, currency, and completeness are important aspects of data quality (Agmon and Ahituv, 1987; Davis and Olson, 1985; Fox et al., 1993; Huh et al., 1990; Madnick and Wang, 1992; Wand and Wang, 1994; Zmud, 1978). Second, error rates significantly greater than zero have been found in all of the studies addressing the extent to which data errors exist in databases (Ham et al., 1985; Johnson et al., 1981; Knight, 1992; Laudon, 1986; Stone and Bublitz, 1984). Third, while some researchers have proposed methods designed to completely rid databases of errors (Janson, 1988; Svanks, 1988; Naus, 1975; Parsaye and Chignell, 1993), others propose tools for determining how to best allocate limited resources to controlling the level of data errors (Ballou and Pazer, 1987; Ballou and Tayi, 1989; Ballou et al., 1987; Bowen, 1992; Paradice and Fuerst, 1991). Fourth, a variety of approaches for using imperfect data have been suggested (Ballou and Pazer, 1985; Ballou and Pazer, 1995; Ballou and Pazer, 1987; Bansal, 1993; Gaba and Winkler, 1992; Garfinkel et al., 1986; O'Leary, 1993; O'Neill and Vizine-Goetz, 1988). The early literature on data quality argues that users are not very capable of finding data errors and then changing the way that they use the data. Several studies found evidence of poor performance in detecting data errors. Davis et al. (1967) conducted a field experiment in which approximately half the subjects failed to detect important errors in a banking statement. …