Who Lies on Surveys, and What Can We Do about It?

Focusing on demographic characteristics, the author seeks partially to replicate, with a larger set of variables, Belli, Traugott, and Beckmann's (2001) recent study on vote overreports using the same data from the U.S. National Election Studies (1948-1998). His analyses show that Blacks and residents of the Southern States in general are most likely to make false statements on how they voted. He suggests a possible solution for inaccuracies in survey data and proposes that, when validation of verbal responses is not possible, it may be prudent, if feasible, to re-estimate models with and without Black and Southern respondents to make sure that findings are robust. Key Words: Center for Political Studies; the National Election Studies; surveys; demography. Empirical support for a scientific theory depends not only on the validity of the theory but also on the quality of the data. Given that a large number of social scientific studies rely on surveys, the quality of survey data, whether respondents answer the questions accurately, is crucially important for the cumulative knowledge in social sciences. If there are systematic (as opposed to random) errors in survey responses, some of what we think we know in social sciences may not be true. Accuracy of responses is not the only determinant of the quality of data, but it is an important one. It is not a sufficient condition for quality data, but it is a necessary one. Of course, under ordinary circumstances, it is very difficult, if not impossible, to ascertain whether respondents are being truthful in their answers because the true answers to the questions are usually unknowable to the interviewers. The National Election Studies, conducted by the Center for Political Studies at the University of Michigan, provide a unique opportunity to ascertain the truthfulness of survey responses, which few other data sets offer. Over the last half-century, they have conducted validation studies: The interviewers first ask the respondents whether they are registered to vote in their precinct and whether they voted in the last election, and then they go to the respondent's precinct to look up the registration and voter turnout record to verify the respondent's verbal responses. The National Election Studies therefore allow researchers to determine who are more likely to give truthful responses to survey questions and who are more likely to misreport. Past analyses of the demographic determinants of misreporting have, however, produced contradictory findings, with respect to the effect of sex (Belli, Traugott, and Beckmann 2001; Hill and Hurley 1984; Traugott and Katosh 1979); age (Belli et al. 2001; Hill and Hurley 1984; Katosh and Traugott 1981; Sigelman 1982; Traugott and Katosh 1979; Weiss 1968); education (Belli et al. 2001; Bernstein, Chadha and Montjoy 2001; Hill and Hurley 1984; Silver, Anderson and Abramson 1986; Traugott and Katosh 1979); and income (Hill and Hurley 1984; Katosh and Traugott 1981; Traugott and Katosh 1979). These studies therefore do not allow researchers unequivocally to predict who are most likely to misreport their behavior on survey questions. The only demographic characteristic that consistently and reliably predicts vote misreporting is race: Blacks are more likely to lie than non-Blacks (Anderson, Silver, and Abramson 1988; Bernstein et al. 2001; Belli et al. 2001; Hill and Hurley 1984; Katosh and Traugott 1981; Sigelman 1982; Traugott and Katosh 1979). This conclusion from the National Election Studies is consistent with findings from other analyses of survey responses on entirely different behavior. For instance, Mensch and Kandel (1988) report that Blacks and Hispanics are most likely to misreport their drug use. Belli et al. (2001) conduct the most comprehensive analysis of factors that predict misreporting of voting in the National Election Studies. They include a host of social (demographic), attitudinal, and contextual factors in an attempt to discriminate overreporters from validated voters and admitted nonvoters. …