The Civil Rights Movement and the Logic of Social Change

This important book assesses the level of unreliability—random measurement error—in individual survey items found in general-population surveys, on which much scholarship in sociology and kindred fields depends. Duane Alwin aspires to reduce measurement error at its source by identifying less error-prone methods of constructing and administering surveys. His study contributes to understanding survey quality by showing how reliability varies with item content and instrument design; many findings provide empirical grounding for well-established survey practices, while others suggest that some common data collection protocols may heighten error. The study rests on an original, unique data base of reliability estimates for nearly 500 individual survey items drawn from longitudinal surveys representing well-defined populations. Questions included measure both basic sociodemographic facts and subjective phenomena (beliefs, attitudes, selfperceptions). Alwin coded item properties (number of response alternatives, length), question content (factual or nonfactual), and survey context (inclusion in a topical series or ‘‘battery’’ of related questions, ordinal position within a questionnaire), and then assessed associations between these design features and reliability. As befits a study of data quality, much of Margins of Error justifies the measurement of its dependent variable, item reliability. Three chapters that outline and critique extant approaches to reliability assessment can be read profitably on their own. But the key here is that Alwin seeks reliability measures for single survey items, not composite scales. He stresses the distinction between multiple measures (verbatimreplicated items) and multiple indicators (distinct items related to a common underlying construct). He finds widely-applied ‘‘internal consistency’’ approaches based on classical test score theory (coefficient a) wanting, because they estimate the reliability of multiple-indicator composites rather than individual items, and because such composites need not be ‘‘univocal’’—that is, they combine indicators that often have imperfectly correlated true scores. A particular difficulty is that those methods understate item reliability by classifying stable, but measure-specific, variance in a survey response as erroneous rather than reliable. Alwin argues that cross-sectional designs cannot adequately estimate the reliability of single items, because respondent memory raises correlations among multiple measures or indicators. He advocates longitudinal designs that administer identically worded questions on at least three occasions, suggesting that those measurements be separated by intervals of up to two years to avoid memoryinduced inflation of reliability estimates. When these demanding data requirements are met, suitable analytic methods can distinguish reliability and stability, and incorporate stable item-specific variance within true score variance. Many results substantiate widely-used and -taught guidelines for constructing survey instruments. For example, reliability tends to be higher for factual questions than for items measuring subjective content, for selfreports than for proxy responses about others, and (usually) for shorter questions. In keeping with much recent methodological research on survey data, Alwin invokes cognitive considerations to interpret such associations; he suggests, for instance, that respondents may better comprehend short questions, and more readily access and retrieve information needed to answer factual ones. Of particular note is Alwin’s finding that the widespread survey practice of presenting items in batteries—sets of consecutive questions using the same response format—tends