Effects of Question Wording and Format on Political Attitude Consistency

Numerous studies in the past decade have suggested that the American voter has become increasingly sophisticated, ideologically, and that this is due largely to the separate or combined impact of the growing educational attainments of the U.S. public and the greater salience of politics in the 1960s and 1970s. This study, however, hypothesizes that much of the change might be due to a simple methodological artifact-changes in question wording and format-and reports data from a recent national experiment which strongly supports such an interpretation. The generalizations derived from The American Voter may, therefore, not be quite as "time-bound" as the current literature would lead us to believe. George F. Bishop is senior Research Associate, Robert W. Oldendick is Research Associate, and Alfred J. Tuchfarber is Director, Behavioral Sciences Laboratory, University of Cincinnati. The research reported in this paper was supported by a grant from the National Science Foundation (SOC77-10109). The authors wish to thank Richard Dawson, Program Director for Political Science at NSF, for his encouraging support of this project. Public Opinion Quarterly ? 1978 by The Trustees of Columbia University Published by Elsevier North-Holland, Inc. 0033-362X/78/0042-0081/$01.75 This content downloaded from 157.55. 39.192 on Thu, 25 Aug 2016 04:18:30 UTC All use subject to http://about.jstor. org/terms 82 GEORGE F. BISHOP ET AL. national election. For it is exactly at this point that all the trend lines for attitude consistency, issue voting, and the like take a sudden upward leap not only in the aggregate but also among the well-educated and the not so well-educated, and among those interested in politics and those not at all interested. The explanation of all these trends, however, has become rather elusive. Some reseaarchers have tried to trace the growth of mass electoral sophistication to the salience of political issues in the 1960s (see, e.g., Bennett, 1973; Nie et al., 1976; Niemi and Weisberg, 1976:76-79), whereas others have emphasized the electorate's increasing educational attainments as a significance causal factor (cf. Bishop, 1976; Miller et al., 1976). But neither of these interpretations can adequately account for the sudden, steplike surge in ideological sophistication during the 1964 election period. For one thing, there is no direct evidence that the electorate became any more interested in politics at this time than at any previous period in the past two decades (see, e.g., Converse, 1972:332-33). Furthermore, we know that educational attainment in the mass public climbed only gradually in the decisive phase, 1960-1964 (see Executive Office of the President, 1973:75-110) and could thus explain only a small part of the total increase. What then accounts for the abrupt shift in the structure of American politial behavior? We have reason to believe that a much more fundamental factor is responsible, one which inflated political attitude consistency (and issue voting) in all segments of the population: changes in question wording and format. For we now know that at the pivotal point of the 1964 election the Michigan Survey Research Center made some major modifications in the wording and format of the basic,issue questions used by numerous researchers to measure the long-term trends in issue voting and attitudinal structure. Prior to 1964 almost all of the domestic and foreign policy questions in the SRC election studies appeared in a five-point, Likert format with response alternatives ranging from "strongly agree" to "strongly disagree '-including a middle category, "not sure; it depends. " But in 1964 SRC changed the issue items to a more balanced, dichotomous-choice format. And in 1968 they introduced still another variation by expanding these dichotomous-choice questions into a seven-point, semantic differential type of format. An illustration of these changes will be helpful. Take, for example, the issue of the federal government's responsibility for employment (and living standards). From 1956 to 1960 SRC interviewers had asked respondents to what extent they agreed or disagreed-on a five-point scale-with the statement: "The government in Washington should see to it that everybody who wants to work can find a job. " The interviewThis content downloaded from 157.55. 39.192 on Thu, 25 Aug 2016 04:18:30 UTC All use subject to http://about.jstor. org/terms QUESTION WORDING AND ATTITUDE CONSISTENCY 83 ers had also asked the following filter question immediately before reading each such statement: "Do you have an opinion on this or not? " In 1964, however, respondents received this new variation of the question and filter: In general, some people feel that the government in Washington should see to it that every person has ajob and a good standard of living. Others think the government should just let each person get ahead on his own. Have you been interested enough in this to favor one side over the other? (If Yes) What is your feeling? Do you think the government should see to it that every person has a job and a good standard of living or should it let each person get ahead on his own? And beginning in 1972, this item reads: Some people feel that the government in Washington should see to it that every person has a job and a good standard of living. Others think the government should just let each person get ahead on his own. And, of course, other people have opinions somewhere in between. Suppose people who believe that the government in Washington should see to it that every person has ajob and a good standard living are at one end of this scale-at point number 1. And suppose that the people who believe that the government should let each person get ahead on his own are at the other end-at point number 7. Where would you place yourself on this scale, or haven't you thought much about this ? To some researchers these variations may seem trivial; others may argue that while such modifications could affect our estimates of the amount or direction of change in political attitude distributions over time (i.e., "the marginals"), they would not appreciably influence the degree of relationship between items, which represents the indicator most commonly used to operationalize consistency and issue voting. But recent experiments in question wording by Schuman and his associates (Schuman and Presser, 1977; Schuman and Duncan, 1974) demonstrate that this is simply not the case. Not only did variations in wording and format alter the marginals in their studies, but the changes also significantly affected the amount of association between sociodemographic and attitudinal variables. These findings thus suggested that a good part of the increase in mass political sophistication reported in the literature might be due to a methodological artifact. An opportunity to test this hypothesis arose when the National Opinion Research Center recently released data from a set of experiments designed, among other things, to evaluate the effects of the differences in wording and format that we have described for the SRC issue