Variations in response style behavior by response scale format in attitude research

Studies concerning the impact of the length of response scales on the measurement of attitudes have primarily focused on the method bias associated with question format. At the same time another line of research has focused on the issue of response styles that affect how respondents answer to attitude questions. So far, research has paid less attention to the issue of whether the length of the response scales is related to response styles. In this study, we explore if differences in length of the response scale (i.e., method factor) have differential effects in evoking extreme and midpoint response style behavior (i.e., style factor). Our hypotheses read as follows. As the number of response categories increases, we expect subjects to be more likely to exert extreme response style. Furthermore, we expect subjects to be more likely to adopt a midpoint response style when they are offered a middle response category. To investigate these hypotheses we developed a split ballot experiment in which the number of response categories is manipulated from 5 to 11 categories. Data are collected by a random sample, large-scale web survey which allows for random assignment to the experimental conditions. The results show clear evidence of extreme response style and moderate evidence of midpoint response style. Extreme response style is not affected by the length of response scales, whereas the exertion of midpoint response style only popped up in the longer scale versions. It is well known in attitude measurement that question format can greatly influence the way subjects respond to attitude questions (Krosnick & Berent, 1993; Schwarz, 1999; Tourangeau & Smith, 1996; Van Herk, Poortinga, & Corresponding author: Natalia D. Kieruj, Tilburg University, FSW-MTO, Room P. 1.103, P.O. Box 90153, 5000 LE, Tilburg, The Netherlands; e-mail: n.d.kieruj@uvt.nl Verhallen, 2004). Seemingly minor details of response scales can lead to systematic method error, which in turn obscures the measurement of the attitude of interest. It is equally well known that response styles such as extreme response style (ERS) and midpoint response style (MRS) can alter results in several nontrivial ways when measuring attitudes (Arce-Ferrer & Ketterer, 2003; Baumgartner & Steenkamp, 2001). ERS is the tendency of respondents to choose the extreme endpoints of a rating scale (Hurley, 1998), whereas MRS is the tendency to make disproportionate use of the middle response category (Weijters, 2006). In this study we examine whether the length of a response scale, i.e., the method used, is related to the occurrence of response style behavior. The effect of varying the number of response categories has been researched in the past, however, with little reference to the issue of response style behavior. For example, Miller (1956) linked the typical use of 7-point rating scales to the amount of information people are able to maintain in the span of immediate memory (which happens to be 7). Alwin and Krosnick (1991) found that increasing the number of answering categories led to higher reliabilities. In line with this, Alwin (1997) found that questions with more categories are more reliable and more valid. Similarly, Scherpenzeel and Saris (1997) have researched the expected level of validity and reliability of any given scale, as a function of the number of answering categories, by means of Multitrait-Multimethod models (MTMM). The novelty of our research is that it links variations in scale format (i.e., method factor) to response style behavior (i.e., style factor). To the best of our knowledge there is little research on this topic. Scrolling through the literature we found quite some references about response format and its relationship to measurement issues, however, with little reference to response styles. Similarly, the literature on response styles rarely discusses the role of variations in response scale formats. Hence, this research aims at bridging the two lines of research. The article is organized as follows. We first review the literature regarding the number of response categories and the two types of response style. Secondly, we describe the split ballot experimental design that is used to explore the relationship between response formats and response behaviors and present the latent class method used for analyzing response bias. Results and conclusions are reported afterward.