DOI: 10.3886/e100113v2
title: Precision and Disclosure in Text and Voice Interviews on Smartphones: 2012 [United States]

description: As people increasingly communicate via asynchronous non-spoken modes on mobile devices, particularly text messaging (e.g., SMS), longstanding assumptions and practices of social measurement via telephone survey interviewing are being challenged. This dataset contains 1,282 cases, 634 cases that completed an interview and 648 cases that were invited to participate, but did not start or complete an interview on their iPhone. Participants were randomly assigned to answer 32 questions from US social surveys via text messaging or speech, administered either by a human interviewer or by an automated interviewing system. 10 interviewers from the University of Michigan Survey Research Center administered voice and text interviews; automated systems launched parallel text and voice interviews at the same time as the human interviews were launched. The key question was how the interview mode affected the quality of the response data, in particular the precision of numerical answers (how many were not rounded), variation in answers to multiple questions with the same response scale (differentiation), and disclosure of socially undesirable information. Texting led to higher quality data—fewer rounded numerical answers, more differentiated answers to a battery of questions, and more disclosure of sensitive information—than voice interviews, both with human and automated interviewers. Text respondents also reported a strong preference for future interviews by text. The findings suggest that people interviewed on mobile devices at a time and place that is convenient for them, even when they are multitasking, can give more trustworthy and accurate answers than those in more traditional spoken interviews. The findings also suggest that answers from text interviews, when aggregated across a sample, can tell a different story about a population than answers from voice interviews, potentially altering the policy implications from a survey.
description: Weights: Our primary goal was to test experimental effects, not to generalize to a population beyond our convenience sample. Thus no weights were calculated. In addition, there was no evidence that nonresponse affected the composition of the respondent sample so no weights were calculated to adjust for nonresponse. This is discussed in Conrad et al., 2017.
description: Response Rates: The response rate below was calculated using AAPOR RR2.46.4% (654/1,409) - Mode Choice
description: Presence of Common Scales: The questionnaire contained 32 questions after answering “yes” to a “safe-to-talk question”. Twenty five questions came from ongoing social surveys (Behavioral Risk Factor Surveillance System [BRFSS], the National Survey on Drug Use and Health [NSDUH], the National Health Interview Survey [NHIS], the General Social Survey [GSS], and published methodological studies [Conrad, Brown, and Cashman 1998; Tourangeau, Couper, and Conrad 2007] administered in the United States. Five questions were developed for this study. Two questions were asked a second time with a definition presented. <br>The 32 questions contained a variety of response options (detailed below).<br>How often do you now smoke cigarettes? Every daySome daysNot at allHave you smoked at least 100 cigarettes in your entire life?YesNoHave you ever, even once, used marijuana or hashish?YesNoDuring the past 30 days, on how many days did you drink one or more drinks of an alcoholic beverage?[Open-ended numeric]During the past 30 days, on how many days did you have 5 or more drinks on the same occasion?[Open-ended numeric]In a typical week, about how often do you exercise? Less than 1 time per week1 or 2 times per week3 times per week,4 or more times per weekHow many sex partners have you had in the last 12 months?[Open-ended numeric]During the past 12 months, have your sex partners been ...Exclusively maleExclusively femaleBoth male and femaleHad no partnersAbout how often did you have sex during the last 12 months? Not at allOnce or TwiceAbout once a monthTwo or three times a monthAbout once a weekTwo or three times a weekNow thinking about the time since your eighteenth birthday (including the recent past that you've already told us about), how many female partners have you had sex with?[Open-ended numeric]Thinking about the time since your eighteenth birthday (including the recent past that you've already told us about), how many male partners have you had sex with?[Open-ended numeric]Now I'm going to read a list of terms that people sometimes use to describe themselves:  "A" heterosexual or straight; "B" homosexual, gay or lesbian; and "C" bisexual.  Which option best describes how you think of yourself?  Heterosexual or straightHomosexual, gay or lesbianBisexualNow I'd like to ask you some questions specific to your diet. [pause] For the next group of questions, I'll ask you how much you favor or oppose engaging in certain behaviors. After I say the behavior, please say one of the following: ‘strongly favor’, ‘somewhat favor’, ‘neither favor nor oppose’, ‘somewhat oppose’ or ‘strongly oppose’. The first behavior...avoiding fast food.Strongly favorSomewhat favorNeither favor nor opposeSomewhat opposeStrongly opposeThe next behavior: Maintaining a healthy diet.Strongly favorSomewhat favorNeither favor nor opposeSomewhat opposeStrongly opposeThe next behavior: Monitoring cholesterol levels closely.Strongly favorSomewhat favorNeither favor nor opposeSomewhat opposeStrongly opposeThe next behavior: Emphasizing the taste of food rather than its nutritional value.Strongly favorSomewhat favorNeither favor nor opposeSomewhat opposeStrongly opposeThe next behavior: Paying close attention to the nutritional information on food packaging.Strongly favorSomewhat favorNeither favor nor opposeSomewhat opposeStrongly opposeThe next behavior: Limiting the amount of red meat in your diet.Strongly favorSomewhat favorNeither favor nor opposeSomewhat opposeStrongly opposeAnd finally, the last behavior: Balancing one's diet across the key food groups.Strongly favorSomewhat favorNeither favor nor opposeSomewhat opposeStrongly opposeDuring the last month, how many times did you eat spicy food?[Open-ended numeric]How often do you attend religious services? At least once a weekAlmost every weekAbout once a monthSeldomNeverHow often do you read the newspaper? Every dayA few times a weekOnce a weekLess than once a weekNeverOn the average day, about how many hours do you personally watch television?[Open-ended numeric]During the last month, how many movies did you watch in any medium?[Open-ended numeric]During the past 12 months, how many movies have you seen in movie theaters?[Open-ended numeric]During the last month, how many times did you shop in a grocery store?[Open-ended numeric]During the last month, how many times did you eat in restaurants?[Open-ended numeric]How many songs do you currently have on your iPhone?[Open-ended numeric]How many apps do you currently have on your iPhone?[Open-ended numeric]How many text messages have you sent and received on your iPhone in your current billing cycle?[Open-ended numeric]Finally, we have two last questions on earlier topics in the interview. Let's return to smoking. This time be sure to include any puffs on any cigarettes, whether or not you inhaled AND whether or not you finished them. Keeping this in mind, have you smoked at least 100 cigarettes in your entire life?YesNoAnd for the last question, let's return to newspapers. This time be sure to consider any newspaper content no matter whether it appears in print, online or on a mobile device. And be sure to count only newspapers from recognized journalistic outlets. Keeping this in mind, how often do you read the newspaper? Every dayA few times a weekOnce a weekLess than once a weekNever
description: computer-assisted telephone interview (CATI)~~mixed mode~~other~~telephone audio computer-assisted self interview (TACASI)~~telephone interview~~web-based survey~~iPhone users were recruited from Craigslist, Facebook, Google Ads, and Amazon Mechanical Turk. They were asked to complete a screening questionnaire to determine if they were eligible to participate. To be eligible, one needed to be 21 or older, and own an iPhone with a US area code. Eligible participants who provided a telephone number in the screener were sent a text message with a link to a web page. The web page captured the user-agent string to determine if the device was an iPhone. Once eligible, phone numbers were assigned an interview mode. Once the interview had been completed, respondents were sent a link via text message to a post-interview debriefing questionnaire concerning their experience. At the conclusion of the post-interview debriefing, respondents were sent a text message with a $20 iTunes gift code as a token of appreciation for their time.
description: iPhone users 21 years of age and older.Smallest Geographic Unit: United States
description: A convenience sample of iPhone users was recruited from Google Ads, Amazon Mechanical Turk, Facebook Ads, and Craigslist (see Antoun et al, 2016 for a discussion of sampling sources).<br><br>Participants recruited were not intended to represent the US population, iPhone users, or smartphone users. The sample was designed to test experimental manipulations through random assignments and conditions on a consistent platform.