DOI: 10.3886/e191703v1
title: Interviewer-Respondent Interaction with Voices Heard 2013-14

description: We examine features of the response process—various behaviors of respondents and interviewers and response quality indicators—in a survey with a diverse set of respondents focused on barriers and facilitators to participating in medical research. <br>Data are from the Voices Heard computer-assisted telephone interview (CATI) survey, which was designed to measure perceptions of barriers and facilitators to participating in medical research studies that collect biomarkers (e.g., saliva and blood) among respondents from different racial and ethnic groups. The study sought to interview 400 individuals from Wisconsin, equally distributed among white, Black, Latine, and American Indian ethnoracial groups. We employed a quota sampling strategy because screening to identify members in non-white groups would have been prohibitively expensive. The quota sample consisted primarily of volunteers (for whom a rate of participation is not calculable) but also used a targeted list of names provided by a commercial vendor. The participation rate for the vendor sample was 12 percent. Interviewers conducted 410 usable interviews (in English only) with an average length of 25.21 minutes between October 2013 and March 2014. Respondents received a $20 cash incentive. The current study is restricted to the 375 respondents for which there were usable recordings to examine respondents’ behaviors and the 23 interviewers who interviewed them.<br>Fourteen trained transcribers listened to the audio recordings and transcribed every question-answer sequence. All transcriptions were verified by a second transcriber. Within a question-answer sequence, interaction was segmented into turns, a unit-of-talk from one actor—the interviewer or respondent—that was not broken up by talk from the other actor. A turn-of-talk ended when the other actor began talking, because the original actor’s talk concluded or the current actor interrupted the original actor. In addition to recording talk verbatim, transcribers wrote out tokens (e.g., “um”), and coded whether the respondent interrupted the interviewer’s initial reading of the question or the turn contained overlapping talk, freestanding laughter (laughter that occurs between words), or laugh tokens (particles of laughter that occur within words or phrases). All of the electronic transcripts were read into Stata, and we used Stata’s string coding functions to parse the respondents’ verbalizations into discrete coding categories. The vast majority of the respondents’ talk was coded automatically by creating libraries of coding categories. For example, codable answers for each question were captured by coding strings that represented verbatim repetitions of the closed-ended response categories. Similarly, libraries of string codes were created to automatically code the various ways respondents said “don’t know” (e.g., “(I) do/would not (actually/really) know,” “I have no idea/clue/knowledge,” “(I am) not (actually/really) sure/certain (on that question/what I think),” “I (actually/really) did/do not understood/understand”) or ask for clarification or repetition (e.g., “can you (repeat/restate/say) (that/the) (question/answers/categories/choices) (again)”). As a respondent’s talk was automatically coded by replacing specific verbalizations with coding categories, the coded strings were deleted from the respondent’s turn until no talk remained to be coded. Initial coding and creation of the libraries for each coding category was done in Stata by a single coder (one of the authors). A second coder read through the string coding commands and verified the coding categories contained strings that were internally consistent (another author). Disagreements were resolved collaboratively. While our process does not allow us to produce reliability statistics as are found when two or more humans serve as coders, our automated coding process ensures a very high rate of computational reproducibility.<br>Fourteen trained transcribers listened to the audio recordings and transcribed every question-answer sequence. All transcriptions were verified by a second transcriber. Within a question-answer sequence, interaction was segmented into turns, a unit-of-talk from one actor—the interviewer or respondent—that was not broken up by talk from the other actor. A turn-of-talk ended when the other actor began talking, because the original actor’s talk concluded or the current actor interrupted the original actor. In addition to recording talk verbatim, transcribers wrote out tokens (e.g., “um”), and coded whether the respondent interrupted the interviewer’s initial reading of the question or the turn contained overlapping talk, freestanding laughter (laughter that occurs between words), or laugh tokens (particles of laughter that occur within words or phrases). All of the electronic transcripts were read into Stata, and we used Stata’s string coding functions to parse the respondents’ verbalizations into discrete coding categories. The vast majority of the respondents’ talk was coded automatically by creating libraries of coding categories. For example, codable answers for each question were captured by coding strings that represented verbatim repetitions of the closed-ended response categories. Similarly, libraries of string codes were created to automatically code the various ways respondents said “don’t know” (e.g., “(I) do/would not (actually/really) know,” “I have no idea/clue/knowledge,” “(I am) not (actually/really) sure/certain (on that question/what I think),” “I (actually/really) did/do not understood/understand”) or ask for clarification or repetition (e.g., “can you (repeat/restate/say) (that/the) (question/answers/categories/choices) (again)”). As a respondent’s talk was automatically coded by replacing specific verbalizations with coding categories, the coded strings were deleted from the respondent’s turn until no talk remained to be coded. Initial coding and creation of the libraries for each coding category was done in Stata by a single coder (one of the authors). A second coder read through the string coding commands and verified the coding categories contained strings that were internally consistent (another author). Disagreements were resolved collaboratively. While our process does not allow us to produce reliability statistics as are found when two or more humans serve as coders, our automated coding process ensures a very high rate of computational reproducibility.