Uncovering Adolescent Perceptions: Experiences Conducting Cognitive Interviews With Adolescents

BACKGROUND economic backgrounds were interviewed by three Census Recently, welfare reform legislation was passed making Bureau researchers. Interviews, lasting approximately one major changes in the welfare system for the first time in 60 hour, took place between January and February 1996 in the years. In order to capture the effects of these welfare Center for Survey Methods Research cognitive laboratory reform initiatives, one of the panels of the Survey of facility, adult education centers and private homes. Parental Income and Program Participation (SIPP) is being and individual consent were obtained for all interviews. extended to ten years. The extension was designed to Interviews were conducted using concurrent think-aloud provide information on spells of actual and potential techniques, structured probing and vignettes. Respondents program participation over a ten-year period; and to were asked to read and think aloud as they completed the examine the causes of program participation and its longquestionnaire. If necessary, interviewers would probe first term consequences on the well-being of recipients and their with general probes, then with structured probes. families. Additionally, respondents were asked a series of debriefing Regarding this latter goal, a self-administered questions after they completed the survey. All interviews supplement was developed for adolescents aged 12 17. were audio taped. The supplement was designed to measure adolescents' Eleven females and nine males were interviewed. Table perceptions of their life chances, their educational and 1 shows that nineteen of the adolescents were currently economic opportunities, as well as their experiences with attending school. One was enrolled in a GED program. parent-child conflict and exposure to violence. This paper Table 2 shows that their ages were spread evenly between focuses on our general experiences conducting cognitive 12 and 17 years. interviews with adolescents, as well as specific experiences with the parent child conflict scale and the perceptions of RESULTS the future questions. Participating in a cognitive interview, being asked to The proposed series of questions raised concerns among “think aloud”, is a unique, sometimes awkward, experience reviewers of the questionnaire with regard to such issues as for many people. Contrary to what we expected, question sensitivity, task difficulty, and the age adolescents were able to handle the cognitive interview appropriateness of selected questions. To address some of task quite well. Many of the adolescent respondents were these concerns, cognitive interviews using the selfable to articulate their thoughts clearly and concurrently, administered questionnaire were conducted. In this paper while others required more intense probing after they we describe our experience conducting cognitive answered questions. Respondents did not have the interviews with adolescents, and present results from the anticipated problems sitting still for the one hour interview cognitive interviews, which included several experimental or focusing on the task. During the debriefing, respondents components. Finally, implications of the research will be were asked how difficult they found the think aloud discussed. process. Many of the adolescents reported that it was Adolescents may provide special challenges both in difficult for them, but became easier as they went along. terms of their cognitive ability and the quality of data they Their performance indicates that they were able to handle provide. In their work conducting cognitive interviews with the task. adolescents, Stussman et al. (1993) found that “most Some of the respondents had a difficult time following teenage respondents lacked the ability or the motivation to skip patterns. As Table 3 shows, the younger respondents spontaneously articulate their thought processes” (p.383). had a more difficult time than their seniors. The younger This articulation of the thought process is necessary for respondents missed a larger number of skip patterns than cognitive interviewing to be successful. In other research, their older counterparts (2). Additionally, a larger number Amato and Ochiltree (1987) found that adolescents can of young respondents missed at least one skip pattern. become bored or distracted by short pauses. This presents Some of the skip problems may be in part due to the layout a problem for cognitive interviewing which often relies of the form. Figure 1 shows that the skip instructions were on pauses to encourage respondents to think harder a slightly different shade of gray than the rest of the about a question.