Cognitive Interviewing of Survey Items for a Computer Adaptive Test of Parent Knowledge and Beliefs
Phase: NA
Conditions: Beliefs About Child Development
Sponsor: University of Chicago

Study title: Cognitive Interviewing of Survey Items for a Computer Adaptive Test of Parent Knowledge 
and Beliefs 
Document Dated: 12/8/2021 
NCT Number: NCT04971265 
 

Cognitive Interviewing of Survey Items for a Computer Adaptive Test of Parent Knowledge 
and Beliefs 
TMW Center for Early Learning + Public Health 
Protocol Narrative 
Background 
A child’s early language exposure is pivotal in language development and in ultimate educational 
and intellectual achievement (Bruner, 1981; Chapman, 2000; Gallaway & Richards, 1994; Hart & Risley, 
1995; Huttenlocher, Haight, Bryk, Selzer, & Lyons, 1991; Rowe, 2008). Hart and Risley’s landmark study 
demonstrated the tremendous impact of a child’s early language milieu on future learning by revealing a 
significant correlation between the number of words a child is exposed to between ages 0-3 and his or 
her ultimate IQ and academic success (Hart & Risley, 1992; 1995). Their findings were both significant 
and alarming, demonstrating the critical and time-sensitive role that early language exposure plays in a 
child’s life. 
The concern for children born into poverty is the overwhelming inequity in their early language 
experience. Hart and Risley’s study revealed a steep socioeconomic gradient in early exposure to words 
and language. At the end of their third year, children from high socioeconomic status (SES) families 
heard approximately forty-five million words; children from economically impoverished backgrounds 
heard only approximately thirteen million. Hart and Risley found that more parental talk equated to 
faster vocabulary growth and higher IQ test scores at three years and older. Their follow-up with these 
children at 9-10 years of age confirmed that preschool/school interventions came too late to alter these 
trajectories; the reverberations of early linguistic shortfall were likely to follow these children 
throughout their lives.  
However, the challenges these children face is not unalterable. It is neither genetics nor a lack of 
potential that lie at the heart of this inequity, but rather parental knowledge. In 2008, Rowe 
demonstrated that the relationship between socioeconomic status and parental communication ability 
is mediated by parental knowledge (Rowe 2008). Other studies have corroborated these findings, 
demonstrating that the effect of socioeconomic status on a child’s vocabulary was primarily dictated by 
maternal speech (Hoff 2003). Additionally, a recent study by the TMW Center shows that parents who 
have higher knowledge about infant development soon after birth are more likely to engage in 
behaviors that foster socio-emotional and cognitive growth, as well as respond to their infants’ cues at 
the age of 9-months (Leung & Suskind, 2020). Encouragingly, studies have also demonstrated that a 
well-planned, parent-directed language intervention in low-income populations may positively affect 
parental speech behaviors by increasing the use of facilitative language with their children (Oneil-Pirozzi, 
2009).  
Given the pivotal role that parents and caregivers—and in particular, the interactions they share 
with children and the quality of language input they provide—play in shaping young children’s cognitive 
and language development, it is necessary to assess what they know and believe about that process. 
Prior to the development of the SPEAK, however, this critical need was unmet in the early childhood 
field. Since 2013, Dr. Suskind and her research group have developed and refined the Survey of 
Parent/Provider Education and Knowledge (SPEAK). The SPEAK is a clinical and research tool to assess 

parents' and providers’ expectations and knowledge of early childhood cognitive and language 
development.  
Two versions of the SPEAK survey, the 10-item SPEAK-C and the 20-item SPEAK-R have 
undergone rigorous item testing and evaluation to create a reliable and valid measure of parental 
knowledge (Suskind et. al, 2017). As the next phase of SPEAK development, the TMW Center is designing 
a computer adaptive test, called the SPEAK CAT, to streamline the assessment of an individual’s 
knowledge and beliefs about child development. The computer adaptive test adjusts, in real time, the 
questions that it asks to tailor to a specific individual’s knowledge level. This will allow researchers and 
clinicians to have a more precise measure that can be completed in a shorter amount of time. This data-
driven approach can then support parents through targeted delivery of educational interventions or 
other support services dependent on the results of their SPEAK assessment.    
Purpose:  
Having a reliable and valid self-report measure, such as the SPEAK, is critical for researchers and 
clinicians to be able to correctly assess a parent’s knowledge and beliefs about a child’s early cognitive 
development. Too often, self-report surveys are developed without consideration of their psychometric 
properties, leading to low-quality results. The purpose of this research protocol is to conduct cognitive 
interviews with survey respondents. Cognitive interviewing is a technique used in survey development 
through which interviewees are asked to respond to survey questions and, through guided questions 
from the researcher, explain their rationale behind the answer they chose. Cognitive interviewing can 
increase validity of a measure to ensure that researchers and survey respondents derive the same 
meaning from questions and answer choices. The TMW Center proposes using cognitive interviews as 
part of the SPEAK CAT development process to ensure that: 
 
Survey questions and answer choices are free of jargon and at an appropriate reading level for 
the target audience (i.e. parents and providers working with children ages 0-6). 
 
Survey respondents are responding to the questions in a manner that is in concordance with 
their understanding of the question meaning  
 
Survey questions intended to measure the same construct are responded to similarly by survey 
respondents, and to gauge whether some phrasing variations of the same content are easier to 
understand than others. 
 
To brainstorm wording and format revisions that could make the survey more acceptable to the 
target audience. 
Responses from the cognitive interviews will allow the TMW Center to finalize an item bank of questions 
that will be used in the SPEAK CAT tool. 
Procedures 
 
The TMW Center will conduct cognitive interviews with enrolled participants to guide the 
development of new SPEAK CAT survey items. In-person interviews will take place in a private room at 
the University of Chicago Medical Center or another community location (such as a local library). Virtual 
interviews will take place over a video calling program, such as Zoom. Each interview will last 
approximately 1 hour and will be audio or video recorded so that the participant’s responses may be 
reviewed and analyzed. 

 
At the start of the first interview, right after consent is obtained, the participant will be asked to 
complete a brief survey (The Family Supports and Beliefs Survey), which will measure their existing 
beliefs about the role of societal resources in supporting families and young children. Participants will 
randomly see one of four fictional scenarios and then be asked questions related to that scenario as well 
as items related to stress and self-efficacy. We hypothesize that the type of scenario a participant is 
presented with before taking a survey may affect how they answer subsequent questions. We may also 
use these survey responses as a way to cross-check that the way participants are responding to the 
SPEAK CAT items during the interview is congruous to other measures of their beliefs.  
We will then begin the cognitive interview. During each session, the participant will complete SPEAK 
survey items and then will answer interviewer questions related to these items. Survey items may be 
completed on paper or via REDCap. The interviewer will ask questions related to the items that the 
participant answered. These questions are meant to probe the design and readability of the question, 
and not the participant’s knowledge about the subject. For example, 
 
What does the word ‘math subjects’ mean to you when you read this question? 
 
Can you please rephrase this question in a different way? 
 
When you were answering this question, how did you interpret the difference between the 
answer choice “definitely true” and the answer choice “probably true”? 
An interview guide will be uploaded with this protocol submission. However, this will be used as a 
starting point for the interviewer. Some questions may be skipped and other questions may be asked 
based on how the respondent answers previous questions. Additionally, as additional SPEAK CAT items 
are developed or refined as part of the cognitive interviewing process, the interviewer may ask 
questions that are not in the guide to measure those items. Due to variations in participant speed and 
length of interview response, some participants may be asked more interview questions than others 
during the one-hour session. 
 
At the end of the interview, the participant will be asked to fill out an optional demographics 
form to help researchers to understand any trends in survey responses. No identifying information will 
be recorded on this survey, which will be collected on paper or through REDCap. It will be linked to their 
survey responses and interview transcripts by a study ID. 
 
Participants who are interested in completing more than one interview will give verbal 
permission to keep their contact information on file in order to contact them about future opportunities 
within this study protocol. There is no requirement that participants come back for more than one 
session, and they may continue to come back for as many sessions as they’d like, or until they have been 
interviewed about all items currently under development. No participant will complete more than 5 
interviews under this protocol. Participants who complete more than one interview will take the 
Demographics and Family Supports and Beliefs survey only once. 
Compensation 
 
For each interview a participant completes (lasting approximately 1 hour), they will receive $20 
in the form of cash or a gift card. The maximum amount any single participant will receive in this study is 
$100 (if they choose to complete the maximum of 5 interviews). Participants who complete their 

interview in-person at the University of Chicago campus will be offered a parking voucher or bus pass to 
cover transportation costs. 
Study Population and Recruitment 
 
Approximately 5000 participants will be enrolled in this study throughout the United States. 
Participation will be limited to adults over the age of 18. Participants who would be unable to respond 
to interviewer questions in English or Spanish will be excluded from participation. For interviews that 
are conducted over video conferencing, participants must use a device that allows their camera to be on 
throughout the interview. Participants who are unable to join via video at the start of the interview will 
be asked to reschedule to another time when they have a camera available. To ensure a representative 
sample, we plan to recruit up to 2500 participants with less than a college education and up to 2500 
participants with a college education or more. Of these participants, we aim to recruit an equal number 
of parents and non-parents.  
A variety of recruitment channels will be pursued in order to enroll a diverse sample. Fliers will 
be placed around the University of Chicago Medical Center campus as well as in various community 
locations, such as grocery stores, laundromats, and other public areas. Additionally, the researchers will 
reach out to existing contacts via phone calls or emails, and rely on word of mouth referrals to expand 
our reach. The research team will work with the University of Chicago Center for Decision Research 
(CDR), who maintains a recruitment listserv of individuals interested in being contacted for research 
opportunities as well runs a walk-in survey center. This center connects interested participants with 
available survey and interview-based studies. CDR staff will post information about the study on their 
website, include information in their email newsletters, and offer the study to individuals who come to 
the walk-in center. CDR staff will not consent participants or administer any study procedures. 
 
Interested participants will be asked to share their first name and phone number and/or email 
address via a REDCap survey so that the research staff can schedule and confirm their interview time 
with them. This contact information will be kept separate from their interview responses. They will also 
be asked to complete a short screening questionnaire in REDCap providing basic demographics 
information. This will allow the researchers to know basic characteristics about gender, education level, 
and parent status in advance of the first interview session, which is important in selecting what items 
that participant will be asked to review. Because not all participants will have time to see each item that 
is under development, asking these screening questions is necessary to ensure that a representative 
sample sees each interview item. Participants will not be excluded from participation based on their 
responses to the screening questions except for those items that specifically ask about the exclusion 
criteria listed above. This survey will also capture the user’s IP address, using a built in functionality in 
REDCap. We will use this information as an additional method of detecting potential duplicates in the 
screener results. Participants will be informed in writing at the start of the screener that by completing 
this survey, their IP address will be stored. 
After completing the screening questionnaire, participants will be giving instructions on how to 
sign-up for an interview timeslot using calendly.com. Calendly (which has received approval to operate 
from BSD ISO) is a platform that will allow the SPEAK CAT researchers to post their availability for 
interviews. Participants can then sign up for a time that works best for them. Time slots that are claimed 
will be automatically removed from the calendar; thereby, streamlining the sign up process. Interested 
participants will receive an email from the research team with a link to the Calendly page and 

instructions on how to sign up for an interview timeslot. This will include entering their name and email 
address. Once they have selected a timeslot, they will receive a confirmation email with information 
about their scheduled session from the research teams BSD email address.  Calendly will be also be used 
in the same manner for participants who have given us permission to store their contact information 
after the first study session and have expressed interest in being contacted for future interviews, as 
described in the Procedures section above.  
In both instances, participants will be given the alternative option to schedule interview sessions 
with the research team directly, via email or phone conversation. While we will encourage participants 
to use Calendly for ease of scheduling, maintaining this option will allow individuals to participate even if 
they are not comfortable sharing their information to the external website of Calendly. 
Potential Risks 
Anticipated risks are minimal. Some participants may feel discomfort describing their opinions 
or reactions to survey questions. To mitigate this risk, we will remind participants that our questions are 
optional and are designed to test the survey items themselves, not the participant. Interviewers will be 
trained on cognitive interviewing techniques to limit the potential discomfort. Because the sessions will 
be audio/video recorded, there is some risk of privacy loss. However, all data will be saved on secure 
BSD servers and only approved research personnel will have access. Names and other identifying 
information will not be associated with the recordings. 
Potential Benefits 
We do not expect participants to benefit directly from their participation in this research. There may be 
a benefit to society because parents and providers can find ways to increase their knowledge and 
encourage positive behaviors to support a child’s cognitive development using the resulting SPEAK CAT 
tool. 
Data Storage and Usage 
 
Survey data, including SPEAK questions answered as part of the interviews, the Family Supports 
and Beliefs Survey, and the demographics questionnaire, will be collected on paper or electronically, via 
REDCap. If data is collected on paper, a research assistant will enter the data into the REDCap project 
and then dispose of the paper copy. Survey data responses will be stored indefinitely, and will contain 
no identifying information. Audio/video recordings of the interview sessions will be saved under an ID 
number on BSD servers and will only accessible by members of the TMW Center team. The TMW Center 
will primarily use these recordings to qualitatively analyze interview responses. These recordings will 
also be used to train a speech recognition algorithm that TMW Center researchers are building. The 
algorithm takes audio/video clips as an input, and conducts speaker identification i.e. predicting 
whether the speaker is male or female or child etc. and runs analyses to track conversational turns 
between speakers. Members of the TMW Center team may listen to segments of the audio/video files 
to validate accuracy of the algorithm.  
Audio/video files will be deleted within 14 years after the date they are collected. These recordings may 
be coded or transcribed. Transcriptions of the audio/video file and any resulting coded data will be kept 
indefinitely. Transcripts will be de-identified and not able to be linked back to a specific individual. 

 
Consent 
 
At the start of the first interviewing session, the interviewer (who will be a member of research 
team, as listed on the study protocol) will read a consent script aloud to the participant, describing the 
study procedures and explaining the risks and benefits of participating. The interviewer will remind the 
participant that the interview is voluntary and that they may discontinue at any time. Participants will be 
asked to verbally acknowledge whether they would like to continue with the interview. We request a 
waiver of documentation of consent, because the study is minimal risk and the consent form would be 
the only document linking the participant’s name to their study data. 
 
If a participant chooses to complete more than one interview session, they will not complete the 
verbal consent process at each session. The interviewer will remind them of key details at the start of 
each session and let them know about any changes to procedures or risks/benefits that may have 
occurred since their last meeting. 
Statistical and Publication Plan 
 
Qualitative results from the cognitive interviews will be used by the TMW Center to inform 
SPEAK item development and refinement. Non-identifiable quotes from the interviews may be used in 
publications or presentations about the SPEAK survey. Responses to SPEAK and demographic data may 
be used in aggregate to validate the SPEAK measure. Responses to the Family Supports and Beliefs 
Survey will be used in aggregate to study people’s beliefs about the role of family supports for the 
success of children and the relation between these beliefs and stress and self-efficacy perceptions. TMW 
Center may publish results in scientific journals. No individually identifying information will be published 
or used in presentations without separate written consent from the participants.  
References 
Bruner, J. S. (1981). The social context of language acquisition. Language & Communication, 1, 
155–178 
 
Chapman, R. (2000). Children’s language learning: An interactionist perspective. Journal of 
Child Psychology and Psychiatry, 41, 33–54 
 
Gallaway, C. & Richards, B. J. (1994). Input and interaction in language acquisition. Cambridge, 
England: Cambridge University Press 
 
Hart, B., & Risley, T. R. (1995). Meaningful differences in the everyday experience of young 
American children.  Baltimore, MD: Brookes 
 
Hart, B., & Risley, T. R. (1992). American parenting of language learning children: Persisting 
differences in family child interaction observed in natural home environments. Developmental 
Pyschology, 28, 1096–1105 
 

Hoff, E. (2003), The specificity of environmental influence: Socioeconomic status affects early 
vocabulary development via maternal speech, Child Development, September/October 2003, 
74(5), 1368-1678 
 
Huttenlocher, J., Haight, W., Bryk, A., Selzer, M., & Lyons, T. (1991). Early vocabulary growth: 
Relation to language input and gender. Development Psychology, 27, 236–248 
 
Leung, C. Y.Y., Hernandez, M., Suskind, D.L., Enriching home language environment among 
families from low-SES backgrounds: A randomized controlled trial of a home visiting 
curriculum, Early Childhood Research Quarterly, 2018 
 
Leung, C.Y.Y. & Suskind, D.L., What parents know matters: Parental knowledge at birth 
predicts caregiving behaviors at 9 months, Journal of Pediatrics, 2020. 
 
O'Neil-Pirozzi, T. M. (2009). Feasibility and benefit of parent participation in a program 
emphasizing preschool child language development while homeless. American Journal of 
Speech-Language Pathology, 18(3), 252-263 
 
Rowe, R. L. (2008). Child-directed speech: relation to socioeconomic status, knowledge of child 
development and child vocabulary skill. Journal of Child Language, 35, 185–205 
 
Suskind, D. L., Y., C. Y., Webber, R. J., Hundertmark, A. C., Leffel, K. R., Fuenmayor Rivas, I. 
E., & Grobman, W. A. (2018). Educating Parents About Infant Language Development: A 
Randomized Controlled Trial. Clinical Pediatrics, 57(8), 945–953 
 
Suskind, D., Leffel, K., Graf, E., Hernandez, M., Gunderson, E., Sapolich, S., Suskind, E., 
Leininger, L. Goldin-Meadow, S., Levine, S. C. (2016). A parent-directed language intervention 
for children of low socioeconomic status: A randomized controlled pilot study. Journal of Child 
Language, 43(2), 366-406 
 
Suskind, D., Leffel, K. R., Hernandez, M. W., Sapolich, S. G., Suskind, E., Kirkham, E., & 
Meehan, P. (2013) An exploratory study of “Quantitative Linguistic Feedback”: Effect of LENA 
Feedback on Adult Language Production. Communication Disorders Quarterly. 34(4) 199-209