Improving Satisfaction, Engagement and Clinical Outcomes Among Traditionally Underserved Children Through Cultural Formulation
Phase: NA
Conditions: Disruptive Behavior
Sponsor: Florida International University

Study Protocol with SAP 
 
 
Improving Satisfaction, Engagement and Outcomes Among Traditionally Underserved Children 
Through Cultural Formulation 
 
NCT03499600 
 
PI: Amanda Sanchez, PhD 
Project Mentor: Jonathan Comer, PhD 
 
Document Date: 8/10/22 
 
 
 

Objectives 
The present randomized examination evaluated the incremental utility of augmenting Clinical 
Assessment with the CFI among culturally diverse families seeking services for child-externalizing 
problems. Initial analyses examined equity-informed implementation outcomes – i.e., CFI fidelity, 
clinical utility, caregiver feeling understood and satisfaction – which can inform broader CFI use, 
sustainability, and ultimate effects on inequities (Eccles & Mittman, 2006; Proctor et al., 2011; 
Woodward et al., 2019). Given the tendency for families from marginalized back grounds to feel 
misunderstood by their provider, initial analyses examined caregiver and provider perceptions at 
post-assessment regarding the extent to which the provider understood the family’s values and 
what is important to them. Next, analyses examined the effects of the CFI on subsequent 
treatment engagement and ultimate clinical response. Treatment engagement was 
conceptualized through family behaviors (e.g., attendance and homework) and the family-
provider relationship (Becker et al., 2018). We first hypothesized that incorporating the CFI would 
be clinically useful, that providers and caregivers would report high satisfaction with the CFI, and 
that they would perceive the CFI as fostering improved provider understanding of the family’s 
values. We further hypothesized that caregivers randomized to receive the CFI as part of their 
child’s intake assessment, relative to caregivers randomized to standard Clinical Assessment 
(CA), would show stronger subsequent treatment engagement (i.e., higher subsequent treatment 
session attendance, homework completion, working alliance, and first treatment module 
completion), and would show higher rates of ultimate treatment response. Exploratory analyses 
further examined whether the language of service delivery (English versus Spanish) moderated 
outcomes. Families receiving services in Spanish, may be particularly vulnerable to poor 
treatment engagement and outcomes due to lower levels of acculturation, and potential 
misalignment of ment with their background and explanatory models (Alegría et al., 2015; Burnett-
Zeigler et al., 2018; Ho et al., 2007; La Roche, 2013; Yeh et al., 2005). Moreover, there is evidence 
that culturally adapted EBTs are more effective for families with lower levels of acculturation and 
those that receive services in their native language (Griner & Smith, 2006). 
 
Methods 
Participants 
Participants were 89 children ages 2–7 (M = 5.10, SD = 1.6), and their primary caregivers ages 
20–57 (M = 33.4, SD = 7.3) receiving services through a Miami- Dade County-funded initiative to 
provide free behavioral parenting intervention services to underserved communities (i.e., those 
with limited access to mental health services) and to assess various engagement strategies. The 
majority of primary caregivers were mothers (N = 85). The initiative serves English- and Spanish- 
speaking families of children with behavior problems (clinically elevated symptoms) and/or history 
of child maltreatment. Comorbid child disorders did not exclude eligibility, although caregivers 
actively abusing illegal substances were not eligible. There were no other exclusion criteria. 
Approximately two- thirds of the children were male and approximately 89% of the children were 
identified as a member of a racial and/or ethnic minority group by the caregiver. Sixty percent of 
participating caregivers did not have a college degree. Roughly half of the families (54.8%) had 
significant financial needs and fell in “extreme poverty,” “poor,” or “low-income” economic ranges 
(see Measures). Regarding housing security, roughly one-third of participating families. lived with 
more than two people per bedroom, and roughly one-fifth reported temporarily living with others 
due to economic difficulties. Nearly 33% of families received treatment and study activities in 
Spanish.  
 
Procedures 
This study was approved by the University of Miami and the Florida International University 
Institutional Review Boards. All services were provided in English or Spanish. Presenting families 
completed informed consent, then were randomized across two parallel pre-treatment 

assessment conditions: Clinical Assessment (CA) or CFI followed by CA (CFI+CA). A master 
random number table generator was used for randomization. Services were offered across three 
sites. The intake assessment included consent, an intake interview (CA or CFI+CA), completion 
of study questionnaires, and structured observations of parent–child interactions, generally over 
2–3 one-hour sessions. Families were not informed of the assessment condition to which they 
were randomized. After intake assessment, patients were offered a course of Parent-Child 
Interaction Therapy (PCIT), a well-supported behavioral parenting intervention for child behavior 
problems and maladaptive family patterns (Eyberg & Funderburk, 2011). A full PCIT course 
entails completion of two sequential treatment modules; the first module teaches caregivers to 
use positive and differential social attention to improve child behavior and the parent–child 
relationship, and the second module teaches caregivers consistent limit setting and effective 
discipline strategies. The precise length of a PCIT course varies as a function of the quickness 
with which parents acquire the skills and the child’s behavior falls within normal limits. The 
standard PCIT protocol was flexibly delivered across conditions. A power analysis assuming an 
alpha level = 0.05, beta = 0.2, and power = 0.8, identified a sample ≥82 participants was needed 
to identify small-to-medium effects (i.e., OR~2.5). Recruitment was closed within one month of 
reaching minimum sample size estimated by power analysis. Families were recruited and 
randomized from August 2017 to October 2018; data collection continued through March 2019.  
 
Study Conditions  
Clinical Assessment (CA)  
CA entailed the standard intake assessment procedures used in the county-funded behavioral 
parenting program. This included a parent interview regarding the child’s developmental 
milestones, educational history, medical history, disruptive behavior symptoms, previous 
treatment experiences, and current parenting strategies. A series of observational parent–child 
interaction tasks followed.  
 
CFI+CA  
Caregivers in CFI+CA participated in an augmented CA that also included the core CFI (CFI; 
American Psychiatric Association [APA], 2013). The CFI is a brief (16-item) semi-structured 
interview designed for providers to assess a patient’s cultural identity and how it might affect key 
aspects of their understanding of mental health difficulties and clinical care. The CFI assesses 
the patient’s individual symptom experience, their perceptions of mental health, and their 
perceptions and experiences of treatment via four domains: (1) cultural definition of the problem, 
(2) cultural perceptions of cause, context, and support, (3) cultural factors affecting self-coping 
and past help seeking, and (4) cultural factors affecting current help seeking (American 
Psychiatric Association [APA], 2013). See Table 2 for the domains of the CFI and the main 
component sections. For CFI+CA cases, providers administered the CFI prior to the CA, 
consistent with its implementation guidelines in DSM-5 (American Psychiatric Association [APA], 
2013). This decision to deliver the CFI first was made to remain consistent with previous studies 
(Aggarwal et al., 2020), enhance fidelity by having a clear implementation plan (Aggarwal et al., 
2014), center caregiver values and perspectives from the beginning of the intake (La Roche, 
2018), and increase rapport (Aggarwal et al., 2015, 2016b).  
 
Providers, Setting, and Training  
Providers in the current study who conducted all baseline clinical assessments and subsequent 
services (N = 4) were master’s- or doctoral-level community practitioners providing clinical 
services within three community clinics that were embedded within neighborhood centers 
regularly accessed by the community (i.e., afterschool care centers, family and neighborhood 
resource programs). The providers were employed by the medical center (i.e., not affiliated with 
the PI) to provide services within the neighborhood centers. Three out of the 4 providers had not 

delivered PCIT prior to the start of the study. They were trained by a certified trainer as part of the 
county-funded program to provide PCIT to underserved communities prior to delivering PCIT. 
These providers did not work for the PI or the study. The services were provided via a partnership 
between a medical center and the three neighborhood centers. Three of the providers were fluent 
in Spanish and delivered services in both English and Spanish. Providers conducted both CA and 
CFI+CA assessments. Randomization occurred at the patient rather than the provider level due 
to the providers working at three distinct community centers within distinct neighborhoods. 
Providers were masked to study hypotheses, but not study condition. For each family, the same 
provider conducted their intake assessment and their subsequent PCIT services. Providers and 
their supervisors at participating clinics completed a two-hour CFI training led by the study PI 
(who trained with one of the CFI developers) and another senior psychologist with CFI training 
experience. Per recommendations (Aggarwal et al., 2016a), training consisted of reviewing 
the CFI’s written guidelines, a video demonstration, role-plays with feedback, a discussion of 
possible barriers to implementation, and peer debriefing exercises to consider implementation 
issues in their organizational climate. To prevent integrity drift and contamination across 
conditions, providers received structured feedback based on fidelity ratings from their first case 
prior to seeing study cases (Aggarwal et al., 2014). Three additional CFI booster trainings were 
implemented to problem solve any challenges with implementing the CFI (e.g., wording of the 
questions, using examples to increase caregiver responses). 
 
Measures 
Patient forms were administered through a secure online platform, or by paper when families 
preferred. Measures were available in English and Spanish. 
 
Demographics 
Primary caregivers reported on child and caregiver age, race, and ethnicity. Additionally, specific 
details regarding country of origin and immigration status were not collected to protect the privacy 
of families during a time of considerable increase in Immigrations and Custom Enforcement raids 
in the community where this study occurred. Caregivers also reported on their education, annual 
household income, and family size. Incometo- needs (INR) ratios were computed reflecting 
household income relative to the poverty threshold, and categorized as “extreme poverty” (INR ≤ 
.5), “poor” (.5< INR ≤1), “low-income” (1< INR ≤2), “adequate-income” (2< INR ≤4), and “affluent” 
(INR > 4). To assess housing security, caregivers were asked two items from the U.S. Department 
of Housing and Urban Development: 1) During the past year, have you had more than 2 people 
per bedroom, and 2) In the past year, have you temporarily lived with other people because of 
economic difficulties? In addition, the Everyday Stressors Index (ESI; Hall, 1990), a 20-item 
measure of daily stressors experienced by economically disadvantaged caregivers with young 
children, was administered at intake. Items are rated on a Likert-style scale ranging from 1 = “not 
at all bothered” to 4 = “bothered a great deal,” as well as “I don’t know” = 0, and tallied with higher 
scores indicating greater daily stress (maximum score = 80). The ESI has demonstrated good 
internal consistency and construct validity in English- and Spanish-speaking samples (Hall et al., 
1996; Gómez et al., 2015; Hall, 1990) (α = .79 in present sample).  
 
Intake Assessment Length  
Providers recorded the total time, in minutes, spent completing each family’s intake assessment. 
The pretreatment assessment sessions included the consent procedures, and CA or CFI+CA 
(depending on randomization). Time spent across the intake assessment was measured by 
provider report of the total length, in minutes, to complete these procedures. The pretreatment 
assessment typically lasted between 2 and 3 sessions.  
 
Implementation Outcomes  

Provider fidelity and condition integrity. To measure CFI fidelity and potential cross-condition 
contamination in CA, the CFI Fidelity Instrument (CFI-FI; Aggarwal et al., 2014) was used to code 
assessments in both conditions. Independent evaluators (IEs) masked to condition, study design, 
and study hypotheses coded recordings of all assessments. IEs met 80% reliability on a study 
video prior to coding further study videos. Consistent with previous research (Aggarwal et al., 
2014), interrater reliability across IEs on a random 20% of study cases was high (i.e., agreement 
range: 80%–100% on Adherence items; mean agreement = 96.25%).  
 
Provider perceptions of clinical utility. The CFI Clinician Questionnaire is a 4-item measure 
adapted from previous work (Aggarwal et al., 2015) to assess provider perceptions of CFI clinical 
utility. Directly following pretreatment assessment, providers rated 4 items related to content and 
quality of information, relationship with caregiver, treatment planning, and differential diagnosis 
(e.g., “How much of an impact did using the CFI have on your relationship with your patient”) on 
a Likert-style scale ranging from 1 = “not at all” to 5 = “very much” (α in present sample =.84).  
 
Perceptions of provider understanding of caregiver values and satisfaction. Two items developed 
for this study, administered directly after the intake assessment (but before treatment began), 
measured caregiver and provider perceptions of how well the provider understood the family’s 
values and what is important to them, and how satisfied they were with the intake assessment. 
The items were rated independently by caregivers and providers on a 5-point scale (1 = not at all; 
5 = very much). The items were translated into Spanish and then back translated into English to 
ensure semantic and cultural accuracy.  
 
Treatment Engagement  
The behavioral domains of engagement (e.g., attendance and homework) were measured for 
each family via provider logs recording: (a) initial treatment session attendance (i.e., did family 
attend a first treatment session after completing intake assessment?), (b) weekly session 
attendance rate (percentage of sessions attended across weeks in treatment), (c) completion of 
first treatment module (i.e., caregiver acquisition of positive-attending skills), and (d) homework 
completion (i.e., mean percentage of weekly homework completed across weeks engaged in 
treatment). While the average PCIT treatment length is between 12 and 20 sessions, the length 
of treatment varies based on family’s specific needs, symptom reduction, and caregiver skill 
acquisition (PCIT International, n.d.).  
 
The relationship domain of engagement was measured by the Working Alliance Inventory-Short 
Form Revised (WAI-SR; Hatcher & Gillaspy, 2006), administered at midtreatment. The WAI-SR 
is a 12-item caregiver-report of therapeutic alliance that assesses (a) perceived agreement with 
the provider on the tasks of therapy, (b) perceived agreement with the provider on the goals of 
therapy, and (c) the perceived affective bond between the provider and patient/ family. Items are 
rated on a 5-point Likert-style scale ranging from 1 = “never” to 5 = “always” and are summed for 
a total score. The WAI-SR has demonstrated strong internal consistency and construct and 
convergent validity in English and Spanish (Andrade- Gónzalez & Fernández-Liria, 2016; Hatcher 
& Gillaspy, 2006; Munder et al., 2010) (α = .94 in the present sample).  
 
Treatment Response  
Families were classified as Treatment Responders in accordance with formal PCIT graduation 
criteria (Eyberg & Funderburk, 2011), that is: (a) caregivers demonstrated skill acquisition, as 
coded against specific criteria in both phases of treatment; and (b) child behavior problems 
dropped into the subclinical range – i.e., Eyberg Child Behavior Inventory (ECBI) falls below 114; 
Eyberg, 1999).  
 

Data Analysis  
Intent-to-treat analyses were employed. Missing data were addressed through Multiple Imputation 
in Mplus (Enders, 2010). The majority of variables were completed for more than 90% of cases. 
Descriptive analyses were conducted to assess the implementation outcomes of fidelity, clinical 
utility, provider and caregiver perceptions of understanding the caregivers’ values, and 
satisfaction with the intake. Additionally, linear regressions tested condition effects on caregiver 
and provider satisfaction, and the extent to which the provider understood the caregivers’ values 
or what is important to them. Next, logistic and linear regressions examined condition effects on 
subsequent treatment engagement and response. Exploratory moderation analyses examined 
the main and interactive effects of language of service delivery (English versus Spanish) with 
study condition (CA versus CFI+CA) on dependent variables. To interpret effects, adjusted odds 
ratios and probabilities were calculated for logistic regressions and Cohen’s d effect sizes were 
calculated for linear regressions. An alpha threshold of .25 determined inclusion of covariates. 
Due to condition differences all analyses controlled for length of intake assessment (in minutes), 
site, caregiver race/ethnicity, daily stress, and language of assessment delivery. Length of intake 
assessment is also controlled for to account for differences in effects solely related to differences 
in the amount of time the provider spent with the caregiver. Baseline ECBI score was included as 
a covariate for the treatment response analyses.