Lessons Learned: Adapting a Health Impact Assessment Tool Using Cognitive Interviews
Creators: name: Roberts, Rossana
name: Perreira, Krista
name: Maras, Shelly
name: Frerichs, Leah
name: Kaufmann, Rachel
name: Carr, Tara
name: Yusuf, Bola 
name: Snipes, Abisola 
name: Watkins, Keiana
name: Veldman, Tim
Publisher: name: The University of North Carolina at Chapel Hill University Libraries
Year: 2024

description: <div class="OutlineElement Ltr SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; clear: both; cursor: text; overflow: visible; position: relative; direction: ltr; font-family: 'Segoe UI', 'Segoe UI Web', Arial, Verdana, sans-serif; font-size: 12px; background-color: #ffffff;">
<p class="Paragraph SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px -2px 10.6667px; padding: 0px; user-select: text; overflow-wrap: break-word; white-space-collapse: preserve; vertical-align: baseline; font-kerning: none; background-color: transparent; color: windowtext;"><span class="TextRun SCXW179010292 BCX0" lang="EN-US" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; font-variant-ligatures: none !important; font-size: 12pt; line-height: 20.5042px; font-family: 'Times New Roman', 'Times New Roman_EmbeddedFont', 'Times New Roman_MSFontService', serif;" xml:lang="EN-US" data-contrast="auto"><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;">Introduction: The National Institutes of Health (NIH) supported the Rapid Acceleration of Diagnostics-Underserved Populations (</span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;">RADx</span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;">-UP) program, which aims to ensure that all Americans have access to COVID-19 testing and promotes health equity, especially in underserved communities. Under </span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;">RADx</span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;">-UP, there are two pilot programs &mdash; Community Collaboration Grants (C2G) and Rapid Research Pilot Program (RP2) &mdash; that fund projects to conduct community-engaged outreach and research to address COVID-19 testing disparities. Our evaluation team used cognitive interviews with project representatives to revise an existing Health Impact Assessment tool to measure health equity </span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;">impacts</span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;"> within these community-centered programs. We highlight our process of adapting the original tool and share lessons learned that other evaluators may apply to their work.</span></span><span class="EOP SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; font-size: 12pt; line-height: 20.5042px; font-family: 'Times New Roman', 'Times New Roman_EmbeddedFont', 'Times New Roman_MSFontService', serif;" data-ccp-props="{&quot;335559685&quot;:-20,&quot;335559737&quot;:-20}">&nbsp;</span></p>
</div>
<div class="OutlineElement Ltr SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; clear: both; cursor: text; overflow: visible; position: relative; direction: ltr; font-family: 'Segoe UI', 'Segoe UI Web', Arial, Verdana, sans-serif; font-size: 12px; background-color: #ffffff;">
<p class="Paragraph SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px -2px 10.6667px; padding: 0px; user-select: text; overflow-wrap: break-word; white-space-collapse: preserve; vertical-align: baseline; font-kerning: none; background-color: transparent; color: windowtext;"><span class="TextRun SCXW179010292 BCX0" lang="EN-US" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; font-variant-ligatures: none !important; font-size: 12pt; line-height: 20.5042px; font-family: 'Times New Roman', 'Times New Roman_EmbeddedFont', 'Times New Roman_MSFontService', serif;" xml:lang="EN-US" data-contrast="auto"><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;">Methods: From August to September 2023, we conducted online cognitive interviews among</span><span class="NormalTextRun CommentStart SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;"> six representatives purposively sampled from 3 C2G and 3 RP2 projects.</span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;"> We developed a semi-structured interview guide to assess participants' opinions about an equity measurement tool developed by Heller et al. (2014). During interviews, we asked participants to use a think-aloud approach to elicit feedback related to comprehension, clarity, structure, and difficulty level on the items and response options within the original tool. A team of three coders deductively developed a codebook and analyzed data using </span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;">Dedoose</span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;">, a mixed-methods web application. After analysis, we revised the original tool based on the summarized feedback. We re-engaged participants to provide </span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;">additional</span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;"> feedback on how the tool improved from the original to the revised version. Once </span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;">finalized</span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;">, the </span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;">final version</span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;"> will be administered to assess the pilot programs&rsquo; health equity impacts.</span></span><span class="EOP SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; font-size: 12pt; line-height: 20.5042px; font-family: 'Times New Roman', 'Times New Roman_EmbeddedFont', 'Times New Roman_MSFontService', serif;" data-ccp-props="{&quot;335559685&quot;:-20,&quot;335559737&quot;:-20}">&nbsp;</span></p>
</div>
<div class="OutlineElement Ltr SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; clear: both; cursor: text; overflow: visible; position: relative; direction: ltr; font-family: 'Segoe UI', 'Segoe UI Web', Arial, Verdana, sans-serif; font-size: 12px; background-color: #ffffff;">
<p class="Paragraph SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px -2px 10.6667px; padding: 0px; user-select: text; overflow-wrap: break-word; white-space-collapse: preserve; vertical-align: baseline; font-kerning: none; background-color: transparent; color: windowtext;"><span class="TextRun SCXW179010292 BCX0" lang="EN-US" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; font-variant-ligatures: none !important; font-size: 12pt; line-height: 20.5042px; font-family: 'Times New Roman', 'Times New Roman_EmbeddedFont', 'Times New Roman_MSFontService', serif;" xml:lang="EN-US" data-contrast="auto"><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;">Result: All participants provided feedback on the tool </span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;">indicating</span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;"> that we should make changes to improve the tool&rsquo;s suitability for measuring pilot projects&rsquo; health equity impacts. For example, participants </span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;">indicated</span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;"> that we needed to improve the clarity and readability of key terms and phrases and provided suggestions for how to tailor the survey to community partners. Participants also recommended we change statements to questions and make questions and response options concise and mutually exclusive, reducing the tool length and response burden.&nbsp;</span></span><span class="EOP SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; font-size: 12pt; line-height: 20.5042px; font-family: 'Times New Roman', 'Times New Roman_EmbeddedFont', 'Times New Roman_MSFontService', serif;" data-ccp-props="{&quot;335559685&quot;:-20,&quot;335559737&quot;:-20}">&nbsp;</span></p>
</div>
<div class="OutlineElement Ltr SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; clear: both; cursor: text; overflow: visible; position: relative; direction: ltr; font-family: 'Segoe UI', 'Segoe UI Web', Arial, Verdana, sans-serif; font-size: 12px; background-color: #ffffff;">
<p class="Paragraph SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px -2px 10.6667px; padding: 0px; user-select: text; overflow-wrap: break-word; white-space-collapse: preserve; vertical-align: baseline; font-kerning: none; background-color: transparent; color: windowtext;"><span class="TextRun SCXW179010292 BCX0" lang="EN-US" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; font-variant-ligatures: none !important; font-size: 12pt; line-height: 20.5042px; font-family: 'Times New Roman', 'Times New Roman_EmbeddedFont', 'Times New Roman_MSFontService', serif;" xml:lang="EN-US" data-contrast="auto"><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;">Significance</span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;">:</span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;"> This </span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;">project</span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;"> highlights the importance of cognitive interviewing as a </span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;">critical </span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;">methodology</span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;"> </span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;">to:</span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;"> </span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;">1)</span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;"> tailor a</span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;">n existing, </span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;">pilot-tested</span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;"> </span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;">tool to a community-based audience</span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;">,</span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;"> and </span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;">2) </span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;">increas</span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;">e</span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;"> compr</span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;">ehension by</span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;"> improving the readability, applicability, and clarity of the existing tool.</span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;"> </span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;">We</span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;"> </span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;">adapted the tool for C2G and RP2 program participants</span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;">, bridging a gap in the </span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;">availability of tools to a</span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;">ssess health equity </span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;">impacts</span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;"> in community-based programs.</span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;"> Through the cognitive interview process, we realized the importance of prioritizing face validity and tailoring tools to target audiences. We also had to consider balancing trade-offs, such as time, staff </span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;">capacity</span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;">, relevance, and participant burden, between creating a new tool versus adapting an existing, published tool.</span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;"> Our process </span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;">and lessons learned</span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;"> can be used as starting points for other</span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;"> community-engaged research</span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;"> practitioners or evaluators looking </span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;">to adapt</span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;"> the </span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;">health equity</span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;"> impact assessment tool</span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;"> or others</span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;"> like it</span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;">.&nbsp;</span><span class="NormalTextRun SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;">&nbsp;</span></span><span class="EOP SCXW179010292 BCX0" style="-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; font-size: 12pt; line-height: 20.5042px; font-family: 'Times New Roman', 'Times New Roman_EmbeddedFont', 'Times New Roman_MSFontService', serif;" data-ccp-props="{&quot;335559685&quot;:-20,&quot;335559737&quot;:-20}"> </span></p>
</div>