Qualitative Methods in Rapid TurnAround Health Services Research 

VA HSR&D Cyberseminar

Spotlight on Women’s Health

Alison B. Hamilton, PhD, MPH

Research Health Scientist & Lead, Qualitative Methods Group

HSR&D Center for the Study of Healthcare Innovation, 

Implementation & Policy

VA Greater Los Angeles Healthcare System

December 11, 2013
Thank you

Dr. Ray Maietta

Dr. Barbara Bokhour

Dr. Susan Zickmund

My VA qualitative research teams 

Participants from 2012 & 2013 ResearchTalk/UNC 

Qualitative Research Summer Intensives and 2013 

ResearchTalk Qualitative Data Analysis Camp
Poll question #1

How familiar are you with qualitative methods?









Very

Somewhat 

Not very

Not at all
Objectives







To provide a rationale for the tailoring of 

qualitative methods for rapid-cycle projects

To address research design and analysis 

strategies

To provide concrete suggestions for employing 

these strategies, using examples from a oneyear Women’s Health Services-funded project
Why do we need a tailored approach 

for rapid projects?
What do we mean by “rapid”?

Rapid Assessment Process (RAP)

 “intensive, team-based qualitative inquiry using 

triangulation, iterative data analysis and 

additional data collection to quickly develop a 

preliminary understanding of a situation from the 

insider’s perspective” (Beebe 2001)

• Rapid = “minimum of four days...maximum of six weeks”

 Rapid projects in HSR = projects of one year or 

less 

 Rapid ≠ rushed!

Beebe, J. Rapid Assessment Process: An Introduction. AltaMira Press, 2001
Why rapid qualitative methods?





Most common critique of qualitative research is 

that it “takes too much time”

Health services research and implementation 

research increasingly rely on qualitative methods 

• Constricted timeframe

• Frequent demand for products 

• High expectation of rigor
When might you need a rapid approach?









Specific timeframe

• Funding is for a year

• Deliverables are due on a certain date

Need for products/progress, e.g. 

• Competition/pressure to publish

• Need data for preliminary studies section of a proposal

• Need to provide partners (operations, community, etc.) with rapid 

feedback 

Use of qualitative data for other aspects of study/project

• Inform each phase of data collection 

• Need to make real-time modifications to an implementation 

strategy 

• Need qualitative data to inform quantitative measures/instruments

• Need to understand unexpected discoveries/findings

Striking while iron is hot (time-sensitive issues/developments)
Traditional (not so rapid) vs. rapid qualitative methods

Traditional

May be more constructivist

--More exploratory, inductive

Continuous data collection

Long-term engagement in setting, with 

participants

Descriptive, broad-based, and 

interpretive

Data analysis occurs after data 

collection

May not be compatible with a mixed 

methods study (time constraints)

Rapid

May be more positivist

--More explanatory, deductive

Multiple time point or punctuated data 

collection: analyses inform each other

Rapid, often minimal and time-limited 

engagement

Initially specific/targeted and often 

explanatory; interpretive later 

Data analysis occurs during data 

collection

May be well-suited for a mixed 

methods study
What is unique about rapid qualitative research?

 Approach is “telescoped” and action-oriented 

 A pragmatic need for qualitative data exists, e.g., to describe: 

• The environment where an intervention will be implemented

• The process that occurs while the intervention is underway

• “Usual” care, services, practices

 Typically and preferably conducted by teams

 Typically need to draw data quickly from multiple sources; 

often triangulate with quantitative data

 Potentially less time to critique, reflect, synthesize
How can we tailor qualitative methods for 

rapid-cycle projects?
Designing a rapid qualitative study

Why rapid?

















What are key research questions/specific aims?

What guides your rapid study (theoretical/conceptual framework)?

What will be your sources of data (i.e., what data will you collect, 

from whom)?

When will you collect data (when in project, how often, logic 

behind timing)?

Who will collect data (training of team, size of team)?

How will you analyze the data (team-based approach, approach 

to data, timeframe for analysis)?

Who will receive your results, when, and how?

How will you tell the story/stories of your data?
Designing a rapid qualitative study: 

matching up the pieces

Specific, targeted questions/aims

+ 

Prepared team

+ 

Feasible data collection & analysis

+ 

Specific, targeted products

TIME

TIME

TIME

TIME
Choosing your qualitative methods

Unstructured Structured

• Focus groups Semi-structured

• Could use activities

• Semi-structured interviews

• Could contain rating/ranking questions

• Could limit sample to key informants, key stakeholders (e.g., purposeful 

sampling)

• Observations

• Could use templates
Example: VA Women’s Health Services Telehealth

Project (FY12)

Timeframe: <One year total, including IRB submissions, 

formation of team, site visits, analysis

Aim: To investigate VA women’s health telehealth efforts in order 

to inform next steps with these services

Team: Nine individuals with varying levels of qualitative methods 

experience (some with none)

--Interview leads + observers

Data collection: In-person or telephone semi-structured 

interviews with key stakeholders at selected Women’s Health 

Practice-Based Research Network (PBRN) sites across the US

Priority products: Final report for WHS; presentations to inform 

Women’s Health CREATE
VA WHS Telehealth Project: interview guide





Semi-structured, brief, prioritized, targeted, flexible

Example question: What kinds of services are available to women 

Veterans at this clinic? 

• Is PACT in place in the women’s clinic? If so, can you describe how or 

whether PACT differs from how primary care was delivered prior to 

PACT? Have any adjustments been made to meet women’s needs within 

the PACT model? If so, can you describe those adjustments?

• Are any services integrated, such as primary care and mental health? 

Can you describe how that works?

• Are you aware of any services that are available via telehealth? There 

is interest at Central Office in developing more telehealth-delivered 

services, such as tele-gynecology. Do you have any thoughts on that 

idea?
How can we analyze qualitative data rapidly?
Rapid data analysis: some considerations

 Rapid analysis may need to be supported by 

individuals with limited/no qualitative methods 

background

 Data analysis NOT limited to coding

• Rapid data analysis necessitates systematic approaches 

other than coding 

 Data reduction is needed to turn preliminary analyses 

around quickly

 Rapid data analysis does not preclude future, more 

time-intensive, “formal” analysis (e.g., inductive 

coding)
Rapid data analysis: reducing the data

 “Data reduction is not something separate from 

analysis. It is part of analysis. The researcher’s 

decisions—which data chunks to code and which to 

pull out, which evolving story to tell—are all analytic 

choices. Data reduction is a form of analysis that 

sharpens, sorts, focuses, discards, and organizes data 

in such a way that “final” conclusions can be drawn 

and verified.” (Miles & Huberman, Qualitative Data Analysis: 

An Expanded Sourcebook, 1994, p. 11)

 Remember to “keep the words,” don’t strip the data from the 

context in which they occurred
Rapid analysis steps at a glance

Step 1: Create a neutral domain name that corresponds with each 

interview question 

Step 2: Create a summary template for use by the team

Step 3: Take the summary template for a “test drive” and assess 

its usability, relevance, etc.

Step 4: After consistency has been established across the team of 

summarizers, divide up the transcripts across the team and 

summarize

Step 5 : Transfer summaries into a matrix (respondent x domain)

***Tailor this process to meet your team’s needs/styles 

and the goals of your project***
How to reduce the data: templated summaries 

 Initially develop a templated summary of each 

data collection episode, according to a relatively 

small set of pre-determined domains (with space 

for the unexpected)

• Domains should mostly line up with interview guide
Sample Interview Question

Are you aware of any services that are 

available to women via telehealth? 

Is PACT in place in the women’s clinic? Have 

any adjustments been made to meet 

women’s needs within the PACT model? If 

so, can you describe those adjustments?

Are any services integrated, such as 

primary care and mental health? Can you 

describe how that works?

Domain

Telehealth services/telegynecology 

PACT

Integrated services, e.g., PC-MHI

Steps for creating a templated summary

Step 1: Create a neutral domain name that corresponds 

with each interview question 
Steps for creating a templated summary (cont.)

Step 2: Draft a summary template for use by the team

TRANSCRIPT SUMMARY

PREPARED BY: Alison

SITE: abc

RESPONDENT ROLE: PCP 

TELEHEALTH/TELE-GYN

PACT/WH-PACT

INTEGRATED SERVICES, e.g., PRIMARY CARE-MENTAL HEALTH
Steps for creating a templated summary (cont.)

Step 2 continued:

–

–

Include “Other observations” at the end, for 

material that doesn’t fit into the domain

Include space at end for important quotations
Steps for creating a templated summary (cont.)

Step 3: Take the summary template for a “test drive”





Have team members use the template for the same subset of 

transcripts

Assess template:

• Are the domains intuitive/ “findable” in the data?

• Are any domains missing, incorrectly labeled, etc.?

• Is it easy to use?

• How long does it take to complete it?

• Should take about an hour to complete
Steps for creating a templated summary (cont.)

Step 3 continued:

 Compare summarizing “styles” across the team

 Assess for:

• Similarities/differences in volume of information per domain

• Use of direct quotes (should be minimal)

• Notes regarding absence of content

• Recommend noting “question wasn’t asked” or “question 

was asked but not answered”

• Notes regarding depth on a particular domain

• Recommend noting “this interview has a lot of data on this 

topic,” “great quotes in this transcript,” etc.
Steps for creating a templated summary (cont.)

Step 4: After consistency has been established across the 

team of summarizers, divide up the transcripts/data across 

the team and summarize; could divide up by site, by role, 

etc.

TRANSCRIPT SUMMARY

PREPARED BY: Alison

SITE: abc

RESPONDENT ROLE: PCP 

TELEHEALTH/TELE-GYN

•Strong telehealth bcs large rural pt pop

•Telegyn is “feasible” bcs peripheral devices can be attached to global media carts

PACT/WH-PACT

•Happy with PACT, but “stressful for my RN”

•Understaffed (LVNs)

•Need to start implem 10% panel size reduction for WVs

•60 mins for WV appts will be a “tough one”
Steps for creating a templated summary (cont.)

What makes for a good summary?











Brief (no more than 2 pages)

Organized

Thorough (major points captured)

Readable 

* Anyone reading the summary should get a sense of 

what the respondent said

Useful (e.g., provides pointers for what’s in the 

transcript)
Instructions for preparing transcript summaries (handout)

Preparing Transcript Summaries

 

1. The summary heading should have the name of the lead interviewer and any other interviewers in the room, if 

possible. The date of the data collection episode should also be included. Please record the name of the person 

preparing the summary.

2. It is most useful to stick to keep the domains in order to easily move the information into a matrix (i.e., don’t change 

the structure of the template).

3. Information not relevant to the pre-set domains should be included under “Other,” or you can create your own 

domain if something is coming up consistently. The latter should be communicated to the team.

4. Quotes are often times better if they are concise. Or, you can paraphrase and include key quotes at the bottom of 

the summary (and put “see quotation below” next to paraphrase).

5. Paraphrasing should be used for complicated/long answers.

6. This is a minimally interpretive process; remember you are trying to generate bullet points about the key domains 

such that anyone reading a summary would get a general sense of what was discussed. 

7. If there is no information for a given domain, indicate why: was the question asked but not answered? Was the 

question asked and the person responded something along the lines of, “I don’t know.” Or was the question not 

asked? Documenting the absence of data is important for the assessment of data collection consistency.

8. This should only take about an hour for a 45-60 minute interview transcript. If you are spending hours on one 

summary, you are probably thinking too much, interpreting, OR the template is not working as planned. Please let 

the team know if summarizing is taking a long time.
Displaying your data using the summaries

Step 5: Transfer (copy & paste) summary points into a 

matrix (e.g., respondent x domain)

“Matrices streamline the process of noting simultaneously and 

systematically similarities, differences, and trends in responses across 

groups of informants” (Averill 2002, p. 856)

• They make the “synthesis and summary of important findings 

accessible to audiences who might otherwise never take the 

time to examine the voluminous data generated by the 

interview process, domain analysis, and thematic analysis” (p. 

864)

Averill JB. Matrix analysis as a complementary analytic strategy in qualitative inquiry. Qual Health 

Res. 2002 Jul;12(6):855-66. 
Data display (cont.)

 Displays are “designed to assemble organized 

information into an immediately accessible compact 

form so that the analyst can see what is happening 

and either draw justified conclusions or move on to the 

next step of analysis...”

 “...the creation and use of displays is not separate 

from analysis, it is a part of analysis.” 

(Miles & Huberman, Qualitative Data Analysis: An Expanded Sourcebook, 1994, p. 11)
Data display (cont.)

 Set up matrix in format that makes sense for 

purpose of analysis (e.g., by site, by role, by wave 

of data collection, etc.
TELEHEALTH/TELEGYN PACT/WH PACT INTEGRATED 

SERVICES

SITE 1 [Could summarize this 

domain here]

Interview 1 •Strong telehealth bcs

large rural pt pop 

•Telegyn is “feasible” bcs

peripheral devices can be 

attached to global media 

carts

•Happy with PACT, but 

“stressful for my RN”

•Understaffed (LVNs) 

•Need to start implem 

10% panel size reduction 

for WVs

•60 mins for WV appts will 

be a “tough one”

•Made space in WH 

for MH providers

•Looking forward to 

psychol coming on 

board in WH soon

Interview 2 •Telegyn “could be done” 

and could be a 

“meaningful thing,” but it 

would still require review 

of pt chart to make sure 

it’s appropriate for the pt

•MH should be “seen as 

inextricably part of that 

PACT”

•“Having a gender-specific 

PACT is certainly a good 

idea”

•Have care 

management and colocated care

•PC-MHI “still needs to 

ramp up”
Matrix analysis: what can the matrix do for you?

With the matrix, you can:

 Quickly peruse content of any given domain

• Get a sense of variation

 Assess gaps in information

• Assess why those gaps exist: Question not asked? 

Question didn’t work well? 

 Develop memos (e.g., what themes are you noticing?)

 Develop summaries of domains, sites, types of 

respondents, etc.
What can you do with your rapid analysis?













Divide up the labor of reviewing transcripts

• Can be done by individuals who don’t have qualitative 

training

Assess quality of data collection across team

Obtain a quick understanding of the major findings

• Especially important if you did not collect all of the data

Use summaries to inform subsequent waves of data collection

Prepare reports/presentations

Develop codebook that is informed by depth and breadth of 

data related to each domain
PACT









PACT mostly happening at VAMC level

Teamlet composition varies

• “Hybrid” teams with >1 PCPs (part-timers)

• Structure of care for women as a minority—a lot of people practicing 

part time in the WH setting

• Residents pose a problem for continuity measures

• Perception that sharing staff doesn’t work

• Difficult for specialty services to meet staffing requirements

Questions/concerns about impact of PACT on PC-MH integration

Importance of pharmacist involvement

• At some sites, pharmacist goes to women’s team meetings 

• Pharmacists critical for reviewing women’s meds

Sample slide from presentation (based on matrix)
Poll question #2

Are you using a similar approach to what’s been 

described during this cyberseminar? 









Yes

I think so

No, but I plan to try it

No, I don’t think it will work for my projects
Hallmarks of credible (rapid) QDA

 Prolonged SYSTEMATIC engagement with the data

 Presentation of clear evidence grounded in the data

 Cross-cutting themes 

• or when theme is rare, clear rationale for inclusion as 

critical

 Team-based approach with discussion and consensus 

on themes and preliminary conclusions
Ms. Ismelda Canelo

Dr. Ann Chou

Dr. Kristina Cordasco

Dr. Alison Hamilton

Dr. Jodie Katon

Dr. Ruth Klap

Dr. Sabine Oishi

Dr. Danielle Rose

Dr. Jessica Zuchowski

With ongoing support from Dr. Elizabeth Yano, Ms. Britney Chow, 

and Ms. Jennifer Peralta

The WHS FY12 Telehealth Project Team
Funding for the Women’s Health Services Telegynecology Project 

generously provided by VACO Women’s Health Services

Special thanks to:

Drs. Patty Hayes and Sally Haskell for supporting our efforts

Dr. Susan Frayne for facilitating connections with WH PBRN sites

Respondents from the participating PBRN sites

Acknowledgements
Some helpful references 

Averill JB. Matrix analysis as a complementary analytic strategy in qualitative inquiry. Qual Health Res. 2002 Jul;12(6):855-66.

Bokhour BG, Solomon JL, Knapp H, Asch SM, Gifford AL. Barriers and facilitators to routine HIV testing in VA primary care. J Gen Intern 

Med. 2009 Oct;24(10):1109-14. 

Brown DR, Hernández A, Saint-Jean G, Evans S, Tafari I, Brewster LG, Celestin MJ, Gómez-Estefan C, Regalado F, Akal S, Nierenberg B, 

Kauschinger ED, Schwartz R, Page JB. A participatory action research pilot study of urban health disparities using rapid assessment 

response and evaluation. Am J Public Health. 2008 Jan;98(1):28-38.

Burks DJ, Robbins R, Durtschi JP. American Indian gay, bisexual and two-spirit men: a rapid assessment of HIV/AIDS risk factors, barriers to 

prevention and culturally-sensitive intervention. Cult Health Sex. 2011 Mar;13(3):283-98.

Devers KJ. How will we know "good" qualitative research when we see it? Beginning the dialogue in health services research. Health Serv

Res. 1999 Dec;34(5 Pt 2):1153-88. 

McMullen CK, Ash JS, Sittig DF, Bunce A, Guappone K, Dykstra R, Carpenter J, Richardson J, Wright A. Rapid assessment of clinical 

information systems in the healthcare setting: an efficient method for time-pressed evaluation. Methods Inf Med. 2011;50(4):299-

307.

Palinkas LA, Horwitz SM, Green CA, Wisdom JP, Duan N, Hoagwood K. Purposeful sampling for qualitative data collection and analysis in 

mixed method implementation research. Adm Policy Ment Health. 2013 Nov 6..

Patton MQ. Enhancing the quality and credibility of qualitative analysis. Health Serv Res. 1999 Dec;34(5 Pt 2):1189-208.

Sobo EJ, Billman G, Lim L, Murdock JW, Romero E, Donoghue D, Roberts W, Kurtin PS. A rapid interview protocol supporting patientcentered quality improvement: hearing the parent's voice in a pediatric cancer unit. Jt Comm J Qual Improv. 2002 Sep;28(9):498-

509.

Sobo EJ, Simmes DR, Landsverk JA, Kurtin PS. Rapid assessment with qualitative telephone interviews: lessons from an evaluation of 

California’s Healthy Families Program & Medi-Cal for Children. Am Journal Eval. 2003 Sep;24(3):399-408.

Solomon PL, Tennille JA, Lipsitt D, Plumb E, Metzger D, Blank MB. Rapid assessment of existing HIV prevention programming in a community 

mental health center. J Prev Interv Community. 2007;33(1-2):137-51.
Questions? 

Alison B. Hamilton

alison.hamilton@va.gov