A New Tool for Assessing 

the Primary Care Experience for Homeless 

Patients, the PCQ-H

Stefan G. Kertesz, MD

Birmingham VA Medical Center & 

University of Alabama at Birmingham School of Medicine, Associate 

Professor

Support: VA Health Services Research & Development Branch
DISCLOSURE STATEMENT

Speaker: Stefan Kertesz, MD

Dr. Kertesz has documented that she (or he) has 

nothing to disclose. 

Opinions are those of the authors and do not represent 

positions or views of VA or any agency of the U.S. 

federal government
Acknowledgements

Birmingham: David Pollio, Nancy 

Johnson-Roe, Erika Austin, 

Jocelyn Steward, Unita Granstaff, 

Calvin Elam, John Andrew Young, 

Alison Borden

Tuscaloosa VAMC, Lori Davis

U. Maryland: Cheryl Holt

Johns Hopkins: David Roth

Harvard/Hebrew Sr. Life: Rich 

Jones

Boston HCH: James O’Connell, 

Erin Stringfellow, Rachel Marston

Boston Medical Center: Theresa 

Kim, Lori Henault

Pittsburgh VA: Adam Gordon, 

Dan Harding

VA Greater Los Angeles: Alex 

Young, Joya Golden
Aim

•To introduce a primary care experience 

questionnaire designed for homelessexperienced patient

•To show a systematic method for its 

development using

• qualitative interviews

• survey testing with patients

• psychometric selection of informative items
What is your primary professional 

category at present?

1. Social Worker

2. Physician/Nurse

3. Psychologist

4. Other direct clinical service

5. Health Care Manager

6. You neglected my category, bonehead!
Have you ever provided direct service to 

homeless clients or patients in your 

professional work?

• Yes

• No
Background

• >600,000 Americans homeless (57,849 veterans) 

in 2013 (point-in-time)

• “Tailored” primary care programs began in 1986

• 19 original HCH programs  220+ today

• 38+ similar VA teams (“Homeless PACTS”) today 

• >50 in coming year

• HHS allowing Federally Qualified Health Centers to 

seek “Patient-Centered Medical Home” designation

• All require patient experience questionnaires
What might service tailoring mean?

• Tailoring includes aspects of service design, outreach, colocation of mental and medical care, consumer 

governance, etc. 

Other aspects of tailoring

• Co-location medical care and behavioral health

• Panel size adjustment

• Homeless-training & mission, leadership/Board 

representation
Question

• If you thought you were delivering care in a better way, 

how would you prove it in a measurable way?

• Why does proving it matter?
Quality Metrics for Primary Care

• Traditional care indicators

• Disease-based, % with Hgb A1c< 8%, % with cancer screening

• Imperfect

• Re: context and preferences,1 complexity/comorbidity2,3

• Example

• Does failure to obtain colonoscopy count as poor care when the 

patient has no escort and no place to recover from the procedure?

1. Weiner, Ann Intern Med (2010). 2. Safford, JGIM (2007) 3. Boyd, JAMA (2005). 
Remedies

• Refine the measures1

• Assess whether decisions take context into account1 

• Survey “Through the Patient’s Eyes”3 

• Patient-reported surveys (PCAS4, CAHPS5) 

• CAHPS family surveys are central to both VA SHEP and HHS 

expectations for Patient-Centered Medical Homes

1. Weiner, Ann Intern Med; 2010 3. Gerteis et al, 1993 4. Safran, Medical 

Care (1998) 5. Agency for Healthcare Research and Quality
When service providers ask me to fill out a 

survey, my internal reaction is most like...

• 1. This might help improve services, so I’m going to fill it 

out

• 2. I have no time for this

• 3. I will fill this out if you promise me a prize

• 4. Wow, what poor survey design! Did you even test the 

psychometrics on this thing?

• 5. Hey Kertesz, you didn’t anticipate my reaction, 

bonehead! Maybe try qualitative first?
How comfortable are you with designing 

surveys and validating them

• Very comfortable

• Somewhat comfortable

• Uncomfortable\ 

• You didn’t anticipate my response!
Comments on “off the shelf” instruments

• CAHPS 12 Month PC + PCMH items

• 1012 words, 9th grade reading level (F-K)

• 12 of 43 items are used to institute skips

• 7 different response sets

• Never, Sometimes, Usually, Always

• Not at all, A little, Some, A Lot

• 0, 1, 2, 3, 4, 5...10

https://cahps.ahrq.gov/surveys-guidance/cg/pcmh/index.html
Examples from CAHPS PC + PCMH

Access? 

In the last 12 months, did you phone this provider’s office to 

get an appointment for an illness, injury or condition that 

needed care right away?

• Note, 2 items are skipped if no phone

• No outreach questions

• No concept of walk-in

Control? 

After an item related to talking about starting a medicine....

When you talked about starting or stopping a prescription 

medicine, how much did this provider talk about the reasons 

you might not want to take a medicine?”
Concerns with CAHPS PCMH

• The mandated industry standard

• Conceptually: a Patient Experience Questionnaire

• Emphasis on procedural/experiential reports from patients 

about their care

• Challenges:

• The procedures/experiences queried may not fit the patient

• Only indirectly related to asking patients what matters to them

• Never tested for feasibility in homeless PC

• Proven difficult in HCH settings
Approach: Top Down + Bottom Up

• Start with general constructs from a respected 

source (similar to CAHPS, PCAS, PCAT) to get 

general constructs

• Let people familiar with homeless primary care 

(patients, expert caregivers 

• Prioritize the constructs

• Add new constructs

• Define the actual themes we ask about
16 Major Constructs Derived from 

2 Respected Reports

Rules for Quality Attributes of Primary Care
Keep in mind safety

SAFETY

Mean all of those who take 

care of a patient work as a 

team and talk to each other

COOPERATION 

Be easy to get

ACCESSIBILITY

Using a Cardsort

Primary care for homeless patients should...

Responses from 26 patients 

and 10 providers/experts from 

multiple states and regions of 

the US 
Accountability

Patient Control

Information Flow

Continuity

EvidenceBased

Cooperation 

among 

Caregivers

Access 

Coordination

• What do you think about the idea that you should have 

control in your primary care? 

• What makes you say that?

• How about times when you didn’t have a regular place to live? 

Constructs prioritized for interview-based study
Interviews

• 36 patients (Boston HCH & Birmingham VA)

• 24 experts in homeless health care

• Template coding approach

• Quotes grouped within 8 a priori constructs

• 3 “novel” constructs emerged by consensus 

• Themes identified inductively, linked to quotes
How quotes lead to items

Quote

I don’t necessarily agree I should have control, but to 

share responsibility, that’s what I think. Having a 

conversation with the doctor, listening to the options 

available, talking through the possibilities and having 

a say in what the final outcome is.

Survey Item

If my primary care provider and I were to 

disagree about something related to my 

care, we could work it out

Control
Item testing

• 877 drafted & ranked by research team

• 78 deployed (PCQ-H 1.0)

• 562 persons across 4 sites

• PC patients with homeless experience in 

medical record
Item reduction

Missing

• Dropped if >10% responses missing

CFI 

• Confirmatory Factor Analysis on 11 factors 

using Mplus (satisfactory)

Collapsing

• 11 preliminary scales tested for correlation 

& collapsed when r>0.9  4 scales
Respect

Accountability

Patient 

Control

Information

Flow

Continuity

EvidenceBased

Substance 

Abuse/

Mental 

Health

Cooperation 

among 

Caregivers

Access 

Coordination

Homeless 

Specific 

Needs

PatientClinician 

Relationship

Cooperation Access/ 

Coordination

HomelessSpecific 

Needs
Item reduction strategy

Missing 

• Dropped if >10% missing

CFI

• Confirmatory Factor Analysis on 11 factors

Collapsing

• 11 preliminary scales tested for correlation & 

collapsed when r>0.9  4 scales

Item 

reduction

• Balanced consideration of informational value 

and capacity to detect unfavorable experiences
Item Response Theory

• A latent trait can be something you assess with questions

• Patient perception of the patient-clinician relationship

• A latent trait may:

• Reflect presumptions from prior work (as we did)

• Emerge from exploratory analyses

• A collection of items make a mathematic contribution to a 

computed latent trait value (Mplus Software)

• Items can provide more information or less

• Items can provide information at different levels of the trait
Two Relationship Items (continuity)

• I feel my PCP has spent enough 

time trying to get to know me

• Unfavorable (“disagree”): 17%

• Test Information: strong with two 

high peaks

• My PCP has known me long 

enough to see when I am 

struggling

• Unfavorable (“disagree”): 19%

• Test Information Function: peaks 

well below 1.0, weaker.

0.6-- 

0.3-- 

0-- 

-4 -2 0 2 4

1.5-- 

1.0- 

0.5

Information

Latent trait

-4 -2 0 2 4

Latent trait

Information
PCQ-H 

• 33 items

• 7th Grade Reading Level

• One Likert-type response option across all 

33 

• Strongly Agree

• Agree

• Disagree

• Strongly disagree
Introductory Text

• We would like to ask you some questions about the 

person you see for primary medical care here at 

_____________________. This is the person you see for 

a check-up or for a general medical problem when it is not 

an emergency. This person could be your regular 

personal doctor, a nurse practitioner, or a physician 

assistant. We would also like to ask you about receiving 

care here at ____________________. Please indicate 

how much you agree or disagree with the following 

statements about the person who provides your primary 

medical care. Again, we are asking for you to make your 

best guess. 
Examples for Constructs % a manifestly 

unfavorable 

response

Patient-Clinician Relationship (15 items)

I feel my PCP has spent enough time trying to get to know me 17%

My PCP makes sure health care decisions fit with other 

challenges in my life

9%

Cooperation (3)

My primary care and other health care providers need to 

communicate with each other more

45%

Access/Coordination (11)

If I walk into this place without an appointment, I have to wait too 

long for care

29%

Homeless-Specific Needs (4)

This place tries to help me with things I might need right away like 

food, shelter or clothing

11%
Results

• Convergent validity: r=0.77* with PCAS

• Divergent validity: r=-0.13* with negative psychiatric 

symptoms (<2% of variance)

• Internal consistency (Cronbach α): All > 0.75

• Item-factor loadings similar/higher than CAHPS

• Criterion validity: Scores higher at “tailored” primary 

care sites
Correlations among 4 scales

Relationship Cooperation Access Homelessspecific 

Needs

Relationship 1 0.66 0.78 .66

Cooperation 1 0.65 .51

Access 1 .69

Homelessspecific 

needs

1 

Convergent validity (overall PCQ-33 with Safran’s Primary Care Assessment 

Survey): r=0.77 (p<.001)

Divergent validity (overall against Gun attitude score): r=0.01 (p=0.82)

Internal validity (alphas):

Relationship (0.92), Cooperation (0.75), Access (0.87), Homeless specific 

needs (0.78)
In the Field

• Side by side use at Boston Health Care for the Homeless 

Program CAHPS vs PCQ-H...PCQ-H easier for patients

• VA HSR&D-supported ARCH study: Aligning Resources to 

Care for Homeless Veterans (ARCH)1

• Being used in the 8-site HRSA-funded Special Project of 

National Significance

• Building a Medical Home for Multiply Diagnosed HIV-Positive 

Homeless Populations (PI: Carol Hohl)2 

1. http://clinicaltrials.gov/show/NCT01550757

2. http://hab.hrsa.gov/abouthab/special/homeless.html
Impacts/Dissemination

• Vision: PCQ-H will be the consumer survey of choice for 

providers serving homeless patients

• Current use:

• HHS “Special Project of National Significance” (8 sites)

• VA HSR&D ARCH study (2 sites)

• Briefed to VA national leaders (quality measurement, homeless)

• Next steps:

• Testing/refining real-world use

• Attaining VACO and HHS consensus
Conclusions and Implications

• Homeless Primary Care is unique enough to 

merit special methods of assessing care quality

• Including a special patient questionnaire

• Top-down and bottom-up approaches, combining 

expert consensus and qualitative research 

produced the PCQ-H 

• The resulting 33-item survey appears to be 

practical and is available for use
Thank you

• Request:

• If you wish to use the PCQ-H English or Spanish version, 

contact my team 

(skertesz@uabmc.edu;nancy.johnson8@va.gov;erika.aus

tin@va.gov) 

• We wish to log (a) where used; (b) number of 

respondents; (c ) positive and negative experiences
Elements of Homeless-Tailored Service 

Design

Explicit homeless mission

Primary care in shelters & streets

Team design assures continuity from streets/shelters to clinic

Formal relationships to community shelters

Homeless-focused staff training

PC and mental health in same clinical space

PC clinic equipped to directly meet tangible needs (clothing, food)

Linkage to national homeless organizations

Formerly homeless persons in organizational governance

>10 years explicit homeless mission focus
QUESTIONS OR COMMENTS?

Please use the Q&A box located on the 

right-hand side of your screen.

skertesz@uabmc.edu

For later follow-up
This instructional version is not for direct administration in clinic, but presents items and scoring methodology. 

If you wish to use the Primary Care Quality-Homeless (PCQ-H), please let the PCQ-H Investigative Team know 

(stefan.kertesz@va.gov & nancy.johnson8@va.gov). For VA use, note that OMB Privacy Act rules apply. 

Primary Care Quality-Homeless Survey – Scoring Version 1.0 

Introduction: We would like to ask you some questions about the person you see for primary medical care here at 

___________________________. This is the person you see for a check-up or for a general medical problem when it 

is not an emergency. This person could be your regular personal doctor, a nurse practitioner, or a physician 

assistant. We would also like to ask you about receiving care here at ____________________. Please indicate how 

much you agree or disagree with the following statements about the person who provides your primary medical 

care. Again, we are asking for you to make your best guess. 

Strongly

Disagree

Disagree Agree Strongly

Agree

I Don’t

Know 

Q1. My primary care provider never doubts my 

health needs.

□ 1 □ 2 □ 3 □ 4 □ 99

Q2. My primary care provider takes my health 

concerns seriously.

□ 1 □ 2 □ 3 □ 4 □ 99

Q3. My primary care provider makes decisions 

based on what will truly help me.

□ 1 □ 2 □ 3 □ 4 □ 99

Q4. I feel my primary care provider has spent 

enough time trying to get to know me.

□ 1 □ 2 □ 3 □ 4 □ 99

Q5. I can get in touch with my primary care 

provider when I need to.

□ 1 □ 2 □ 3 □ 4 □ 99

Q6. I can get enough of my primary care 

provider’s time if I need it.

□ 1 □ 2 □ 3 □ 4 □ 99

Q7. If my primary care provider and I were to 

disagree about something related to my care, we 

could work it out.

□ 1 □ 2 □ 3 □ 4 □ 99

Q8. My primary care provider makes sure health 

care decisions fit with other challenges in my life.

□ 1 □ 2 □ 3 □ 4 □ 99

Q9. I worry about whether my primary care 

provider has the right skills to take good care of 

me.

□ 1 □ 2 □ 3 □ 4 □ 99

Q10. I can be honest with my primary care 

provider if I use drugs or alcohol.

□ 1 □ 2 □ 3 □ 4 □ 99

Q11. I worry my primary care provider might 

report my health information to the authorities.

□ 1 □ 2 □ 3 □ 4 □ 99
This instructional version is not for direct administration in clinic, but presents items and scoring methodology.

If you wish to use the Primary Care Quality-Homeless (PCQ-H), please let the PCQ-H Investigative Team know 

(stefan.kertesz@va.gov & nancy.johnson8@va.gov). For VA use, note that OMB Privacy Act rules apply. 

Instructions: For the next questions, we will ask about how your primary care provider works with other 

health care providers. The other health care providers can be other doctors, therapists, or other providers 

who help in your medical care.

Strongly

Disagree

Disagree Agree Strongly

Agree

I Don’t 

Know

Q12. My primary care and other health care 

providers need to communicate with each other 

more.

□ 1 □ 2 □ 3 □ 4 □ 99

Q13. I have been frustrated by lack of 

communication among my primary care and 

other health care providers.

□ 1 □ 2 □ 3 □ 4 □ 99

Q14. My primary care and other health care 

providers are working together to come up with a 

plan to meet my needs.

□ 1 □ 2 □ 3 □ 4 □ 99

Q15. My primary care provider helps to reduce 

the hassles when I am referred to other services.

□ 1 □ 2 □ 3 □ 4 □ 99

Q16. I have to wait too long to get the health care 

services my primary care provider thinks I need.

□ 1 □ 2 □ 3 □ 4 □ 99

Instructions: The next questions are about the place where you go for primary medical care. This is the 

place you normally go for a check-up or for general medical problems when it is not an emergency. The 

place might be a clinic, hospital, or a program. For these questions think about the place and the staff 

who are there.

Strongly 

Disagree

Disagree Agree Strongly

Agree

I Don’t 

Know

Q17. Someone from my primary care provider’s 

office returns my phone or pages.

□ 1 □ 2 □ 3 □ 4 □ 99

Q18. At this place, I have sometimes not gotten 

care because I cannot pay.

□ 1 □ 2 □ 3 □ 4 □ 99

Q19. If I could not get to this place, I think the 

staff would reach out to try to help me get care.

□ 1 □ 2 □ 3 □ 4 □ 99

Q20. f I walk-in to this place without an 

appointment, I have to wait too long for care.

□ 1 □ 2 □ 3 □ 4 □ 99

Q21. This place is open at times of the day that 

are convenient for me.

□ 1 □ 2 □ 3 □ 4 □ 99

Q22. This place helps me get care without 

missing meals or a place to sleep.

□ 1 □ 2 □ 3 □ 4 □ 99

Q23. It is often difficult to get health care at this 

place.

□ 1 □ 2 □ 3 □ 4 □ 99

Q24. This place tells me about what services are □ 1 □ 2 □ 3 □ 4 □ 99
This instructional version is not for direct administration in clinic, but presents items and scoring methodology.

If you wish to use the Primary Care Quality-Homeless (PCQ-H), please let the PCQ-H Investigative Team know 

(stefan.kertesz@va.gov & nancy.johnson8@va.gov). For VA use, note that OMB Privacy Act rules apply. 

available

Q25. The health care services I need are close to 

each other.

□ 1 □ 2 □ 3 □ 4 □ 99

Q26. If my primary care provider is unavailable 

there is someone else that can help me.

□ 1 □ 2 □ 3 □ 4 □ 99

Q27. When I need information about my health 

care, like test results, I can get it easily.

□ 1 □ 2 □ 3 □ 4 □ 99

Q28. The staff at this place listens to me. □ 1 □ 2 □ 3 □ 4 □ 99

Q29. This place tries to help me with things I 

might need right away, like food, shelter or 

clothing.

□ 1 □ 2 □ 3 □ 4 □ 99

Q30. The people who work at this place seem to 

like working with people who have been 

homeless.

□ 1 □ 2 □ 3 □ 4 □ 99

Q31. If I miss an appointment, this place still 

finds a way to help me.

□ 1 □ 2 □ 3 □ 4 □ 99

Q32. At this place, I always have to choose 

between health care and dealing with other 

challenges in my life.

□ 1 □ 2 □ 3 □ 4 □ 99

Q33. Staff at this place treats some patients 

worse if they think that they have addiction 

issues.

□ 1 □ 2 □ 3 □ 4 □ 99

This survey is the product of the Primary Care Quality and Service Customization Study. It is intended 

for clients who have been homeless or who are leaving homelessness behind. Clinics or agencies 

seeking to use it must seek permission from the developer, who can provide scoring 

information. Contact: Stefan G Kertesz, MD, emailing both skertesz@uabmc.edu and Nancy Johnson, 

RN, (nancy.johnson8@va.gov), or phone 205-212-3970.

___________________________________________________________________________________________________

Scoring

Questions are referred to as “Q1” “Q2” etc 

Generally a positively worded item is scored 4 if the client strongly agrees, 3 if they “agree”, 2 for “disagree” and 1 

for “strongly disagree”. 

For example, item Q1: My primary care provider never doubts my health needs 

A negatively worded item (Q22: It is often difficult to get health care at this place) is scored differently. Agreeing with 

it must ultimately result in lower scores, not higher scores. The scoring formula below achieves that by inserting a 

value of “5-Q22” 

Where items are missing, we generally do not score if >50% of items are missing for a particular scale. 
This instructional version is not for direct administration in clinic, but presents items and scoring methodology.

If you wish to use the Primary Care Quality-Homeless (PCQ-H), please let the PCQ-H Investigative Team know 

(stefan.kertesz@va.gov & nancy.johnson8@va.gov). For VA use, note that OMB Privacy Act rules apply. 

Optimum response if <50% of items are missing is to assume that the response to the missing one would be the 

mean of the responses available. 

The simplest way to do this is to compute the mean of the items available, and report that as the final score.

So if the Homeless-Specific Needs scale has 4 items normally, but the individual responded to just 2 of them, giving 

a value of 3 for each, then the computed mean of those 2 items (obviously) is (3+3)/2.

Our research team can provide an Excel sheet to do this automatically. Check with us (skertesz@uabmc.edu and 

nancy.johnson8@va.gov) if you plan to use the instrument in clinical settings.

Relationship (15 items): 

RawTotal_Relat: Q1 + Q2 + Q3 + Q4 + Q5 + Q6 + Q7 + Q8 + [5 - Q9] + Q10 + [5- Q11] + Q17 + Q27 + 

Q28 + [5 – Q33]

Relationship Score = RawTotal_Relat/15

Cooperation (3 items) : 

RawTotal_Coop:[5-Q12]+ [5- Q13] + Q14

Cooperation Score = RawTotal_Coop/3

Access/Coordination (11 items): 

RawTotal_AccCoor: Q15+ [5 – Q16] + [5 - Q18] + Q19 + [5 - Q20] + Q21+ Q22+ [5 – Q23] + Q24+ Q25+ 

Q26

AccCoor Score = RawTotal_AccCoor/11

Homeless-Specific Needs (4 items): 

RawTotal_HomeSp: Q29+ Q30+ Q31+ [5 – Q32]

HomSp Score = RawTotal_HomeSp/4

Missing:

Where <50% of items were missing for a particular subscale, a mean of answered items was imputed for missing items. 

SAS code to accomplish this is available on request from the authors.
I have attached 2 versions of the PCQ-H survey (one to illustrate 

scoring formulas, and one that can be handed directly to patients 

in paper format). I have also attached our team's publication in 

which the PCQ-H was used to compare experiences among 

patients in tailored and non-tailored settings.

 

Although some people will wish to program a scoring calculator 

(and we can provide an excerpt of SAS code for this; just notify 

myself and unita.granstaff@va.gov), there is also an Excel 

workbook that will auto-calculate responses, attached

 

Stefan Kertesz, MD

Birmingham VA Medical Center
Name ID Q1 Q2 Q3 Q4 Q5 Q6 Q7

X1 1 1 3 4 2 3 4

X2 2 3 3 1 1 4 3 2

X3 3 4 4 4 3 3 3 3

X4 4 4 3 3 3 3 2 2

X5 5 3 2 2 2 1 1 1

X6 6 3 3 3 3 3 3 3

X7 7 4 4 4 4 4 4 4

X8 8 2 2 2 2 2 2 2

X9 9 1 1 1 1 1 1 1

X10 10 2 2 2 2 2 2 2

X11 11 3 3 3 4 4 4 4

X12 12 3 2 1 1 1 3 3

X13 13 2 2 3 3 4 4 4

Instructions: For each question listed, enter a response value of 1,2, 3 

response value is missing or "I don't know" then leave blank. After e 

activate formulas.

Note: Any response values entered other than 1, 2, 3 or 4 will result 
Q8 Q9 Q10 Q11 Q12 Q13 Q14 Q15 Q16

4 3 3 3 4 3 4 3 4

4 4 3 4 3 4 4 4 3

2 2 2 2 1 1 1 3 3

2 2 3 3 3 2 2 2 3

3 3 4 4 3 3 3 4 4

3 3 3 3 3 3 3 3 3

4 4 4 4 4 4 4 4 4

2 2 2 2 2 2 2 2 2

1 1 1 1 1 1 1 1 1

2 2 2 2 2 2 2 2 2

4 4 4 4 1 1 1 1 1

3 4 4 4 4 4 2 2 2

3 3 3 2 2 2 2 1 1

3, or 4 as indicated from patient survey. If survey 

 entering response values, pull down scoring cells to 

in calculation errors. 
Q17 Q18 Q19 Q20 Q21 Q22 Q23 Q24 Q25

3 4 4 3 4 3 3 3 4

3 3 2 2 4 1 1 1 4

4 4 3 2 2 2 2 4 4

2 1 1 1 1 4 4 4 2

4 4 4 1 2 3 4 1 2

3 3 3 3 3 3 3 3 3

4 4 4 4 4 4 4 4 4

2 2 2 2 2 2 2 2 2

1 1 1 1 1 1 1 1 1

2 2 2 2 2 2 2 2 2

1 2 2 3 3 4 4 4 4

2 2 2 2 4 4 4 4 4

1 1 3 3 3 3 3 4 4

PCQ-H 33 Scoring Ca 
Q26 Q27 Q28 Q29 Q30 Q31 Q32 Q33

4 4 4 4 3 4 3 3

2 3 4 3 3 2 1

3 4 4 4 4 3 3 2

3 3 3 3 3 4 3 2

3 4 1 2 3 3 3 1

3 3 3 3 3 3 4

4 4 4 4 4 4 3

2 2 2 2 2 4 2 2

1 1 1 1 1 1 1 1

2 2 2 2 2 2 2 3

4 3 3 3 2 1 1 4

4 4 2 2 2 2 3 3

4 4 4 4 4 4 3

alculator 1.0
Any Invalid Values 

Entered?

Relationship 

Score

Cooperation 

Score

Access 

Coordination 

Score

NONE 2.928571429 2.333333333 2.818181818

NONE 2.666666667 2.333333333 2.636363636

NONE 3.266666667 3 2.727272727

NONE 2.733333333 2.333333333 2.545454545

NONE 2.333333333 2.333333333 2.363636364

NONE 2.733333333 2.333333333 2.636363636

NONE 3.466666667 2 2.909090909

NONE 2.2 2.666666667 2.363636364

NONE 1.6 3 2.090909091

NONE 2.133333333 2.666666667 2.363636364

NONE 2.866666667 3 2.909090909

NONE 2.2 1.333333333 3.090909091

NONE 2.933333333 2.666666667 3.090909091

Check Your Data 

Entry for Errors
Homeless 

Specific 

Score

Min 

Value 

Max 

Value

Average 

Relationship 

Score

Average 

Cooperation 

Score

Average 

Access 

Coordination 

Score

3.25 1 4 2.62014652 2.461538462 2.657342657

2.666666667 1 4

3.25 1 4

3 1 4

2.5 1 4

3 3 4

3 3 4

2.75 2 4

1.75 1 1

2.25 2 3

2.5 1 4

2 1 4

3 1 4
Average 

Homeless 

Specific 

Score

2.685897436
Primary Care Quality - Homeless Survey – Patient Version 1.0

Introduction: We would like to ask you some questions about the person you see for primary medical care here at 

_____________________. This is the person you see for a check-up or for a general medical problem when it is not 

an emergency. This person could be your regular personal doctor, a nurse practitioner, or a physician assistant. We 

would also like to ask you about receiving care here at ____________________. Please indicate how much you agree 

or disagree with the following statements about the person who provides your primary medical care. Again, we are 

asking for you to make your best guess. 

Strongly

Disagree

Disagree Agree Strongly

Agree

I Don’t

Know 

Q1. My primary care provider never doubts my health needs. □ □ □ □ □

Q2. My primary care provider takes my health concerns seriously. □ □ □ □ □

Q3. My primary care provider makes decisions based on what will 

truly help me.

□ □ □ □ □

Q4. I feel my primary care provider has spent enough time trying to 

get to know me.

□ □ □ □ □

Q5. I can get in touch with my primary care provider when I need 

to.

□ □ □ □ □

Q6. I can get enough of my primary care provider’s time if I need it. □ □ □ □ □

Q7. If my primary care provider and I were to disagree about 

something related to my care, we could work it out.

□ □ □ □ □

Q8. My primary care provider makes sure health care decisions fit 

with other challenges in my life.

□ □ □ □ □

Q9. I worry about whether my primary care provider has the right 

skills to take good care of me.

□ □ □ □ □

Q10. I can be honest with my primary care provider if I use drugs or 

alcohol.

□ □ □ □ □

Q11. I worry my primary care provider might report my health 

information to the authorities.

□ □ □ □ □

Instructions: For the next questions, we will ask about how your primary care provider works with other health 

care providers. The other health care providers can be other doctors, therapists, or other providers who help in 

your medical care.

Strongly

Disagree

Disagree Agree Strongly

Agree

I Don’t 

Know

Q12. My primary care and other health care providers need to 

communicate with each other more.

□ □ □ □ □

Q13. I have been frustrated by lack of communication among my 

primary care and other health care providers.

□ □ □ □ □

Q14. My primary care and other health care providers are working 

together to come up with a plan to meet my needs.

□ □ □ □ □

Q15. My primary care provider helps to reduce the hassles when I 

am referred to other services.

□ □ □ □ □

Q16. I have to wait too long to get the health care services my 

primary care provider thinks I need.

□ □ □ □ □
Instructions: The next questions are about the place where you go for primary medical care. This is the place 

you normally go for a check-up or for general medical problems when it is not an emergency. The place might 

be a clinic, hospital, or a program. For these questions think about the place and the staff who are there.

Strongly 

Disagree

Disagree Agree Strongly

Agree

I Don’t 

Know

Q17. Someone from my primary care provider’s office returns my 

phone or pages.

□ □ □ □ □

Q18. At this place, I have sometimes not gotten care because I 

cannot pay.

□ □ □ □ □

Q19. If I could not get to this place, I think the staff would reach out 

to try to help me get care.

□ □ □ □ □

Q20. f I walk-in to this place without an appointment, I have to wait 

too long for care.

□ □ □ □ □

Q21. This place is open at times of the day that are convenient for 

me.

□ □ □ □ □

Q22. This place helps me get care without missing meals or a place 

to sleep.

□ □ □ □ □

Q23. It is often difficult to get health care at this place. □ □ □ □ □

Q24. This place tells me about what services are available □ □ □ □ □

Q25. The health care services I need are close to each other. □ □ □ □ □

Q26. If my primary care provider is unavailable there is someone 

else that can help me.

□ □ □ □ □

Q27. When I need information about my health care, like test 

results, I can get it easily.

□ □ □ □ □

Q28. The staff at this place listens to me. □ □ □ □ □

Q29. This place tries to help me with things I might need right 

away, like food, shelter or clothing.

□ □ □ □ □

Q30. The people who work at this place seem to like working with 

people who have been homeless.

□ □ □ □ □

Q31. If I miss an appointment, this place still finds a way to help 

me.

□ □ □ □ □

Q32. At this place, I always have to choose between health care 

and dealing with other challenges in my life.

□ □ □ □ □

Q33. Staff at this place treats some patients worse if they think that 

they have addiction issues.

□ □ □ □ □

This survey is the product of the Primary Care Quality and Service Customization Study. It is intended for clients who have been homeless or who are 

leaving homelessness behind. Clinics or agencies seeking to use it must seek permission from the developer, who can provide scoring information.

Contact: Stefan G Kertesz, MD, emailing both skertesz@uabmc.edu and Nancy Johnson, RN, (nancy.johnson8@va.gov) or Phone 205-212-3970. 
Primary Care Quality – Homeless Survey – Scoring Version 1.0

Introduction: We would like to ask you some questions about the person you see for primary medical care here at 

___________________________. This is the person you see for a check-up or for a general medical problem when it 

is not an emergency. This person could be your regular personal doctor, a nurse practitioner, or a physician 

assistant. We would also like to ask you about receiving care here at ____________________. Please indicate how 

much you agree or disagree with the following statements about the person who provides your primary medical 

care. Again, we are asking for you to make your best guess. 

Strongly

Disagree

Disagree Agree Strongly

Agree

I Don’t

Know 

Q1. My primary care provider never doubts my 

health needs.

□ 1 □ 2 □ 3 □ 4 □ 99

Q2. My primary care provider takes my health 

concerns seriously.

□ 1 □ 2 □ 3 □ 4 □ 99

Q3. My primary care provider makes decisions 

based on what will truly help me.

□ 1 □ 2 □ 3 □ 4 □ 99

Q4. I feel my primary care provider has spent 

enough time trying to get to know me.

□ 1 □ 2 □ 3 □ 4 □ 99

Q5. I can get in touch with my primary care 

provider when I need to.

□ 1 □ 2 □ 3 □ 4 □ 99

Q6. I can get enough of my primary care 

provider’s time if I need it.

□ 1 □ 2 □ 3 □ 4 □ 99

Q7. If my primary care provider and I were to 

disagree about something related to my care, we 

could work it out.

□ 1 □ 2 □ 3 □ 4 □ 99

Q8. My primary care provider makes sure health 

care decisions fit with other challenges in my life.

□ 1 □ 2 □ 3 □ 4 □ 99

Q9. I worry about whether my primary care 

provider has the right skills to take good care of 

me.

□ 1 □ 2 □ 3 □ 4 □ 99

Q10. I can be honest with my primary care 

provider if I use drugs or alcohol.

□ 1 □ 2 □ 3 □ 4 □ 99

Q11. I worry my primary care provider might 

report my health information to the authorities.

□ 1 □ 2 □ 3 □ 4 □ 99

[This space intentionally left blank]
Instructions: For the next questions, we will ask about how your primary care provider works with other 

health care providers. The other health care providers can be other doctors, therapists, or other providers 

who help in your medical care.

Strongly

Disagree

Disagree Agree Strongly

Agree

I Don’t 

Know

Q12. My primary care and other health care 

providers need to communicate with each other 

more.

□ 1 □ 2 □ 3 □ 4 □ 99

Q13. I have been frustrated by lack of 

communication among my primary care and 

other health care providers.

□ 1 □ 2 □ 3 □ 4 □ 99

Q14. My primary care and other health care 

providers are working together to come up with a 

plan to meet my needs.

□ 1 □ 2 □ 3 □ 4 □ 99

Q15. My primary care provider helps to reduce 

the hassles when I am referred to other services.

□ 1 □ 2 □ 3 □ 4 □ 99

Q16. I have to wait too long to get the health care 

services my primary care provider thinks I need.

□ 1 □ 2 □ 3 □ 4 □ 99

Instructions: The next questions are about the place where you go for primary medical care. This is the 

place you normally go for a check-up or for general medical problems when it is not an emergency. The 

place might be a clinic, hospital, or a program. For these questions think about the place and the staff 

who are there.

Strongly 

Disagree

Disagree Agree Strongly

Agree

I Don’t 

Know

Q17. Someone from my primary care provider’s 

office returns my phone or pages.

□ 1 □ 2 □ 3 □ 4 □ 99

Q18. At this place, I have sometimes not gotten 

care because I cannot pay.

□ 1 □ 2 □ 3 □ 4 □ 99

Q19. If I could not get to this place, I think the 

staff would reach out to try to help me get care.

□ 1 □ 2 □ 3 □ 4 □ 99

Q20. f I walk-in to this place without an 

appointment, I have to wait too long for care.

□ 1 □ 2 □ 3 □ 4 □ 99

Q21. This place is open at times of the day that 

are convenient for me.

□ 1 □ 2 □ 3 □ 4 □ 99

Q22. This place helps me get care without 

missing meals or a place to sleep.

□ 1 □ 2 □ 3 □ 4 □ 99

Q23. It is often difficult to get health care at this 

place.

□ 1 □ 2 □ 3 □ 4 □ 99

Q24. This place tells me about what services are 

available

□ 1 □ 2 □ 3 □ 4 □ 99

Q25. The health care services I need are close to 

each other.

□ 1 □ 2 □ 3 □ 4 □ 99

Q26. If my primary care provider is unavailable □ 1 □ 2 □ 3 □ 4 □ 99
there is someone else that can help me.

Q27. When I need information about my health 

care, like test results, I can get it easily.

□ 1 □ 2 □ 3 □ 4 □ 99

Q28. The staff at this place listens to me. □ 1 □ 2 □ 3 □ 4 □ 99

Q29. This place tries to help me with things I 

might need right away, like food, shelter or 

clothing.

□ 1 □ 2 □ 3 □ 4 □ 99

Q30. The people who work at this place seem to 

like working with people who have been 

homeless.

□ 1 □ 2 □ 3 □ 4 □ 99

Q31. If I miss an appointment, this place still 

finds a way to help me.

□ 1 □ 2 □ 3 □ 4 □ 99

Q32. At this place, I always have to choose 

between health care and dealing with other 

challenges in my life.

□ 1 □ 2 □ 3 □ 4 □ 99

Q33. Staff at this place treats some patients 

worse if they think that they have addiction 

issues.

□ 1 □ 2 □ 3 □ 4 □ 99

This survey is the product of the Primary Care Quality and Service Customization Study. It is intended 

for clients who have been homeless or who are leaving homelessness behind. Clinics or agencies 

seeking to use it must seek permission from the developer, who can provide scoring 

information. Contact: Stefan G Kertesz, MD, emailing both skertesz@uabmc.edu and Nancy Johnson, 

RN, (nancy.johnson8@va.gov), or phone 205-212-3970. ___________________________________________________________________________________________________

Scoring 

Questions are referred to as “Q1” “Q2” etc 

Generally a positively worded item is scored 4 if the client strongly agrees, 3 if they “agree”, 2 for “disagree” and 1 

for “strongly disagree”. 

For example, item Q1: My primary care provider never doubts my health needs 

A negatively worded item (Q22: It is often difficult to get health care at this place) is scored differently. Agreeing with 

it must ultimately result in lower scores, not higher scores. The scoring formula below achieves that by inserting a 

value of “5-Q22” 

Where items are missing, we generally do not score if >50% of items are missing for a particular scale. 

Optimum response if <50% of items are missing is to assume that the response to the missing one would be the 

mean of the responses available. 

The simplest way to do this is to compute the mean of the items available, and report that as the final score.

So if the Homeless-Specific Needs scale has 4 items normally, but the individual responded to just 2 of them, giving 

a value of 3 for each, then the computed mean of those 2 items (obviously) is (3+3)/2.
Our research team can provide an Excel sheet to do this automatically. Check with us (skertesz@uabmc.edu

and nancy.johnson8@va.gov) as you complete your work to see if it is available.

Relationship (15 items): 

RawTotal_Relat: Q1 + Q2 + Q3 + Q4 + Q5 + Q6 + Q7 + Q8 + [5 - Q9] + Q10 + [5- Q11] + Q17 + Q27 + 

Q28 + [5 – Q33]

Relationship Score = RawTotal_Relat/15

Cooperation (3 items) : 

RawTotal_Coop:[5-Q12]+ [5- Q13] + Q14

Cooperation Score = RawTotal_Coop/3

Access/Coordination (11 items): 

RawTotal_AccCoor: Q15+ [5 – Q16] + [5 - Q18] + Q19 + [5 - Q20] + Q21+ Q22+ [5 – Q23] + Q24+ Q25+ 

Q26 

AccCoor Score = RawTotal_AccCoor/11

 

Homeless-Specific Needs (4 items): 

RawTotal_HomeSp: Q29+ Q30+ Q31+ [5 – Q32]

HomSp Score = RawTotal_HomeSp/4

Missing:

Where <50% of items were missing for a particular subscale, a mean of answered items was imputed for missing items. 

SAS code to accomplish this is available on request from the authors.
Comparing Homeless Persons’ Care Experiences in

Tailored Versus Nontailored Primary Care Programs

Stefan G. Kertesz, MD, MSc, Cheryl L. Holt, PhD, Jocelyn L. Steward, MSM, Richard N. Jones, ScD, David L. Roth, PhD, Erin Stringfellow, MSW,

Adam J. Gordon, MD, MPH, Theresa W. Kim, MD, Erika L. Austin, PhD, Stephen Randal Henry, DrPH, N. Kay Johnson, RN, MPH,

U. Shanette Granstaff, MPH, James J. O’Connell, MD, Joya F. Golden, PhD, Alexander S. Young, MD, MPH, Lori L. Davis, MD, and

David E. Pollio, PhD, MSW

In the United States, homelessness affects more

than 1.5 million persons per year and more

than 630 000 individuals nightly.1,2 A lack of

stable domiciles makes medical care much

more challenging, and is associated with increased prevalence of medical and psychiatric

conditions3 and mortality.4 Accompanying

these challenges is a pattern of excess hospital

and emergency department utilization,5 lack

of a usual source of care,6 uninsured status,7

and inadequate access to care when needed.8

Health care programs to remediate homeless

persons’ challenges in accessing health care

began in 1985, when the Robert Wood

Johnson Foundation and the Pew Memorial

Trust initiated funding for specialized Health

Care for the Homeless Programs. In 1987, this

funding was assumed by the federal government.9 Since that time, experts have advocated

tailoring service design and delivery to assure

that high quality care is provided to homeless

patients.10---12 However, it is helpful to view

tailored service design on a continuum because

modifications are diverse and may include

a combination of elements, such as outreach

workers to develop relationships outside of

traditional settings, primary care services in

shelters or on the streets, team-based care,

co-location of homeless-dedicated primary and

mental health providers, special education for

providers, capacity to assist with sustenance

needs such as food or hygiene items, and

a robust consumer role in organizational governance.12---14 Some modified programs have

reported improved appointment attendance,15

reductions in hospital admissions, and improvements in disease outcomes.16 However,

such modifications are not required of publicly

funded homeless primary care providers, and

they are not the norm. For example, among

208 federally funded Health Care for the

Homeless Programs in 2010, only 15 had

designated outreach providers (National

Health Care for the Homeless Council, analysis

of health resources and services administration

uniform data set, unpublished data, 2012).

One policy analysis reported that primary care

for the homeless remains inadequate, even

when insurance is available.17 The Patient

Protection and Affordable Care Act (PPACA),

coupled with efforts to encourage the PatientCentered Medical Home model in public and

private settings18 lends impetus to efforts to

assure that vulnerable populations obtain care

that is truly patient-centered.7,19,20

To date, there has been no evaluation of

whether efforts to tailor service delivery for

homeless clients yield a superior patient experience. Homeless patients’ experiences in primary care are important for several reasons.

Patient ratings of their care correlate with

whether care relationships are sustained,21

recommendations are adhered to,22 and in

some reports, whether behavioral conditions

improve.23,24 Also, as emphasized by developers of other primary care rating tools,25,26

patients are the optimum reporters of whether

primary care delivery fulfills the desired attributes described by the Institute of Medicine

and others,27,28 including accessibility, comprehensiveness, continuity, ease of communication, and sensitivity to context.29

Our study compares patients’ assessments of

their own care across 5 primary care settings

that varied in the degree of homeless-tailored

service design, from none (i.e., “mainstream

primary care”) to intensive tailoring. Given the

lack of a population-specific survey, we developed a new patient-reported instrument

specifically for homeless persons. We hypothesized that ratings would be superior for care

obtained in settings that tailored services for

homeless clientele. Because homeless patients

have reported significant negative experiences

Objectives. We compared homeless patients’ experiences of care in health

care organizations that differed in their degree of primary care design service

tailoring.

Methods. We surveyed homeless-experienced patients (either recently or

currently homeless) at 3 Veterans Affairs (VA) mainstream primary care settings

in Pennsylvania and Alabama, a homeless-tailored VA clinic in California, and

a highly tailored non-VA Health Care for the Homeless Program in Massachusetts

(January 2011-March 2012). We developed a survey, the “Primary Care QualityHomeless Survey," to reflect the concerns and aspirations of homeless patients.

Results. Mean scores at the tailored non-VA site were superior to those from

the 3 mainstream VA sites (P < .001). Adjusting for patient characteristics, these

differences remained significant for subscales assessing the patient–clinician

relationship (P < .001) and perceptions of cooperation among providers (P = .004).

There were 1.5- to 3-fold increased odds of an unfavorable experience in the

domains of the patient–clinician relationship, cooperation, and access or coordination for the mainstream VA sites compared with the tailored non-VA site;

the tailored VA site attained intermediate results.

Conclusions. Tailored primary care service design was associated with a superior

service experience for patients who experienced homelessness. (Am J Public Health.

2013;103:S331–S339. doi:10.2105/AJPH.2013.301481)

RESEARCH AND PRACTICE

Supplement 2, 2013, Vol 103, No. S2 | American Journal of Public Health Kertesz et al. | Peer Reviewed | Research and Practice | S331
in care,30,31 our secondary hypothesis was

that unfavorable experiences would be more

common in mainstream primary care compared with tailored settings.

METHODS

This study entailed a survey-based comparison of patient experiences of primary care at 5

federally funded sites (4 within the US Department of Veterans Affairs [VA] and 1

funded through the Department of Health and

Human Services) that differed in the degree

to which primary care service delivery was

tailored for the homeless. The assessment of

patient-perceived primary care was administered as part of a 40 to 60 minute survey

battery conducted face-to-face by research

associates at each site from January 2011

through March 2012. Participants received

a single payment of $20 for their time.

Sites

To test the primary hypothesis, we selected

5 sites with (1) variation in service tailoring

characteristics (based both on investigator direct knowledge supplemented by interviews

with staff), (2) adequately sized panels of

currently or recently homeless patients, and (3)

the presence of a local investigative team

capable of conducting a random sampling and

survey methodology. The sites differed in

service design and delivery in ways that the

literature and clinical experience suggested

could be important to homeless care (Figure 1

and Table 1).10,12 Notably, 3 were traditional

primary care operations (in Pennsylvania and

Alabama) within VA settings where most primary care occurred in standard clinics serving

homeless and housed patients alike without

unique staff or other resources (mainstream

VA A, mainstream VA B, and mainstream VA

C). Among these, mainstream VA A was

a multisite health system that included, for

some patients, primary care provided in shelters and a VA domiciliary. Because of the

minimal provision of primary care services

outside standard clinics and the absence of

additional tailoring modifications, we classified

this site as mainstream. The remaining 2 sites

offered tailored services; one was a VA site

(CA), and the other was a non-VA site (MA).

The tailored non-VA site was a 26-year old

Health Care for the Homeless (HCH) Program32 that provided care in streets, shelters,

and hospital-based clinics, with a specialized

electronic record system, homeless-focused

medical and nursing staff, as well as representation of homeless and formerly homeless

persons in governing and advisory bodies. The

tailored VA site, which began operations in

2002, was designed and funded specifically for

homeless patients, including co-located mental

health and primary care, with an emphasis on

access without scheduling delay.33a

Recruitment

Recruitment was designed to obtain a random sample of homeless-experienced, Englishspeaking persons who received primary care

from each site of care. An automated query of

records was used to identify persons with (1)

presumptive past or current homelessness and

(2) receipt of primary care at the site of care 2

or more times in the past 2 years. In the 4

VA sites, presumptive past or current homelessness was based on an International Classification of Diseases-9-CM code of V60.0 (i.e.,

homeless) diagnosis.33b In the tailored non-VA

site, past or current homelessness was based on

utilization of the site for care. A random subset

at each site was targeted for recruitment,

with the number targeted based on consideration of (1) number of homeless served at the

site and (2) numbers necessary to identify

differences in satisfaction with sufficient power.

Entry was further restricted based on the

respondent verbally affirming receipt of primary care at the site of interest. Across all 5

sites, 6371 persons met the criteria from July 1,

2008, to June 30, 2010. After random

0

10

20

30

40

50

60

70

Relationship** Cooperation** Access/Coordination* Homeless specific

needs**

Tailored non‐VA Tailored VA Mainstream VA‐A Mainstream VA‐B Mainstream VA‐C

Primary Care Scale

Unfavorable Experience, %

Note. VA = US Department of Veterans Affairs. Unfavorable experience was defined based on the number of “frankly

unfavorable” responses falling into the highest tertile observed for that particular scale (i.e., agree or strongly agree with

a negatively worded item, or disagree or strongly disagree with a positively worded item). The highest tertile for unfavorable

responses was found to be 2 or more frankly unfavorable responses for the relationship, cooperation and access/

coordination scales, and 1 or more frankly unfavorable responses for the homeless-specific needs scale. Tailored non-VA

refers to a 26-year old Health Care for the Homeless Program. Mainstream VA sites are non-tailored primary care sites in

Pennsylvania (A), and Alabama (B and C). Tailored VA is a VA homeless-tailored primary care program (California).

*P < .05; **P < .01.

FIGURE 1—Unfavorable experiences in primary care, among persons receiving primary care

in 5 different primary care programs, January 2011–March 2012.

RESEARCH AND PRACTICE

S332 | Research and Practice | Peer Reviewed | Kertesz et al. American Journal of Public Health | Supplement 2, 2013, Vol 103, No. S2
selection, 2584 (41%) were subject to recruiting, 870 (14%) were successfully contacted

and screened, 634 (10%) entered the study,

and 601 (9.4%) were included in analysis. Of

the 33 entrants excluded from analysis, 19 did

not receive care at the site, 1 had a mistaken

identity, 1 had prohibitive behavioral issues, 1

refused to sign a Health Insurance Portability

and Accountability Act consent, and 11 had

survey administration that was compromised

(e.g., stopped because of competing appointment;

data available as a supplement to the online

version of this article at http://www.ajph.org).

The Primary Care Quality-Homeless

Survey

Although patient-reported primary care assessments exist,34,35 there is no standard survey instrument to assess primary care for

homeless populations. Because research has

shown that the concerns, priorities, and aspirations of homeless patients can differ from

those in mainstream populations,36,37 we developed a new instrument, the Primary Care

Quality-Homeless (PCQ-H) Survey.

A 2-year qualitative research process began

with review of 2 Institute of Medicine reports27,38 to define major constructs relevant

to primary care quality. These constructs were

explored in semistructured interviews with 36

homeless-experienced patients and 22 experts

in homeless health care. Based on interview

coding, we developed a preliminary set of

78 items. After administration to all study

participants, items were submitted to factor

analysis to identify groups of items measuring

distinct domains, and then within domains,

item response theory (2 parameter graded

response analysis39) was applied to identify

a subset of these items that was optimally

discriminating for and adequately covered the

range of responses for each domain. Items that

were psychometrically equivalent and did not

address conceptually unique aspects of the

measured domains were dropped. The resulting

33-item PCQ-H Survey consisted of 4 scales:

(1) patient---clinician relationship (15 items), (2)

cooperation among clinicians (3 items), (3)

accessibility or coordination (11 items), and (4)

homeless-specific needs (4 items; data available

as a supplement to the online version of this

article at http://www.ajph.org).

Each item on the survey is a simple statement regarding the primary care provider or

the site or program where the client obtained

primary care (e.g., “My primary care provider

makes sure health care decisions fit with the

other challenges in my life”), scored on a

4-point Likert scale with reverse-coding for

negatively worded items. Averaging the

responses generated a subscale score from 1

(least favorable) to 4 (most favorable).

Covariates

To more accurately compare ratings of

primary care across sites, our analysis controlled for a range of patient characteristics.

These characteristics were selected on the basis

of existing empirical literature regarding

patient-level predictors of satisfaction40---42 and

characteristics identified by the Behavioral

Model for Vulnerable populations43 (which

predicts service use among the homeless8,44,45).

Items related to residential status were

drawn from previous work focused on residence in the past 2 weeks,46---48 homelessness

in the past 6 months,49 and questions to assess

whether the person had been “chronically

homeless” in the past 3 years (‡ 4 episodes in

3 years or a period longer than 1 year).50,51

Health status was measured by the single-item

general self-reported health question, which

was shown to strongly predict both mortality

and health care utilization.52,53 Psychiatric

symptoms were assessed with the 14-item

Colorado Symptom Index.54 In some models,

a binary indicator of “severe psychiatric

symptoms” was based on affirming any 1 of the

following experiences several times per week

or more: being told one acted suspicious or

TABLE 1—Variation in Degree of Primary Care Service Tailoring in Tailored vs Nontailored Care Programs: 5 Study Sites,

January 2011–March 2012

Primary Care Service Design Characteristics

Mainstream VA B

Mainstream VA C (Alabama)

Mainstream VA A

(Pennsylvania)

Tailored VA

(California)

Tailored Non-VA

(Massachusetts)

Explicit homeless mission X X

Primary care in shelters and streets X X

Team design assures continuity from streets/shelters to clinic X

Formal relationships to community shelters X X

Homeless-focused staff training X X

PC and mental health in same clinical space X X X

PC clinic equipped to directly meet tangible needsa

Linkage to national homeless organizations X

Formerly homeless persons in organizational governance X

> 10 y explicit homeless mission focus X

Note. PC = primary care; VA = Veterans Affairs. Tailored non-VA refers to a 26-year old Health Care for the Homeless Program. Mainstream VA sites are non-tailored primary care sites in Pennsylvania

(A), and Alabama (B and C). Tailored VA is a VA homeless-tailored primary care program (California). Characteristics pertain to the design of the primary care service utilized by study participants.

Survey questions defined primary care as the person or team seen for a check-up or for a general medical problem when it is not an emergency. By design, recruitment was initiated only with

persons who had recorded evidence of 2 or more primary care visits. An X indicates the characteristic pertains to the primary care service utilized. Thus “clinic equipped to directly meet tangible

needs” is designated only when the primary care site utilized by patients was specifically equipped to directly meet such needs, and not designated if such a service was available elsewhere in the

same hospital-based system of care.

a

Such as clothing and food.

RESEARCH AND PRACTICE

Supplement 2, 2013, Vol 103, No. S2 | American Journal of Public Health Kertesz et al. | Peer Reviewed | Research and Practice | S333
paranoid; hearing voices others could not hear,

feeling suspicious or paranoid, or feeling like

hurting or killing oneself or another (a safety

algorithm included clinical response to persons

affirming this item). Substance use measures

came from the World Health Organization

Alcohol, Smoking and Substance Involvement

Screening Test (ASSIST) 3.0.55

Analysis

Analysis proceeded in 3 phases. First, respondents participating at the 5 recruitment

sites were compared with respect to a range of

demographic, health, and health service utilization measures using analysis of variance for

continuous variables and the v2 test for categorical variables. Second, continuous PCQ-H

scores were compared across sites, first without

covariate adjustment and then controlling for

case mix, including all covariates in a multiple

linear regression model. Because the recruitment strategy risked enriching the sample with

“more stable” or “less vulnerable” homelessexperienced persons, analyses included a priori

plans to assess for differences within stratified

groups: persons with a history of chronic

homelessness; persons with fair or poor general

health status; and persons with current severe

psychiatric symptoms. Last, in light of the interest in reducing negative experiences in

homeless persons’ health care, we developed

a categorical “unfavorable experience” indicator

based on the number of unfavorable responses

in the top tertile (33%) for each subscale,

analogous to bottom-box scoring. An unfavorable response would be agree or strongly agree

with a negative item (e.g., “It is often difficult to

get health care at this place”) or disagree or

strongly disagree with a positive item (e.g. “My

primary care provider takes my health concerns

seriously”). We used the v2 test to compare the

prevalence of unfavorable experiences, adjusting for case mix with multiple logistic regression.

All site comparison analyses were conducted

using SAS versions 9.2 and 9.3 (SAS Institute,

Cary, NC), whereas factor analyses were completed

with MPlus version 6.12,56 and item response

theory analyses utilized the R module ltm.57

RESULTS

Participants were mostly men (85%), with a

mean age of 53 years (68.3 years). Sixty-eight

percent reported it was hard to pay for basic

necessities like food and shelter (Table 2).

Overall, 70% had met temporal criteria for

chronic homelessness in the previous 3 years

(an episode > 1 year, or ‡ 4 times in 3 years).

As might be expected given the recruitment

approach (mail and telephone contact), literal

homelessness in the 6 months before study

participation (e.g., any nights on the streets or in

shelters) was less common (24% overall).

Nearly half the participants characterized

their general health as fair or poor (43.1%)

compared with 9.6% of Americans in the

general population.58 Severe psychiatric

symptoms several times per week were common (24.5%), especially among participants at

mainstream site VA C (42.1%).

For each of the 4 subscales assessed, PCQ-H

mean scores (rang = 1---4) were as low (unfavorable) as 2.43 and as high (favorable) as

3.32, with standard deviations ranging from

0.35 (relationship) to 0.6 (cooperation).

Comparison of Primary Care

Quality-Homeless Continuous Scores

Comparisons of unadjusted mean PCQ-H

scores across the 5 sites showed that scores

differed on all 4 subscales (patient---clinician

relationship and cooperation, P < .001; accessibility or coordination, P = .024; and

homeless-specific needs, P = .033). Specifically,

scores at the tailored non-VA site were higher

(reflecting more positive experience with care)

than those at the 3 mainstream VA sites. The

tailored VA site generally had scores that were

either similar to those of the 3 mainstream VA

sites or somewhat higher, depending on the

subscale of interest. The mainstream VA 3 had

consistently lower scores, often close to a full

SD below the tailored VA site, which was

a large effect size.

Adjusting for patient characteristics, differences remained significant for the relationship

(P < .001) and cooperation (P = .005) subscales, whereas they fell short of statistical

significance in the case of access or coordination (P = .055) and homeless-specific needs

(P = .21).

As shown in Table 3, the magnitude of

difference favoring the tailored non-VA site

over mainstream VA B site was 0.25 to 0.50

SD for relationship and cooperation, which was

a small to moderate effect size. The magnitude

of difference was more modest in comparison

with the mainstream VA A site (because it

performed better) and more pronounced in

comparison with the mainstream VA C site

(which performed worse).

Adjusted differences among the sites were of

similar magnitude and statistically significant

after restriction to subgroups defined by history of chronic homelessness or fair or poor

general health status (data available as a supplement to the online version of this article

at http://www.ajph.org). The tailored VA site

typically attained an intermediate position in

these comparisons.

Comparison of Categorical “Unfavorable

Experience” Indicator

Finally, an unfavorable experience was 1.5

to 2 times more common at the mainstream VA

sites compared with the tailored non-VA site

(all P < .01 in unadjusted v2 comparisons), and

these contrasts remained statistically significant

after adjusting for patient characteristics for

the relationship, cooperation, and access or

coordination scales. Analyses comparing the

tailored VA site with the tailored non-VA site

obtained an intermediate result. Specifically,

patients at the tailored VA site were more likely

than patients at the tailored non-VA site to

have an unfavorable experience on the access

or coordination subscale, but not with the other

3 subscales.

DISCUSSION

We tested the hypothesis that homelessexperienced patients would rate their primary

care experience more highly when care was

obtained in settings that explicitly tailored

services for this population through variations

in service design (Table 1). This hypothesis

was supported by our results, both in comparison of mean scores from the PCQ-H subscales,

and in the likelihood of a categorical outcome

indicating an unfavorable experience in primary care.

Patient experiences are typically influenced

by patient, provider, and environmental characteristics.29 Despite the range of characteristics

that could dilute detection of an effect from

tailored service design, survey scores for

patient---clinician relationship and perceptions

of cooperation among providers were highest

RESEARCH AND PRACTICE

S334 | Research and Practice | Peer Reviewed | Kertesz et al. American Journal of Public Health | Supplement 2, 2013, Vol 103, No. S2
TABLE 2—Characteristics of Participants by Site in a Comparison of Homeless Persons’ Care in Tailored vs Nontailored Care Programs: 5 Study Sites,

January 2011–March 2012

Characteristics

Tailored Non-VA

(Massachusetts, n = 195),

No. (%) or Mean

6SD

Tailored VA (California, n= 94),

No. (%) or Mean

6SD

Mainstream VA A

(Pennsylvania, n = 124),

No. (%) or Mean

6SD

Mainstream VA B

(Alabama, n = 150),

No. (%) or Mean

6SD

Mainstream VA C

(Alabama, n = 38),

No. (%) or Mean

6SD P

a

Demographics

Age, y 51.1

610.5 55.9

66.8 53.6

66.9 53.7

67.1 50.9

69.6 < .001

Gender

Male 133 (68.2) 94 (100) 112 (90.3) 139 (92.7) 34 (89.5) < .001Female 59 (30.3) 0 11 (8.9) 11 (7.3) 4 (10.5)Other/transgender 3 (1.5) 0 1 (0.8) 0 0

Race

Hispanic, Latino, or Spanish origin 20 (10.5) 6 (06.5) 2 (1.6) 1 (0.7) 0 (0.0) < .001White 77 (39.5) 13 (13.8) 49 (39.5) 33 (22.0) 12 (31.6) < .001Black, African American 75 (38.5) 73 (77.7) 66 (53.2) 111 (74.0) 21 (55.3)American Indian/Alaska Native 2 (01.0) 1 (01.1) 5 (4.0) 1 (0.7) 2 (5.3)Asian/Pacific Islander 2 (01.0) 1 (01.1) 0 (0.0) 0 (0.0) 0 (0.0)Other 39 (20.0) 5 (06.4) 4 (3.2) 5 (3.3) 3 (7.9)Race non-White 119 (61.0) 81 (86.2) 75 (60.5) 117 (78.0) 26 (68.4) < .001

Ever served in military 27 (14.4) 94 (100) 124 (100) 150 (100) 38 (100) < .001Difficulty paying for basic necessities 63 (67.7) 123 (67.7) 84 (68.9) 97 (66.9) 27 (77.1) .823

Residential history

Chronically homeless in past 3 y 164 (84.1) 68 (72.3) 67 (54.0) 85 (56.7) 21 (55.3) < .001Spent ‡ 1 nights in street, in car or abandoned

building or in emergency shelters, last 2 wk

37 (18.7) 7 (7.5) 10 (8.1) 14 (9.3) 1 (2.6) .002

Self-designates as currently homeless 61 (31.3) 27 (29.0) 33 (26.6) 50 (33.3) 14 (36.8) .685Currently domiciled, last 2 wkb 117 (60.0) 57 (60.6) 92 (74.2) 88 (58.7) 23 (60.5) .064

Health services

Health insurancec 181 (92.8) 83 (88.3) 106 (85.5) 141 (94.0) 33 (86.8) .078

“How many times have you seen this provider in last 6 months?” 4.7

65.1 2.3

61.8 1.6

61.2 3.3

66.8 2.2

62.3 < .001

Health status

General health status fair or poor 89 (46.6) 35 (38.0) 48 (39.0) 66 (44.3) 18 (47.4) .539Psychiatric symptom intensity in last 6 mo, Colorado (range = 0–56)d 16.6

611.1 15.8

610.6 15.2

610.6 17.6

612.4 25.3

611.6 < .001

Severe psychiatric symptoms several times per wkd 40 (20.9) 21 (22.3) 18 (14.6) 50 (34.0) 16 (42.1) < .001

Drug Global Continuum Score (ASSIST)e 27.6

627.2 23.8

618.8 25.0

623.9 18.6

615.4 19.8

618.2 .005

Alcohol Specific Substance Involvement Score (ASSIST),e 9.8

69.9 5.8

66.5 7.5

68.3 18.6

615.4 8.5

68.3 < .004

Note. VA = Veterans Affairs. Tailored non-VA refers to a 26-year old Health Care for the Homeless Program. Mainstream VA sites are non-tailored primary care sites in Pennsylvania (A), and Alabama (B and C). Tailored VA is a VA homelesstailored primary care program (California).

a

P values are based on comparison among the 5 sites using the v2 test (categorical) or analysis of variance (continuous). b“Currently domiciled” was designated for persons reporting at least 1 night during the past 2 weeks in their “own apartment/house” or in “permanent supportive housing provided by an organization or the government.” cInsured status included affirmation of having “a source of insurance that can pay for your medical expenses,” including private insurance, Medicare, Medicaid, or VA benefits. dPsychiatric symptoms probed using previously validated Colorado Symptom Index.48 “Severe” symptoms were being told one acted suspicious or paranoid, hearing voices others could not hear, feeling suspicious or paranoid, feeling like

hurting or killing oneself or another person.

eScores related to drug and alcohol use were devised from the World Health Organization Alcohol, Smoking and Substance Involvement Screening Test (ASSIST).49

RESEARCH AND PRACTICE

Supplement 2, 2013, Vol 103, No. S2 | American Journal of Public Health Kertesz et al. | Peer Reviewed | Research and Practice | S335
where homeless-tailoring was most pronounced.

The comparison of “unfavorable experiences”

produced a more striking contrast. For 3 subscales (relationship, cooperation, and access or

coordination), such unfavorable experiences

were 1.5 to 2 times more frequent in the 3

mainstream settings. The tailored VA site

obtained results that were, on whole, intermediate, which often lined up with the 2 better

performing mainstream VA sites (in analyses of

mean scores), and were sometimes more like

the tailored non-VA site (e.g., for unfavorable

experience on items related to relationship,

cooperation, and homeless-specific needs).

A review of study sites (Table 1) underscored that service tailoring was not all-ornone, but rather could be viewed on

a continuum. The tailored VA site had fewer

elements of tailoring than the tailored non-VA

site, and attained ratings that were, on the

whole, intermediate between the mainstream

VA sites and the tailored non-VA site. The

mainstream VA A site offered some shelterbased and domiciliary-based primary care for

a subset of patients, which was also a potential

element of tailored service design.

Tradeoffs characterize the question of

whether to prioritize mean score comparisons

versus a categorical unfavorable experience.

We believe both are useful. As noted for the

widely used Consumer Assessment of Health

Plans,59 mean scores are more precise in

capturing performance relative to average, but

experience the downside that scores are compressed in a small part of the available range

and could be hard for consumers to interpret.

Conversely, categorical results better illustrated

important outcomes to consumers, although

they sacrificed precision.59 In this study, both

sets of results aligned with the expectation that

high levels of tailoring were associated with

a better experience. Importantly, however, we

could make no claim to a correlation between

improved patient experiences with any medical

outcome, such as decreased morbidity or

mortality. Health improvements might only be

demonstrable through study of persons whose

poor experiences led to disengagement from

primary care, a group we were not positioned

to study.

Limitations and Strengths

This study had limitations. First, individuals

were not randomly assigned to clinics. Thus, it

was possible that some characteristics of the

patients or the 5 clinical settings, other than

service tailoring, could account for the results.

It could be speculated that VA facilities delivered a less patient-friendly experience than

non-VA organizations. However, the higher

TABLE 3—Comparison of Primary Care Quality-Homeless Scores by Site of Care in Tailored vs Nontailored Care Programs:

January 2011–March 2012

Tailored, Mean 6SD, Mean (SE), or OR (95% CI) Mainstream, Mean 6SD, Mean (SE), or OR (95% CI)

Scores No. Tailored Non-VA Tailored VA Mainstream VA A Mainstream VA B Mainstream VA C P

a

Subscale scores, unadjusted

Relationship 600 3.32 60.41 3.28 60.46 3.28 60.55 3.13 60.49 2.95 60.49 < .001b

Cooperation 566 2.97 60.52 2.85 60.82 2.81 60.82 2.75 60.70 2.43 60.62 < .001b

Access/coordination 600 3.17 60.40 3.10 60.48 3.15 60.53 3.12 60.44 2.90 60.39 .024c

Homeless-specific needs 596 3.17 60.46 3.17 60.49 3.05 60.60 3.05 60.51 2.96 60.50 .033

Subscale scores, adjustedd

Relationship 551 3.45 (0.09) 3.38 (0.10) 3.37 (0.09) 3.26 (0.09) 3.07 (0.12) < .001

Cooperation 522 3.15 (0.13) 2.96 (0.15) 2.93 (0.14) 2.89 (0.14) 2.71 (0.18) .005

Access/coordination 551 3.29 (0.08) 3.19 (0.10) 3.24 (0.09) 3.24 (0.09) 3.03 (0.11) .055

Homeless-specific needs 547 3.38 (0.09) 3.38 (0.11) 3.25 (0.10) 3.31 (0.10) 3.26 (0.13) .209

Unfavorable experienced,e

Relationship 551 Ref 1.84 (0.96, 3.54) 2.60 (1.49, 4.56) 2.57 (1.48, 4.47) 2.38 (1.03, 5.48) .005

Cooperation 522 Ref 1.87 (0.88, 4.00) 2.73 (1.46, 5.09) 2.73 (1.45, 5.14) 2.70 (1.06, 6.84) .01

Access/coordination 551 Ref 2.36 (1.22, 4.57) 2.50 (1.42, 4.41) 1.96 (1.11, 3.46) 2.38 (1.02, 5.56) .018

Homeless-specific needs 547 Ref 0.96 (0.51, 1.81) 2.15 (1.27, 3.63) 1.45 (0.87, 2.44) 1.90 (0.85, 4.28) .021

Note. CI = confidence interval; OR = odds ratio; VA = Veterans Affairs. Tailored non-VA refers to a 26-year old Health Care for the Homeless Program. Mainstream VA sites are non-tailored primary

care sites in Pennsylvania (A), and Alabama (B and C). Tailored VA is a VA homeless-tailored primary care program (California).

a

In unadjusted comparisons, the P value is from an overall test for significant differences among the 5 included sites. In the 2 adjusted analysis, the P value reflects a test of whether the site (5

categories, degrees of freedom = 4) was significant after controlling for variables detailed under footnote b.

b

Post hoc tests revealed significant pairwise differences between tailored non-VA and mainstream VA B; tailored non-VA and mainstream VA C; tailored VA and mainstream VA C; and mainstream VA A

and mainstream VA C.

c

Post hoc tests revealed significant pairwise differences between tailored non-VA and mainstream VA C; and mainstream VA A and mainstream VA C.

d

Adjusted analyses control for age, gender, Black race, having had one’s own domicile (apartment or house) in past 2 weeks, psychiatric symptoms (Colorado score), drug and alcohol risk scores on

the World Health Organization Alcohol, Smoking and Substance Involvement Screening Test (ASSIST), general self-reported health, and low income status. e

Unfavorable experience was defined based on the number of “frankly unfavorable” responses falling into the highest tertile observed for that particular scale (i.e., agree or strongly agree with

a negatively worded item, or disagree or strongly disagree with a positively worded item). The highest tertile for unfavorable responses was found to be 2 or more unfavorable responses for the

relationship, cooperation and access/coordination scales, and 1 or more unfavorable responses for the homeless-specific needs scale.

RESEARCH AND PRACTICE

S336 | Research and Practice | Peer Reviewed | Kertesz et al. American Journal of Public Health | Supplement 2, 2013, Vol 103, No. S2
quality reported from the VA in comparison

with the private sector settings,60 coupled

with VA satisfaction ratings similar to those

from commercial insurance,61 made this less

plausible. Similarly, although homeless veterans differed from homeless nonveterans

in some samples, the differences were not

systematic and consistent across studies.62

Ultimately, more robust efforts to disentangle

potential VA-specific influences await research

from a wider range of VA and non-VA care

settings.

We measured and adjusted for several patient characteristics, but it can be assumed that

other environmental factors influenced results.

For example, the 38 participants at the mainstream VA C site offered considerably lower

ratings. Because that facility included a regional

psychiatric and rehabilitative facility, it was

conceivable that aspects of its referral pattern

differed in ways not captured by our measures

or by the homeless-tailoring concept.

Second, by studying 4 VA sites and a health

center in Massachusetts, few in the sample

lacked financial coverage for care, and questions concerning financial access might have

been less informative. Because financial barriers arise elsewhere,63 we believed these items

might prove helpful in other settings.

Lastly, recruitment utilized a random record

query, with initial contact often via telephone

or mail. Although reducing a bias toward

“happy customers” obtained with waiting

room samples, our sample was dominated by

persons who were homeless-experienced

rather than homeless at the time of the survey.

Because results favoring tailored care were

more pronounced among vulnerable subgroups (i.e., persons who met the federal

chronic homeless definition, or persons in fair

or poor health), we believed this was not

a severe limitation.

Alongside limitations, our study had

strengths in the breadth of characteristics

assessed and statistically adjusted, and in randomly sampling a population that could be

challenging to recruit. Finally, our might be the

first to utilize homeless patients in every aspect

of questionnaire design. The resulting 33-

item instrument might be useful with homeless

and potentially other vulnerable populations

facing challenges to obtaining high-quality

primary care.30,36,64

Implications for Primary Care Settings

Understanding how specific organizational

characteristics affect patients’ primary care

experiences will require further research.

However, some speculation could be offered

regarding the relationship and cooperation

subscales, in which differences were most pronounced. Patient perceptions of cooperation

among caregivers might be influenced by

actual co-location of services (e.g., primary

medical, mental, social work) as well as demonstration that team members actually communicated with each other in ways that went

beyond the medical record. Communication

might become evident to patients when they

see providers speaking with each other, when

providers mention each other by name, or

when they describe conversations with other

team members. Such events might occur more

readily and be most apparent to patients when

care is delivered in distinct mission-driven

teams, as in the 2 tailored settings. Items that

scaled together as the “relationship” subscale

encompassed aspects of communication and

trust. In mainstream settings, homeless patients

might feel either mistrusted or unwelcome.30

Tailored clinics might remediate these challenges in part by recruiting providers who wish

to work with the homeless population. Mutual

support and mentorship might be more readily

fostered among like-minded professionals, so

that challenges to the clinical relationship (e.g.,

a lost prescription, a missed appointment) are

handled constructively.

Implications for Health Care Policy

A recent policy analysis found that the dominant mainstream model for delivering primary

care to homeless individuals is not adequate,

but little empirical evidence exists to guide

selection of a superior approach.17 Both the US

Department of Veterans Affairs and the US

Department of Health and Human Services

support some tailored programs focused on the

primary care of homeless individuals, and they

have the wherewithal to encourage tailored

approaches to care, to the degree that evidence

in their favor is persuasive. Patient reports of

their own experiences offer important evidence

in favor of expanding these tailored approaches.

More broadly, these findings have implications for how health care system changes

may affect homeless patients, many of whom

will become eligible for Medicaid under the

PPACA. The Act funds payment and delivery

arrangements across insurers, and implements

demonstration projects that seek to improve

quality and accountability. However, identifying the optimum mechanism to assure good

care experiences for vulnerable patient populations remains a key challenge. Federally

Qualified Health Centers, which PPACA expands, have in recent years been encouraged to

seek recognition as Patient-Centered Medical

Homes according to national criteria.65 A

number of criteria embody principles of practice adopted by homeless-tailored programs,

including an emphasis on coordination, accessibility, and individualizing care to patient

needs and circumstances.12---14,66 In this regard,

the experience of successful homeless primary

care programs could inform policymakers

dedicated to vulnerable patient populations.

The present study suggests that tailored service

delivery matters to patients in ways that are

readily measurable. Future work will be needed

to learn which aspects of service tailoring matter

most, and whether they are easily translated

across service environments for both homeless

and nonhomeless patient populations. j

About the Authors

Stefan G. Kertesz, Erika L. Austin, N. Kay Johnson, and U.

Shanette Granstaff are with the Birmingham VA Medical

Center, Birmingham, AL. Cheryl L. Holt is with the

University of Maryland, Baltimore. Jocelyn L. Steward is

with the University of Alabama, Birmingham. Richard

N. Jones is with Brown University, Providence, RI. David

L. Roth is with Johns Hopkins University, Baltimore, MD.

Erin Stringfellow is with Washington University, St. Louis,

MO. Adam J. Gordon is with the VA Pittsburgh Healthcare

System, Pittsburgh, PA. Theresa W. Kim is with the

Boston Medical Center and the Boston Healthcare for the

Homeless Program, Boston, MA. Stephen Randal Henry,

Joya F. Golden, and Alexander S. Young are with the VA

Medical Center, Greater Los Angeles, Los Angeles, CA.

James J. O’Connell is with the Boston Health Care for the

Homeless Program, Boston. Lori L. Davis is with the

Tuscaloosa VA Medical Center, Tuscaloosa, AL. David

E. Pollio is with the University of Alabama, Tuscaloosa.

Correspondence should be sent to Stefan G. Kertesz, MD,

MSc, Birmingham VA Medical Center, Center for Surgical,

Medical Acute Care Research and Transitions, Stop

Code 151, 700 South 19th Street, Birmingham. AL

35233 (e-mail: Stefan.Kertesz@va.gov, skertesz@uabmc.

edu). Reprints can be ordered at http://www.ajph.org by

clicking the “Reprints” link.

This article was accepted May 29, 2013.

Contributors

S. G. Kertesz conceptualized and designed the study,

designed the data and collection procedures, collected

data, analyzed and interpreted results, wrote the primary

RESEARCH AND PRACTICE

Supplement 2, 2013, Vol 103, No. S2 | American Journal of Public Health Kertesz et al. | Peer Reviewed | Research and Practice | S337
draft of results, and wrote revisions of the article. C. L.

Holt conceptualized and designed the study, designed

the data and collection procedures, analyzed and interpreted results, and wrote revisions of the article. J. L.

Steward designed the data collection procedures, collected data, and analyzed and interpreted results. R. N.

Jones, D. L. Roth, A. J. Gordon, T. W. Kim, and J. J.

O’Connell conceptualized and designed the study, analyzed and interpreted the results, and wrote revisions of

the article. E. Stringfellow designed the data collection

procedures, collected data, analyzed and interpreted

results, and wrote the primary draft of the results. E. L.

Austin, U. S. Granstaff, and J. F. Golden collected data,

analyzed and interpreted the results, and wrote revisions

or the article. S. R. Henry helped design data collection

procedures and assisted with survey design. N. K. Johnson designed the data collection procedures, collected

data, analyzed and interpreted results, and wrote revisions of the article. A. S. Young analyzed and interpreted the results and wrote revisions of the article. L. L.

Davis conceptualized and designed the study and wrote

revisions of the article. D. E. Pollio designed the data

collection procedures, analyzed and interpreted results,

and wrote revisions of the article. All authors gave final

approval of the article.

Acknowledgments

Research was funded by the Department of Veterans

Affairs, Health Services Research & Development Branch

(Grant IAA-07-069-2). E. Stringfellow received additional support from the National Institute on Drug Abuse

(T32DA01035).

Although we had no commercial conflicts of interest,

we want to acknowledge that most of the authors, except

for J. F. Jones, C. L. Holt, and D. E. Pollio, have been

employed or are currently employed by institutions

studied in this articles (4 VA hospitals, and 1 independent

nonprofit Health Care for the Homeless Program).

Preliminary versions of the findings in this article were

previously presented at the Annual Meetings of the

Society of General Internal Medicine (2012, 2013), the

VA Health Services Research and Development Conference (2012), the Annual Conference of the American

Public Health Association (2012), and Academy Health

(2012).

We would like to thank Carol Ashton, Joshua Richman, Alison Borden, John D. Harding, Lori Henault, Jane

Liebschutz, Sonia Schwartz, Dawn Glover, John Andrew

Young, and Calvin Elam for their assistance in the design

or execution of this study.

Note. The opinions expressed are those of the authors

and do not represent the positions of the Department of

Veterans Affairs or any other entity of the United States

federal government.

Human Participant Protection

Research was conducted under the approval of 5 separate institutional review boards at the 5 included study

sites. We affirm that the research was conducted in

keeping with the Principles of the Ethical Practice of

Public Health.

References

1. Burt MR, Aron LY, Lee E, Valente J. Helping

America’s Homeless: Emergency Shelter or Affordable Housing? Washington, DC: The Urban Institute Press; 2001.

2. United States Department of Housing and Urban

Development. 2012 Point-in-Time Estimates of

Homelessness: Volume 1 of the 2012 Annual Homeless

Assessment Report. 2012. Available at: http://www.onecpd.

info/resource/2753/2012-pit-estimates-of-homelessnessvolume-1-2012-ahar. Accessed December 26, 2012.

3. Kleinman LC, Freeman H, Perlman J, Gelberg L.

Homing in on the homeless: assessing the physical health

of homeless adults in Los Angeles County using an

original method to obtain physical examination data in

a survey. Health Serv Res. 1996;31(5):533---549.

4. O’Connell JJ. Premature Mortality in Homeless Populations: A Review of the Literature. Nashville: National

Health Care for the Homeless Council; 2005.

5. Salit SA, Kuhn EM, Hartz AJ, Vu JM, Mosso AL.

Hospitalization costs associated with homelessness in

New York City. N Engl J Med. 1998;338(24):1734---1740.

6. Gallagher TC, Andersen RM, Koegel P, Gelberg L.

Determinants of regular source of care among homeless

adults in Los Angeles. Med Care. 1997;35(8):814---830.

7. Kaiser Commission on Medicaid and the Uninsured.

Medicaid Coverage and Care for the Homeless Population: Key Lessons to Consider for the 2014 Medicaid

Expansion. Washington, DC: The Henry J. Kaiser Family

Foundation; 2012.

8. Baggett TP, O’Connell JJ, Singer DE, Rigotti NA. The

unmet health care needs of homeless adults: a national

study. Am J Public Health. 2010;100(7):1326---1333.

9. National Health Care for the Homeless Council.

Frequently Asked Questions About Health Care for the

Homeless. Nashville, TN: NHCHC; 2011.

10. Brickner PW, Scanlan BC, Conanan B, et al.

Homeless persons and health care. Ann Intern Med.

1986;104(3):405---409.

11. Brickner PW, Scharer LK, Conanan BA, Savarese M,

Scanlan BC. Under the Safety Net: The Health and Social

Welfare of the Homeless in the United States. New York,

NY: W.W. Norton & Co.; 1990.

12. McMurray-Avila M, Gelberg L, Breakey WR. Balancing act: clinical practices that respond to the needs of

homeless people. In: Fosburg LB, Dennis DL, eds. The

1998 National Symposium on Homelessness Research,

1998. Arlington, VA: US Department of Health and

Human Services and Department of Housing and Urban

Development; 1999:8-1---8-49.

13. O’Connell JJ. Dying in the shadows: the challenge

of providing health care for homeless people. CMAJ.

2004;170(8):1251---1252.

14. Brickner PW. Health Care of Homeless People. New

York, NY: Springer Publishing Co.; 1985.

15. Henry R. Determinants of appointment keeping by

veterans with hypertension [dissertation]. Los Angeles,

CA: University of California Los Angeles School of Public

Health; 2006.

16. O’Toole TP, Buckel L, Bourgault C, et al. Applying

the chronic care model to homeless veterans: can a

population approach to primary care impact utilization

and clinical outcomes? Am J Public Health. 2010;100

(12):2493---2499.

17. Shortt SE, Hwang S, Stuart H, Bedore M, Zurba N,

Darling M. Delivering primary care to homeless persons:

a policy analysis approach to evaluating the options.

Healthc Policy. 2008;4(1):108---122.

18. Ku L, Shin P, Jones E, Bruen B. Transforming Community Health Centers into Patient-Centered Medical

Homes: The Role of Payment Reform. New York, NY: The

Commonwealth Fund; 2011.

19. Health Resources and Services Administration.

Program Assistance Letter: HRSA Patient-Centered

Medical/Health Home Initiative. Washington, DC:

Health Resources and Services Administration; 2011.

20. Piette J, Holtz B, Beard A, et al. Improving chronic

illness care for veterans within the framework of the

Patient-Centered Medical Home: experiences from the

Ann Arbor Patient-Aligned Care Team Laboratory.

Behav Med Pract Policy Res. 2011;1(4):615---623.

21. Safran DG, Montgomery JE, Chang H, Murphy J,

Rogers WH. Switching doctors: predictors of voluntary

disenrollment from a primary physician’s practice. J Fam

Pract. 2001;50(2):130---136.

22. Roumie CL, Greevy R, Wallston KA, et al. Patient

centered primary care is associated with patient hypertension medication adherence. J Behav Med. 2011;34

(4):244---253.

23. Kim TW, Samet JH, Cheng DM, Winter MR, Safran

DG, Saitz R. Primary care quality and addiction severity:

a prospective cohort study. Health Serv Res. 2007;42

(2):755---772.

24. Levy BD, O’Connell JJ. Health care for homeless

persons. N Engl J Med. 2004;350(23):2329---2332.

25. Safran DG. Defining the future of primary care:

what can we learn from patients? Ann Intern Med. 2003;

138(3):248---255.

26. Starfield B, Cassady C, Nanda J, Forrest CB, Berk R.

Consumer experiences and provider perceptions of the

quality of primary care: implications for managed care.

J Fam Pract. 1998;46(3):216---226.

27. Committee on the Future of Primary Care for the

Institute of Medicine. Primary Care: America’s Health in

a New Era. Washington, DC: National Academy Press; 1996.

28. Starfield B. Primary Care: Balancing Health Needs,

Services, and Technology. New York, NY: Oxford University Press; 1998.

29. Sofaer S, Firminger K. Patient perceptions of the

quality of health services. Annu Rev Public Health. 2005;

26:513---559.

30. Wen CK, Hudak PL, Hwang SW. Homeless people’s

perceptions of welcomeness and unwelcomeness in

healthcare encounters. J Gen Intern Med. 2007;22(7):

1011---1017.

31. Martins DC. Experiences of homeless people in the

health care delivery system: a descriptive phenomenological study. Public Health Nurs. 2008;25(5):420---430.

32. O’Connell JJ, Oppenheimer SC, Judge CM, et al. The

Boston Health Care for the Homeless Program: a public

health framework. Am J Public Health. 2010;100

(8):1400---1408.

33a. Blue-Howells J, McGuire J, Nakashima J. Colocation of health care services for homeless veterans:

a case study of innovation in program implementation.

Soc Work Health Care. 2008;47(3):219---231.

33b. International Classification of Diseases, Ninth Revision, Clinical Modification. Hyattsville, MD: National

Center for Health Statistics; 1980. DHHS publication

PHS 80-1260.

34. Hays RD, Shaul JA, Williams VS, et al. Psychometric

properties of the CAHPS 1.0 survey measures. Consumer

Assessment of Health Plans Study. Med Care. 1999;37

(3 suppl):MS22---MS31.

RESEARCH AND PRACTICE

S338 | Research and Practice | Peer Reviewed | Kertesz et al. American Journal of Public Health | Supplement 2, 2013, Vol 103, No. S2
35. Safran DG, Kosinski M, Tarlov AR, et al. The Primary

Care Assessment Survey: tests of data quality and measurement performance. Med Care. 1998;36(5):728---739.

36. Nickasch B, Marnocha SK. Healthcare experiences

of the homeless. J Am Acad Nurse Pract. 2009;21(1):

39---46.

37. Davis E, Tamayo A, Fernandez A. “Because Somebody Cared about Me. That’s How It Changed Things”:

homeless, chronically ill patients’ perspectives on case

management. PLoS ONE. 2012;7:e45980.

38. Committee on Quality of Health Care in America

IoM. Crossing the Quality Chasm: A New Health System for

the 21st Century. Washington, DC: National Academy

Press; 2001.

39. Samejima F. Graded response model. In: Van der

Linden W, Hambleton RK, eds. Handbook of Item Response Theory. New York: Springer-Verlag; 2010:85---

100.

40. Kontopantelis E, Roland M, Reeves D. Patient

experience of access to primary care: identification of

predictors in a national patient survey. BMC Fam Pract.

2010;11:61.

41. Salisbury C, Wallace M, Montgomery AA. Patients’

experience and satisfaction in primary care: secondary

analysis using multilevel modelling. BMJ. 2010;341:

c5004.

42. Paddison C, Elliott M, Parker R, et al. Should

measures of patient experience in primary care be

adjusted for case mix? Evidence from the English General

Practice Patient Survey. BMJ Qual Saf. 2012;21(8):634---

640.

43. Gelberg L, Andersen RM, Leake BD. The Behavioral

Model for Vulnerable Populations: application to medical

care use and outcomes for homeless people. Health Serv

Res. 2000;34(6):1273---1302.

44. Kim TW, Kertesz SG, Horton NJ, Tibbetts N, Samet

JH. Episodic homelessness and health care utilization in

a prospective cohort of HIV-infected persons with alcohol

problems. BMC Health Serv Res. 2006;6:19.

45. Kertesz SG, Larson MJ, Cheng DM, et al. Need and

non-need factors associated with addiction treatment

utilization in a cohort of homeless and housed urban

poor. Med Care. 2006;44(3):225---233.

46. LaGory M, Ritchey FJ, Fitzpatrick K, Irwin J. A Needs

Assessment of the Homeless of Birmingham and Jefferson

County. Birmingham, AL: University of Alabama at

Birmingham; 2005.

47. Irwin J, LaGory M, Ritchey F, Fitzpatrick K. Social

assets and mental distress among the homeless: exploring

the roles of social support and other forms of social

capital on depression. Soc Sci Med. 2008;67(12):1935---

1943.

48. Mitchell CU, LaGory M. Social capital and mental

distress in an impoverished community. City Community.

2002;1(2):199---216.

49. Kertesz SG, Horton NJ, Friedmann PD, Saitz R,

Samet JH. Slowing the revolving door: stabilization programs reduce homeless persons’ substance use after

detoxification. J Subst Abuse Treat. 2003;24(3):197---

207.

50. Homeless Emergency Assistance and Rapid Transition. Housing (HEARTH) Act of 2009. 42 USC 11301.

United States of America; 2009.

51. Notice of Funding Availability (NOFA) for the

Collaborative Initiative to Help End Chronic

Homelessness. 2003. Available at: http://archives.hud.

gov/funding/2003/other/hmless.htm. Accessed August

2, 2013.

52. Idler EL, Kasl SV. Self-ratings of health: do they also

predict change in functional ability? J Gerontol B Psychol

Sci Soc Sci. 1995;50(6):S344---S353.

53. DeSalvo KB, Fan VS, McDonell MB, Fihn SD.

Predicting mortality and healthcare utilization with

a single question. Health Serv Res. 2005;40(4):1234---

1246.

54. Conrad KJ, Yagelka JR, Matters MD, Rich AR,

Williams V, Buchanan M. Reliability and validity of a

modified Colorado Symptom Index in a national homeless sample. Ment Health Serv Res. 2001;3(3):141---153.

55. Humeniuk R, Ali R. WHO ASSIST Phase II Study

Group. Technical Report: Validation of the Alcohol, Smoking and Substance Involvement Screening Test (ASSIST)

and Pilot Brief Intervention: A WHO Technical Report of

Phase II findings. Geneva, Switzerland: World Health

Organization Department of Mental Health and Substance Dependence; 2005.

56. Muthen LK, Muthen BO. Mplus: Statistical Analysis

With Latent Variables User’s Guide. Los Angeles, CA:

Muthen & Muthen; 2007.

57. Rizopoulos D. ltm: An R Package for Latent Variable

Modeling and Item Response Analysis. J Stat Softw.

2006;17(5):1---25.

58. National Center for Health Statistics. Health, United

States, 2011: With Special Feature on Socioeconomic

Status and Health. Hyattsville, MD: Centers for Disease

Control; 2012.

59. American Institutes for Research. How to Report

Results of the CAHPS Clinician & Group Survey. Princeton,

NJ: Robert Wood Johnson Foundation; 2010.

60. Trivedi AN, Matula S, Miake-Lye I, Glassman PA,

Shekelle P, Asch S. Systematic review: comparison of the

quality of medical care in Veterans Affairs and nonVeterans Affairs settings. Med Care. 2011;49(1):76---88.

61. Office of Quality SaV. VHA Facility Quality and

Safety Report. Washington, DC: Department of Veterans

Affairs; 2012.

62. Tessler R, Rosenheck R, Gamache G. Homeless

veterans of the mil-volunteer force: a social selection

perspective. Armed Forces Soc. 2003;29(4):509---524.

63. Hoffman C, Sered SS. Threadbare: Holes in America’s

Health Care Safety Net. Menlo Park, CA: The Henry A.

Kaiser Family Foundation; 2005.

64. Song J, Bartels DM, Ratner ER, Alderton L, Hudson

B, Ahluwalia JS. Dying on the streets: homeless persons’

concerns and desires about end of life care. J Gen Intern

Med. 2007;22(4):435---441.

65. National Committee for Quality Assurance. NCQA’s

PCMH 2011 and CMS Stage 1 Meaningful Use Requirements Appendix 2. Washington, DC: NCQA; 2011.

66. Buck DS, Rochon D, Davidson H, McCurdy S.

Involving homeless persons in the leadership of a health

care organization. Qual Health Res. 2004;14(4):513---

525.

RESEARCH AND PRACTICE

Supplement 2, 2013, Vol 103, No. S2 | American Journal of Public Health Kertesz et al. | Peer Reviewed | Research and Practice | S339