-----------------------------------------------------------

 

-----------------------------------------------------------

-----------------------------------------------------------

The Effect of Working Conditions 

on Patient Care: A Systematic Review Evidence-based Synthesis Program 

APPENDIX A. SEARCH STRATEGIES
 Staffing

Database: Ovid MEDLINE(R) 

1 exp Medical Errors/ 

2 (medical errors or medication errors or diagnostic 

errors).mp. 

3 quality of health care/ 

4 *safety/ or safety/st or safety management.mp. 

5 Iatrogenic Disease/ or iatrogenic disease.mp. 

6 quality assurance health care/ 

7 (patient safety or safety of patient$).mp. 

8 *treatment outcome/ 

9 Patient$.ti 

10 exp Physician-Patient Relations/
 

11 exp Patient Satisfaction/ 

12 or/1-11
 

13 workload/ or workload.mp. or overwork.mp. 

14 exp professional competence/ 

15 work schedule tolerance/ or teamwork.tw.
 

16 “Personnel Staffing and Scheduling”/ or personnel 

staffing.mp. 

17 Professional Autonomy/ or professional autonomy. 

mp. 

18 professional power.mp. 

19 exp Time Management/ 

20 or/13-19 

21 12and 20 

22 limit 21to (english language and humans) 23 limit 

22 to yr=”2000 -Current” 

24 exp Physicians/
 

25 exp Nurse Practitioners/
 

26 exp Physician Assistants/ 

27 or/24-26
 

28 23 and 27
 

29 limit 28 to (comment or editorial or letter or news) 

30 28 not 29
 

Workflow

Database: Ovid MEDLINE(R) 

1 exp Medical Errors/ 

2 (medical errors or medication errors or diagnostic 

errors).mp. 

3 quality of health care/ 

4 *safety/ or safety/st or safety management.mp. 

5 Iatrogenic Disease/ or iatrogenic disease.mp. 

6 quality assurance health care/ 

7 (patient safety or safety of patient$).mp. 

8 *treatment outcome/ 

9 Patient$.ti. 

10 exp Physician-Patient Relations/ 

11 exp Patient Satisfaction/ 

12 or/1-11 

13 exp Efficiency, Organizational/ 

14 exp “Task Performance and Analysis”/ 

15 exp Information Systems/ 

16 exp Electronic Health Records/ 

17 exp Equipment Design/ 

18 exp Equipment Safety/ 

19 Personnel Management/ or job performance.mp. 

20 exp User-Computer Interface/ 

21 exp Expert Systems/ 

22 (distraction or interruption).mp. 

23 multitask.mp. 

24 paging.mp. 

25 User-Computer Interface/ or human computer 

interactions.mp. 

26 exp “Referral and Consultation”/ 

27 or/13-26 

28 12 and 27 

29 limit 28 to (english language and humans) 

30 limit 29 to yr=”2000-Current” 

31 exp physicians/ 

32 exp nurse practitioners/ \33 exp physician 

assistants/ 

34 or/31-33 

35 30 and 34 

36 limit 35 to (comment or editorial or letter or news) 

37 35 not 36 

Organizational culture 

Database: Ovid MEDLINE(R) 

1	 exp Medical Errors/ 

2	 (medical errors or medication errors or diagnostic 

errors).mp. 

3	 quality of health care/ 

4	 *safety/ or safety/st or safety management.mp. 

5	 Iatrogenic Disease/ or iatrogenic disease.mp. 

6	 quality assurance health care/ 

7	 (patient safety or safety of patient$).mp. 

8	 *treatment outcome/ 

9	 Patient$.ti. 

10 exp Physician-Patient Relations/ 

11 exp Patient Satisfaction/ 

12 or/1-11 

13 exp Interprofessional Relations/ or exp 

Organizational Culture/ or professional culture.mp. 

14 organizational climate.mp. 

15 exp Leadership/ 

16 management style.mp. 

17 managerial style.mp. 

18 skill mix.mp. 

42
 
-----------------------------------------------------------

 

 

----------------------------------------------------------

 

 

The Effect of Working Conditions 

on Patient Care: A Systematic Review Evidence-based Synthesis Program 

19 exp Models, Organizational/ or shared leadership. 

mp. or exp Organizational Innovation/ 

20 open door policies.mp. 

21 exp Management Quality Circles/ 

22 exp Institutional Management Teams/ 

23 or/13-22
 

24 12 and 23
 

25 limit 24 to (english language and humans) 

26 limit 25 to yr=”2000 -Current” 

27 exp physicians/ 

28 exp nurse practitioners/ 

29 exp physician assistants/ 

30 or/27-29
 

31 26 and 30
 

32 limit 31 to (comment or editorial or letter or news) 

33 31 not 32
 

Physical environment 

Database: Ovid MEDLINE(R) 

1	 exp Medical Errors/ 

2	 (medical errors or medication errors or diagnostic 

errors).mp 

3	 quality of health care/ 

4	 *safety/ or safety/st or safety management.mp. 

5	 Iatrogenic Disease/ or iatrogenic disease.mp. 

6	 quality assurance health care/ 

7	 (patient safety or safety of patient$).mp. 

8	 *treatment outcome/ 

9	 Patient$.ti. 

10 exp Physician-Patient Relations/ 

11 exp Patient Satisfaction/ 

12 or/1-11 

13 exp Air Pollution/ 

14 exp Air Pollution, Indoor/ 

15 exp Light/ or exp Lighting/ or indoor lighting.mp. 

16 exp Acoustics/ 

17 exp Noise/ or indoor noise.mp. 

18 exp “Interior Design and Furnishings”/ 

19 exp Humidity/ 

20 exp Ventilation/ or exp Temperature/ or indoor 

temperature.mp. or exp Environmental Monitoring/ 

21 exp “Facility Design and Construction”/ or clinic 

design.mp 

22 human factors engineering.mp. 

23 exp Environment Design/ or facility environment. 

mp. 

24 or/13-23 

25 12 and 24 

26 limit 25 to (english language and humans) 

27 limit 26 to yr=”2000-Current” 

28 exp physicians/ 

29 exp nurse practitioners/ 

30 exp physician assistants/ 

31 or/28-30 

32 27 and 31 

33 limit 32 to (comment or editorial or letter or news) 

34 32 not 33 

Team 

Database: Ovid MEDLINE(R) 

1	 exp Patient Care Team/ or team-based.mp. 

2	 practice based care team.mp. 

3	 shared case.mp. 

4	 exp Interprofessional Relations/ or shared care.mp. 

5	 collaborative care.mp. 

6	 multidisciplinary care teams.mp. 

7	 multidisciplinary care team.mp. 

8	 6 or 7 

9	 or/1-8 

10 exp Medical Errors/ 

11 (medical errors or medication errors or diagnostic 

errors).mp. 

12 quality of health care/ 

13 *safety/ or safety/st or safety management.mp. 

14 Iatrogenic Disease/ or iatrogenic disease.mp. 

15 quality assurance health care/ 

16 (patient safety or safety of patient$).mp. 

17 *treatment outcome/ 

18 Patient$.ti. 

19 exp Physician-Patient Relations/ 

20 exp Patient Satisfaction/ 

21 or/10-20 

22 9 and 21 

23 limit 22 to (english language and humans) 

24 limit 23 to yr=”2000 -Current” 

25 exp Physicians/ 

26 exp Nurse Practitioners/ 

27 exp Physician Assistants/ 

28 or/25-27
 

29 24 and 28
 

30 limit 29 to (comment or editorial or letter or news) 

31 29 not 30
 

43
 
 

The Effect of Working Conditions 

on Patient Care: A Systematic Review Evidence-based Synthesis Program 

APPENDIX B. CRITERIA USED IN QUALITY ASSESSMENT OF 

NON-RANDOMIzED STUDIES 

We evaluated each non-randomized trial based on the five elements below. To be considered 

low risk of bias for any element, a “yes” response was required for each of the questions (a, b, 

c) pertaining to the element, if applicable. Plots were developed to show the percent of the nonrandomized trials in each area (human resources practices, organizational culture, and physical 

environment) that were assigned a yes (met criteria) or no (failed to meet criteria) for each 

element. 

1)	 Population 

a.	 Is the sample representative of the population of interest? 

b.	 Did researchers apply inclusion/exclusion criteria uniformly to all comparison groups 

and is the selection of the comparison group appropriate? 

c.	 Is the sampling method appropriate (i.e. appropriate database or sample for research 

question, adequate response rate for survey studies, etc.)? 

2)	 Outcomes 

a.	 Are important outcomes assessed and reported (i.e. not just intermediate or surrogate 

outcomes)? 

b.		 Was the length of follow-up appropriate for the research questions (consider benefits 

and harms)? 

c.	 Is the impact of loss to follow-up (or differential loss to follow-up) considered in the 

analysis? 

3)	 Measurement 

a.	 Are outcome, predictor and covariates assessed in the same way for everyone? 

b.	 Is this blinded such that, for example, a person’s exposure status would not be known 

at the time outcome status was assessed? This is where recall bias and other types of 

differential assessment come into play. 

c.	 Are the tools used to assess exposures and outcomes accurate and reliable (i.e., are 

standard measures used)? 

4)	 Confounding 

a.		 Are the statistical methods and study design adequate for minimizing confounding? 

b.	 Aside from the exposure of interest, are groups balanced in terms of factors that might 

bias the exposure and outcome association? 

c.	 Are the appropriate confounding factors included in the analysis? 

5)	 Intervention (if applicable) 

a.	 Is the intervention clearly described and transferrable (i.e. could someone else repeat 

this study with different staff and patients and get similar results)? 

44
 
ns on Patient Care: A Systematic Review Evidence-based Synthesis Program 

R REVIEW COMMENTS/AUTHOR RESPONSES

RESPONSE 

 and methods for this review clearly described? 

No response needed No response needed No response needed No response needed 

tualized with appropriate questions to guide the review. 

ature search and review of the literature. 

Thank you. 

e 3 areas specifically: HR, organizational culture, and 

warrants some enhancement. Further, the definitions 

gories seems somewhat arbitrary. For example, 

re a appropriate roles/functions for team based care 

onal culture? Regarding outcomes, you use the term 

lear that you really mean to include all quality metrics 

es such as admissions and ED utilization. What about 

hronic disease outcomes such as glycemic control etc?? 

d these as well. 

We acknowledge that these categorizations are arbitrary, but we do not think that how we’ve organized this (by the categorizations that we’ve used) undermines our presentation of the evidence, which in most cases is lacking. Our main rationale for using these categorizations is that we wanted to build on the previous similar AHRQ report, but because of the substantial overlap collapsed a few of the categories. Nonetheless, we’ve inserted a disclaimer about this categorization. Regarding patient safety, this is a valid point. We agree that there may be some overlap with patient safety and effectiveness (which we point out in the report), where the latter would include “performance metrics such as chronic disease outcomes such as glycemic control etc.” We have added some discussion to clarify this. 

bias in our synthesis of the evidence? 

No response needed No response needed No response needed No response needed 

m for choosing studies. Excellent use of criteria for 

atic reporting of findings. 

Thank you. No response needed 

r unpublished studies that we may have 

No response needed 

ture review was conducted; however I have not done 

o know if there are additional references 

No response needed 

45
ns on Patient Care: A Systematic Review Evidence-based Synthesis Program 

RESPONSE Thank you for the additional suggested articles. We’ve pulled all of these 

zer M, et al. Physician, Practice, and Patient references and discuss them here: 

ary Care Physician Physical and Mental Health: Results 1. Williams et al. – this article does not have any patient outcomes that 

vey. Health Serv Res 2002;37(1):121-143. we examine so it does not meet our inclusion criteria. However, the 

e DM, et al. Organizational climate, staffing, and safety study does relate well to some of our discussion of provider outcomes, 

dlestick injuries and near-misses in hospital nurses. Am so we will make sure this paper is added/discussed in that section. 

07-216 2. Clarke et al. – this article does not meet our inclusion criteria because 

et al. Hospital nurse staffing and patient mortality, nurse it is about needlesticks in hospital settings and deals with nurses’ 

 JAMA 2002;288:1987–1993. workplace condition (while we focus on MDs, PAs, and NPs only). 

attke S, et al. Nurse-staffing levels and the quality of 3. Aiken et al - this article does not meet our inclusion criteria because 

 Medicine 2002;346(22):1715–1722. it deals with hospital settings and nurses’ workplace condition (while we 

man P, et al. Organizational Climate of Staff Working focus on MDs, PAs, and NPs only in primary care settings). 

grative Model. Advances in Patient Safety: From 4. Needleman et al – same as #3. 

olumes 1-4, AHRQ Publication Nos. 050021 (1-4). 5. Stone et al – we will add this citation to our background/framework 

thcare Research and Quality, Rockville, MD. http:// section. 

Volume 2, Concepts & Methodology, pp 467-481 We only cite the first Institute of Medicine report. We have changed the 

ses do not appear in the citations, but the first is citation from Kohn et al. to Institute of Medicine. 

referenced. 

s Human: Building a Safer Health System. Washington, 

ces; 2000. 

g the Quality Chasm: A New Health System for the 21st 

onal Academy of Sciences; 2001. 

ee on the Work Environment for Nurses and Patient 

Transforming the Work Environment of Nurses. Ann 

National Academy of Sciences; 2004. 

m not aware of them in the prescribed area of interest. 

 effects of working conditions and workload of nurses. 

No response needed 

hat I am familiar with has made it into this report. No response needed 

r comments 

the evidence of an association of working conditions 

en lacking, alternative hypotheses are not explicitly 

h professionals are capable of “buffering” pateints from 

onditions, leading to null or mixed effects. 

We have added this caveat. Thank you. 

46
ns on Patient Care: A Systematic Review Evidence-based Synthesis Program 

clusions. The first sentence seems to indicate greater 

th greater workloads and less control over work tasks. 

d that greater workloads and less control over work 

er stress, burnout, and job satisfaction.” I would suggest 

 intent. 

or Future Research. This section was weak in 

port. The content is very general with little specific 

y areas for future research. Given the focus on general 

et constraints what we are facing in the VA, the authors 

 specific areas or research questions that need 

or tough times ahead. Are there specific practices or 

sical environment that their findings would point to as 

 

 review this report. I appreciated the detail and the clarity 

k is important as we advance this area of knowledge. 

Re pg. 5, 5th paragraph: we’ve edited this. 

Re pg. 38: we’ve added some discussion. 

uch overlap between HR, org culture, and physical This is a valid point, but we used fairly exhaustive lists of terms for all 

f some studies may have been overlooked because of these vague constructs, which may be unclear in the main part of the 

s. This review will be helpful more to point out the text (though can be seen in our appendices with search terms). We will 

re, and the lack of clear relationship observed thus far add some discussion about this. 

and function and specific outcomes. 

mendations on how this report can be revised to 

st implementation needs. 

appendices short checklists or worksheets, designed 

rs, safety professionals, and worker teams to help 

ons that can adversely impact both employee health 

e and b) develop interventions to improve those 

d standard hazard evaluations, because it would flag 

ated with patient outcomes. Many of these would be 

 hazard evaluations do not normally address. These 

spital interventions to address systems-level problems 

We have forwarded your suggestion to the topic nominator. . 

es to the healthcare personnel it represents in the clinic 

ed in the acute care settings. Given that the majority of 

prised of nurses in acute care settings, I would hope 

onducted for acute care and include nurses as part 

dies related to nursing impact are being done but a 

 conducted and might be valuable as healthcare strives 

elivery of services and improvement in outcomes. 

Thank you. Additional topics (such as acute care settings) can be nominated at the VA ESP Web site: http://www.hsrd.research.va.gov/ publications/esp/ 

mportant studies evaluating effect and impact of PCMH 

ed here because did not specifically include the three 

The evidence group at the Minneapolis VA has reviewed the literature on PCMH for another VA program. To our knowledge there are no additional published reports of PCMH interventions. 

47
ns on Patient Care: A Systematic Review Evidence-based Synthesis Program 

ENCE TABLES 

ion of Human Resources Practices Studies – United States 

Sample 

Study design 

Working 

Conditions 

Studiedb 

Patient/Provider 

Outcomes Studiedc Study Qualityd Patientsa Providers/Clinics 

nience sample of 218 

patients 

e 

0% 

Ethnicity: 86% Mexican 

3% 25-32 years 

Convenience sample of 15 licensed NPs from 11 urban clinics 

Cross-sectional ii. Training v. Patient Satisfaction 

with Provider 

1/4 

2,758 MDs (62% response Cross-sectional vi. Electronic i. Quality of Care 3/4 rate) from the 2007 AMA file Medical Records iv. Medication Errors 

e 

132 MDs with at least 100 months working in hospital affiliated practices in urban area 

Cross-sectional iv. Hours i. Quality of Care 

v. Patient Satisfaction with Provider 

2/4 

ximately 1,500 diabetes VD patients from 2005-

e 

NR 

Ethnicity: 7-12 % 

te 

ears): 61 (diabetes), 70 

15 Kaiser Permanente clinics: 167 PCPs with at least 20 diabetes patients 143 PCPs with at least 20 CVD patients 

Retrospective cohort 

vi. Electronic Medical Records 

i. Quality of Care 4/4 

tients 

e 

~30% 

Ethnicity: NR 

2 % 18-50 years 

54 MDs and PAs at 7 urban community clinics 

Pre-post of repeated crosssections 

i. Training v. Patient Satisfaction 

with Provider 

3/5 

48
ns on Patient Care: A Systematic Review Evidence-based Synthesis Program 

Sample 

Study design 

Working 

Conditions 

Studiedb 

Patient/Provider 

Outcomes Studiedc Study Qualityd Patientsa Providers/Clinics 

patients 

e 

31% 

Ethnicity: 62% White, 

ack 

ears): 60 

119 clinics in 5 regions (urban & rural): 218 general internists and 204 family practitioners 

Cross-sectional iii. Workload 

v. Autonomy 

i. Quality of Care iii. Non-medication Treatment Errors 

3/4 

patients 

e 

25% 

Ethnicity: 1% White, 9% 

85% Hispanic 

ears): 44 

5 urban clinics Randomized trial i. Skills v. Patient Satisfaction 

with Provider 

Allocation concealment: No Blinding: Providers were blinded Intention to treat analysis: No Withdrawals adequately described: Yes 

71,980 PCPs with at least Cross-sectional iv. Workload i. Quality of Care 2/4 10 Medicare patients (using Medicare data) 194 family practitioners and Cross-sectional iv. Hours i. Quality of Care 3/4 general internists from 25 v. Patient Satisfaction out-patient clinics of a single with Provider medical group in western Washington 

 Kaiser Permanente 

a patients (60% 

se rate) 

e 

39% 

Ethnicity: NR 

6% 18-54 years 

139 MDs, 63 PA/NPs Cross-sectional i. Skills v. Patient Satisfaction 

with Provider 

4/4 

49
ns on Patient Care: A Systematic Review Evidence-based Synthesis Program 

Sample 

Study design 

Working 

Conditions 

Studiedb 

Patient/Provider 

Outcomes Studiedc Study Qualityd Patientsa Providers/Clinics 

 referrals 

e 

33% 

Ethnicity: 54% non-white 

0% 21-39 years 

10 PC clinics Pre-post of 

repeated crosssections 

vi. Electronic Medical Records 

i. Quality of Care 5/5 

e 

21 NYU Student Health Center clinicians (14 MDs, 6 NPs, 1 PA) 

Pre-post ii. Training i. Quality of Care 

v. Patient Satisfaction with Provider 

4/5 

 report the following descriptive statistics (means/percents) on the main patient sample analyzed: age, gender, race, and veteran status. “NR” 

orted in the study and “N/A” means the statistics were not applicable to the sample studied. 

 resources practices, noting that each construct may be measured differently across studies: 

s or Computerized Systems 

and provider outcomes (vii-viii), noting that each construct may be measured differently across studies: 

ffectiveness or Access 

Errors 

cation Treatment Errors 

n Treatment Errors 

ovider 

nic/Care 

following ways. For non-randomized studies, we assessed the risk of study bias on the following dimensions: population (e.g., representative, 

 outcomes (important outcomes assessed and measured, appropriate follow-up), measurement (variables uniformly assessed, blinded, construct 

n and methods minimize confounding) and whether the intervention can be replicated if applicable. Study quality for these studies is reported as 

sk was assessed as low) out of the total possible dimensions evaluated for risk. For randomized studies, we assessed study quality based on the 

can Medical Association, CVD = cardiovascular disease, GP = general practitioner, MD = physician, N/A = not applicable, NP = Nurse practitioner, 

Assistant, PC = primary care, PCP = primary care provider 

50
ns on Patient Care: A Systematic Review Evidence-based Synthesis Program 

ion of Human Resources Practices Studies – Europe 

Sample 

Study design Working Conditions Studiedb Patient/Provider Outcomes Studiedc Study Qualityd Patientsa Providers/Clinics 

 randomly selected 

nts (49% response rate) 

ple 

 41% 

: NR 

41% 16-44 years 

22 practices (55% response rate) in mostly urban areas 

Cross-sectional i. Skills v. Patient Satisfaction 

with Provider vi. Patient Satisfaction with Practice/Care 

2/4 

 patients (38% response 

ple 

60 randomly selected practices across 6 districts in England (80% response rate) 

Retrospective cohort 

iii. Workload i. Quality of Care 1/4 

 patients (66% response 

ple 

54 volunteer practices (27% response rate) in urban areas 

Cross-sectional iii. Workload i. Quality of Care 2/4 

atients 

ple 

 29% 

: NR 

years): 49 

41 GPs (23% response rate) Cross-sectional iii. Workload v. Patient Satisfaction 

with Provider 

3/4 

 patients Convenience sample of Randomized i. Skills vi. Patient Satisfaction Allocation 

12 NPs and 50 GPs in 15 controlled trial with Provider concealment: Yes 

ple 

 39% 

: NR 

52 % 16 to 45 years 

clinics Blinding: No 

(reported to be impossible for this study) Intention to treat analysis: No Withdrawals adequately described: Yes 

51
ns on Patient Care: A Systematic Review Evidence-based Synthesis Program 

Sample 

Study design Working Conditions Studiedb Patient/Provider Outcomes Studiedc Study Qualityd Patientsa Providers/Clinics 

atients (44% response 

ple 

20 GPs (41% response rate) Cluster 

randomized crossover trial 

ii. Training v. Patient Satisfaction 

with Provider 

Allocation concealment: Yes Blinding: Yes (assessors of clinic visits) Intention to treat analysis: No Withdrawals adequately described: No 

atients (66% response 

ple 

26 GPs in England Longitudinal 

(cohort of GPs, repeated crosssections of patients) 

iv. Hours v. Patient Satisfaction 

with Provider vi. Patient Satisfaction with Practice/Care 

1/4 

 patients 

ple 

 46% 

: NR 

51% 16 to 45 years 

1,075 GPs Cross-sectional iii. Workload vi. Patient Satisfaction 

with Practice/Care 

4/4 

atients (50% response 

ple 

 40% 

: NR 

years): 63.9 

30 GPs, 5 NPs, in 20 clinics Cross-sectional i. Skills v. Patient Satisfaction 

with Provider 

2/4 

 patients 

ple 

 42% 

: NR 

47% 16 to 45 years 

NR Cross-sectional iii. Workload v. Patient Satisfaction 

with Provider 

4/4 

52
ns on Patient Care: A Systematic Review Evidence-based Synthesis Program 

Sample 

Study design Working Conditions Studiedb Patient/Provider Outcomes Studiedc Study Qualityd Patientsa Providers/Clinics 

46 hospitalizations of 

s age 65+ 

ple 

 NR 

: NR 

years): 77 for men, 81 

omen 

34 health districts in Madrid Cross-sectional 

ecological 

iii. Workload i. Quality of Care 4/4 

ple stats NR 276 MDs (62% response 

rate) with at least 49 patient surveys each 

Cross-sectional v. Autonomy vi. Patient Satisfaction 

with Practice/Care 

1/4 

 patients (84% response 

ple 

 39% 

: 98% white 

years): 52 

150 GPs in 27 practices in England 

Cross-sectional iii. Workload v. Patient Satisfaction 

with Provider vi. Patient Satisfaction with Practice/Care 

4/4 

Table 1 

ion of Human Resources Practices Studies – Outside of US or Europe 

Sample 

Study design Working Conditions Studiedb Patient/Provider Outcomes Studiedc Study Qualityd Patientsa Providers/Clinics 

 prescriptions 

e 

57% 

NR; ears): 34 

680 primary health clinics from 40 rural counties 

Cross-sectional iii. Workload iv. Medication Errors 4/4 

51 MDs who participated in a remedial professional development program (RPDP) 

Pre-post ii. Training i. Quality of Care 3/5 

Table 1 

53 
ns on Patient Care: A Systematic Review Evidence-based Synthesis Program 

of Care Outcomes - Human Resources Practices Studies 

Measurea 

Access Effectiveness 

Measured as: Main Finding Measured as: Main Finding 

NR NR Physician response to: has a) 90% in full system vs. 75% in 

– gives the EMR ever helped to: basic system; (p=0.004) 

ders for a) alert to critical lab value b) 69% in full system vs. 41% in 

 care, b) provide preventive care basic system (p<0.001) 

ests vs. c) order a critical laboratory c) 68% in full system vs. 36 in 

m – no test basic system. (p<0.001) 

ability or d) order a genetic test d) 17% in full system vs. 8% in 

 support basic system (p=0.03) 

me (PT) 

T) 

NR NR Compliant with quality 

measure: whether 70% of patients had appropriate screening for Pap smear, mammography, and cholesterol 

80% of PT PCPs versus 75% of FT PCPs were compliant (p-value = 0.04) 

nic tool NR NR “Care score” based on % of After implementation, diabetes 

are gaps 

 

care recommendations met by PCPs per member month (out of 100) 

care score increased by 7.64 (p<0.001) and CVD care score increased by 5.10 (p<0.001) 

me 

ent/per 

office (0/1) 

work 

scale (0/1) 

NR NR 3 quality indices based on 

management of 3 chronic conditions: a) hypertension b) diabetes c) heart failure 

Greater time pressure yielded slightly lower quality. A chaotic office had no effect on quality. Having greater work control resulted in greater quality. 

Medicare 

NR NR a) % of appropriate women 

who get mammograms b) % of diabetics who receive hemoglobin A1c test c) preventable hospitalization rate 

Providers with at least a) 328 women, b) 438 diabetics, and c) 19,069 patients are needed to detect a 10% difference in quality of care of Medicare patients relative to the national mean 

uous 

rs (3 to 35 

NR NR a) % of patients receiving 

cancer (Pap smear and mammography) screening b) % of patients receiving recommended diabetes care 

a) Cancer screening coefficient: -0.07 (p=0.01) b) Diabetes management coefficient= -0.11 (p=0.008) 

onic 

Getting a specialty appointment scheduled from a referral (0/1) 

OR of getting a specialty appointment scheduled increased by 4.32 (p <0.001) after implementation 

NR NR 

54
ns on Patient Care: A Systematic Review Evidence-based Synthesis Program 

Measurea 

Access Effectiveness 

Measured as: Main Finding Measured as: Main Finding 

skills 

NR NR Chart Audits for documented 

risk screenings of: a) smoking b) depressed mood c) anhedonia d) sexual activity e) drinking alcohol 

Mantel-Haenszel RRs: a) 1.65 (p=0.03) b) 1.40 (p=0.04) c) 1.47 (p=0.01) d) 1.73 (p=0.002) e) 1.77 (p=0.04) 

 (amount 

each 

Score based on guideline concordant care for three conditions: a) adult asthma b) angina c) type 2 diabetes mellitus 

Mean unadjusted differences between scores of practices with 10+ intervals between appointments and those with 5 minute intervals: a) adult asthma – 21.6 (p <0.001) b) angina – 10.2 (p=0.002) c) type 2 diabetes – 10 (p=0.028) 

Two measures created based on patient report of how quickly usually seen after appointment request: a) See doctor the same or next day (0/1) b) See doctor within 2-3 days (0/1) 

Correlations: a) -0.37(p=0.007) b) -0.21 (p=0.133) 

NR NR Rate of Ambulatory Care 

Sensitive Hospitalizations (ACSH) 

Each additional patient per workday was associated with 6% to 7% higher relative rate of ACSH (p<0.001) 

ROPE 

icipation Expert physician peer review a) 46% of providers improved in 

ofessional of medical records on: clinical investigation (p<0.001) 

ogram a) clinical investigation b) 29% improved in diagnostic 

b) diagnostic accuracy accuracy (p=0.01) and c) treatment and follow-up c) 36% improved in treatment and 

follow-up (p<0.001) 

human resources practices: 

s or Computerized Systems 

vascular disease, EMR = electronic medical record, GP = general practitioner, MD = physician, NP = nurse practitioner, NR = not reported, NS = 

ds ratio, PA = physician assistant, PCP = primary care provider, RR = relative risk 

55
ns on Patient Care: A Systematic Review Evidence-based Synthesis Program 

afety Outcomes – Human Resources Practices Studies 

& 

Diagnostic Errors Non-Medication Treatment Errors Medication Errors 

Measured as: Main Finding Measured as: Main Finding Measured as: Main Finding 

 

der 

 

cal 

 

NR NR NR NR Physician report of 

whether EMR ever helped: 1) prevent drug 

allergy 

2) prevent 

dangerous medication interaction 

1) 80 vs. 66% of MDs in full vs. basic report system helped with drug allergies (p=0.01) 2) 71 vs. 54% of MDs in full vs. basic report system prevented dangerous interactions (p=0.002) 

time 

per 

work 

cale 

NR NR Score based on 

chart audits to gauge missed treatment opportunities, inattention to behavioral factors, and guideline nonadherence (0/100) 

No significant effect of workload on prevention, hypertension or diabetes management errors. Having more autonomy resulted in a lower total error score (more errors) (coefficient = -2.80, (-5.72, 0.12). 

NR NR 

ROPE 

onth 

NR NR NR NR Polypharmacy 

(Rx’s with 5 or more drugs) per 100 patient-visits/month (0/1) 

OR of Polypharmacy w/ higher workload versus less workload = 1.70 [1.26, 2.29] 

human resources practices: 

s or Computerized Systems 

vascular disease, EMR = electronic medical record, GP = general practitioner, MD = physician, NP = nurse practitioner, NR = not reported, NS = 

ds ratio, PA = physician assistant, PCP = primary care provider, RR = relative risk 

56
ns on Patient Care: A Systematic Review Evidence-based Synthesis Program 

atisfaction Outcomes – Human Resources Practices Studies 

 Measure 

Patient Satisfaction with Provider Patient Satisfaction with Practice or Care 

Measured as: Main Finding Measured as: Main Finding 

eipt 

etence 

Patient Satisfaction Questionnaire (PSQ-III) 

Patient satisfaction positively correlated with NP’s culture competence training (r=0.32, p-value=NR) 

NR NR 

me (PT) 

) 

% of patients surveyed rating PCP as “excellent” or “good” 

FT = 92%, PT = 95% (p=0.13) 

NR NR 

shop on 

 effectively 

Patient reported satisfaction scaled from 1 (better) to 5 (worse) based on 30 items 

Overall satisfaction: Pre-test= 1.12 Post-test = 1.14 (p = NS) 

NR NR 

Satisfaction mean score measured by a 15 item satisfaction survey (5-point scale) 

Overall Satisfaction Baseline: MD =4.6; NP = 4.59 (p= 0.89) 6 month F/U: MD = 4.46; NP = 4.45 (p=0.87) 

NR NR 

s.

P 

Practitioner interaction (5 items) 

1.16 (p<0.05) times more likely to be satisfied with practitioner interaction when seeing a PA/NP vs. an MD 

Care access (4 items) No significant difference 

satisfaction with care access whether patient saw an MD vs. PA/NP 

uous mea

intment 

ours) 

Patient satisfaction = excellent 

Coefficient: -0.05 (p=0.21) NR NR 

skills 

10 point item on satisfaction with patient-provider communication1 

No change in patient satisfaction after training 

s.

NR NR Survey question on satisfaction 

with last visit dichotomized to be equal to one if patient reports “excellent” or “very good” satisfaction and 0, otherwise 

No significant difference in satisfaction except patients who saw a NP were more satisfied with the amount of time spent with provider than those who saw a GP (p<0.05) 

57
ns on Patient Care: A Systematic Review Evidence-based Synthesis Program 

 Measure 

Patient Satisfaction with Provider Patient Satisfaction with Practice or Care 

Measured as: Main Finding Measured as: Main Finding 

0 

6 point survey question on how satisfied with doctor you visited dichotomized to be equal to one if patient reports “very satisfied” and 0, otherwise 

No significant effect of GP listsize on patient satisfaction 

NR NR 

s.

10 point scale (details not reported) on overall patient satisfaction 

No significant difference in patient satisfaction across GP vs. NP patients (p=0.83) 

NR NR 

n making 

ation (RC) 

Patient satisfaction with the decision made (single item) 

No significant effect of either training on satisfaction: SDM coefficient = 0.1 (p=NS) RC coefficient = 0.5 (p=NS) 

NR NR 

n call” or 

General Satisfaction subscale on Consultant Satisfaction Score (CSQ) 

Visits surrounding “On call” = 75.6 Visits surrounding “Off duty” 77.1 (p=NS) 

Professional Care subscale on CSQ 

Visits surrounding “on call” =75.3 Visits surrounding “Off duty” =76.8 (p=NS) 

ns per 

P’s list 

NR NR Patient response to 

a) how satisfied with wait time to get an appointment (4 point scale) b) satisfaction with amount of time the GP spent (4 point scale) 

Probit coefficients: a) 0.938 (p < 0.05) b) 0.055 (p=0.13) 

.

Overall satisfaction using the “Chronically ill patients evaluate general practice” scale (6 point scale) 

Satisfaction with: a) GP = 4.1 b) NP = 4.4 (p = 0.03) 

NR NR 

han stated 

han stated 

Satisfaction (5 point scales) with a) doctor taking questions/ problems seriously b) getting a referral c) length of time with doctor 

Longer listsize than stated adjusted ORs: a) 2.0 [0.84 , 4.75] b) 1.03 [0.68, 1.57] c) 0.84 [0.62, 1.16] Shorter listsize than stated adjusted ORs: a) 0.41 [0.23,0.72] b) 0.48 [0.33,0.72] c) 0.63 [0.44, 0.92] 

Satisfaction (5 point scales) with a) confidence in treatment prescribed b) waiting time 

Longer listsize than stated adjusted ORs: a) 2.17 [0.98,4.82] b) 0.66 [0.51, 0.84] Shorter listsize than stated adjusted ORs: a) 0.46 [0.27, 0.78] b) 1.67 [1.17, 2.39] 

on the 

ent in 

e Index 

NR NR Patient rating of 

a) how treated by receptionists b) length of time you have to wait (higher is better) 

Correlations (r): a) -0.15 (p=0.02) b) -0.21 (p<0.01) 

58
ns on Patient Care: A Systematic Review Evidence-based Synthesis Program 

 Measure 

Patient Satisfaction with Provider Patient Satisfaction with Practice or Care 

Measured as: Main Finding Measured as: Main Finding 

0 patients) 

Overall satisfaction (7 point scale) 

Coefficient = 0.01 (p=0.32) Satisfaction with: 

a) ability to get an appointment (6 point scale) b) access (0 to 100 scale created from 6 questions about contacting practice, making an appointment) 

Coefficients a) 0.13 (p=0.001) b) 0.68 (p=0.25) 

human resources practices: 

s or Computerized Systems 

ascular disease, GP = general practitioner, MD = physician, NP = nurse practitioner, NR = not reported, NS = not statistically significant, OR = odds 

 = primary care provider, RR = relative risk 

ion of Organizational Culture Studies 

Sample Study design Working Conditions Studied Patient/ Provider Outcomes Studied Study Quality Patients Providers/Clinics 

on (n=12) 

=8) 

% 

% white, 35% black 

ge (years): 

e = 49, Usual care 

NR Case-control vii. Team-based care ii. Quality of Care 

-Effectiveness vii. Patient Satisfaction with Care 

0/5 

59
ns on Patient Care: A Systematic Review Evidence-based Synthesis Program 

Sample Study design Working Conditions Studied Patient/ Provider Outcomes Studied Study Quality Patients Providers/Clinics 

male veterans (62% 

dents were from 

8 Veterans Affairs Medical Centers in 3 

Cross-sectional (survey) 

ix. Care environment (women’s clinic vs. 

vii. Patient Satisfaction with Care 

1/5 

clinics, 38% from states traditional primary care 

 primary care) clinic) 

% white, 10% black, 

%): 100 

s): 58.3 

2% 

% white. 46% merican, 4% other 

NR Cluster-randomized 

controlled trial 

vii. Team-based care vii. Patient Satisfaction 

with Care 

Allocation concealment: No Blinding: No Intention to treat analysis (ITT): Yes 

s): 77.6 Withdrawals/ dropouts 

adequately described: Yes 

ovided baseline 

N/A Prospective cohort vii. Team-based care Patient: 

ii. Quality of Care -

1/5 

ved psych services 

Effectiveness Provider: 

% 

s): 83.6% (over 25) 

% White, 6% Other 

Coverage for psych 

3.8% No, 32.3% 

% Don’t Know 

ation: 43% Rural, 

n 

Physician satisfaction 

60
ns on Patient Care: A Systematic Review Evidence-based Synthesis Program 

Sample Study design Working Conditions Studied Patient/ Provider Outcomes Studied Study Quality Patients Providers/Clinics 

0 NR Pre-post viii. PCMH ii. Quality of Care -

Effectiveness 

2/5 

7% 

s): 73.8 

ns/1000 members/ 

6 

ions/ 

: 46 

NR Randomized vii. Team-based care ii. Quality of Care -

Allocation concealment: 

3% 

controlled trial Effectiveness No 

Blinding: Yes 

s): 71.2 Intention to treat analysis 

(ITT): Yes Withdrawals/ dropouts adequately described: Yes 

119 clinics in 5 regions (urban & rural) 

Cross-sectional x. Clinic values ii. Quality of Care -

Effectiveness 

3/4 

% 

nicity: 62% White, 

k 

s): 60 

218 general internists and 204 family practitioners 

04 N= 82 

83% Response rate 

Prospective pre-post 

viii. PCMH Patient: 

ii. Quality of Care -

2/5 

nic Effectiveness 

 Sample 

l clinics 

0) 

t attendance (%): 

Male: 16.3% vi. Patient Satisfaction 

with Provider vii. Patient Satisfaction with Care 

self-management 

orkshops (%): 0.02 

k assessment 

n (%): 1.8 

utreach (%): 1.1 

cy/urgent care 

(%): 6.5 

Provider: Staff Burnout 

61
ns on Patient Care: A Systematic Review Evidence-based Synthesis Program 

Sample Study design Working Conditions Studied Patient/ Provider Outcomes Studied Study Quality Patients Providers/Clinics 

2% 

 

s): 74 

gth of time with 

 10.9 years 

on (Pharmacist 

n=431 

e: n=458 

N=48 agreed to participate Age: NR Male: 67% Race: NR Years since graduation: 21.7 Intervention (Pharmacist consult): n=24 Usual Care: n=24 

Randomized controlled trial 

vii. Team-based care v. Medication Errors Allocation concealment: 

No Blinding: No Intention to treat analysis (ITT): No Withdrawals/ dropouts adequately described: Yes 

 report the following descriptive statistics (means/percents) on the main patient sample analyzed: age, gender, race, and veteran status. “NR” 

orted in the study and “N/A” means the statistics were not applicable to the sample studied. 

zational culture components: 

Home (PCMH) 

and provider outcomes (vii-viii), noting that each construct may be measured differently across studies: 

ffectiveness or Access 

Errors 

cation Treatment Errors 

n Treatment Errors 

ovider 

nic/Care 

following ways. For non-randomized studies, we assessed the risk of study bias on the following dimensions: population (e.g., representative, 

 outcomes (important outcomes assessed and measured, appropriate follow-up), measurement (variables uniformly assessed, blinded, construct 

n and methods minimize confounding) and whether the intervention can be replicated if applicable. Study quality for these studies is reported as 

sk was assessed as low) out of the total possible dimensions evaluated for risk. For randomized studies, we assessed study quality based on the 

 practitioner, MD = physician, N/A = not applicable, NP = Nurse practitioner, NR = not reported, PA = Physician Assistant, PC = primary care, 

ome, PCP = primary care provider 

62
ns on Patient Care: A Systematic Review Evidence-based Synthesis Program 

of Care Outcomes – Organizational Culture Studies 

onal 

tice & 

e 

Access Effectiveness 

Measured as: Main Finding Measured as: Main Finding 

d NR NR Median # of Hospitalizations Hospitalizations: 

am 

weekly 

n, 

and ER visits 

Team Care (n=12): Baseline = 0 6 month = 0 

ont 

Usual Care (n=8): Baseline = 0 6 month = 0 ER visits – Team Care (n=12): 6 months before = 0 6 month = 0.5 Usual Care (n=8): 6 months before = 0.5 6 month = 0.5 

d 

n of 

 to 

 clinic 

NR NR - Outcome Questionnaire 

45 (OQ-45) – standardized symptom distress inventory -EuroQoL(EQ-5D) - and index of health-related quality of life 

OQ-45 improved in 60% of patients EQ-5D (quality of life) improved for 83% of patients who completed first and last assessment (n=178; p<0.001) 

ulti- NR NR Admissions (members/year) Admissions: 

Readmissions (members/year) PCMH = 257 admissions/ 1000 members/ year; -18% [95% CI -30% to -5%; P<0.01] 

Control= 313 admissions/ 1000 members/ year§ Readmission: PCMH= 38/1000 members/year; -36% [95% CI, -55 to -3%; p=0.02] Control= 59/1000 members/year§ 

d care 

and 

am 

e) 

3 

ners, 

cist 

A Chronic Disease Management (CDM) Quality of Care (QOC) composite score based on 12 indicator processes for 4 chronic diseases (CAD, diabetes, CHF, and COPD) 

CDM QOC +9.29%; [p<0.001] Preventive Care +16.5%; [P<0.001] 

63
ns on Patient Care: A Systematic Review Evidence-based Synthesis Program 

onal 

tice & 

e 

Access Effectiveness 

Measured as: Main Finding Measured as: Main Finding 

s NR NR Association of clinic values 

and total quality based on management of: a) hypertension b) diabetes, and c) Preventive care from medical record audits. 

Quality emphasis 0.94 (4.07 to 5.95) Information and communication emphasis 4.65 (0.07 to 9.23) Trust in organization 1.88 (2.97 to 6.73) Workplace cohesiveness 0.85 (3.37 to 5.07) Values alignment 1.15 (3.47 to 5.78) 

ti

NR NR Contacts/ 1000 members/ 

year 

Admissions (ACSC): PCMH= 12/1000; RR=0.89; P<0.001 Usual Care= 13/1000 members/year Admissions: PCMH=100/1000 members/year RR=1.03 (NS) Usual Care= 100/1000 members/year 

ganizational culture components: 

Home (PCMH) 

ulatory Care Sensitive Conditions, BP = blood pressure, CAD = coronary artery disease, CHF = congestive heart failure, COPD = chronic 

= emergency room, LDL = low density lipoprotein, LEAP = lower extremity amputation prevention, NS = not statistically significant, PCMH = 

C = quality of care 

MH participants representing the expected outcomes from the active group if the PCMH had never been implemented 

afety Outcomes – Organizational Culture Studies 

al 

ce 

Diagnostic Errors Non-Medication Treatment Errors Medication Errors 

Measured as: Main Finding Measured as: Main Finding Measured as: Main Finding 

 

NR NR NR NR At least 1 drug 

related problem identified by the pharmacist 

344/431 (79.8%) 2.5 drug related problems/ senior *No comparison data from non pharmacist control 

orted 

64 
ns on Patient Care: A Systematic Review Evidence-based Synthesis Program 

Satisfaction Outcomes – Organizational Culture Studies 

nal Culture Patient Satisfaction with Provider Patient Satisfaction with Practice or Care 

 Measure Measured as: Main Finding Measured as: Main Finding 

Care – care 

of weekly nurses, and 

NR NR Patient Self-rated well-being 

Patient Satisfaction 

Patient self-rated well-being: Team based = +8% Usual care = no change Patient Satisfaction: Team based = satisfied or very satisfied increased from 75% at baseline to 92% at 6 months Usual care = “All control patients were very satisfied or satisfied at baseline and follow-up 

ment 

vs. traditional 

ic) 

NR NR Primary Care Satisfaction 

Survey for Women (PCSSW) 

a) Overall Satisfaction b) Getting Care c) Privacy/Comfort d) Communication e) Complete Care f) Follow-up Care 

Odds Ratios 

a) OR=1.42(1.00-2.02) b) OR=1.69(1.14-2.49) c) OR=1.63(1.11-2.39) d) OR=1.66(1.16-2.37) e) OR=1.69(1.17-2.43) f) OR=1.70(1.16-2.47)

Care- 

N trained in 

grated into 

work with 2-5 

NR NR Patient Assessment of 

Chronic Illness Care (PACIC) 

Compared to usual care, patients who received guided were twice as likely to rate chronic care highly (AOR=2.13 [95% CI=1.30 to 3.5 p=0.003)] 

Ambulatory Care Experiences Survey (ACES)* *Survey results from n=1,024 at PCMH clinic and n=1,662 at 2 control clinics 

ACES (Adjusted mean difference in scores): Quality of GP-patient interactions= 2.12; p<0.01 Shared Decision Making= 2.76; p<0.01 Coordination of Care= 3.38; p<0.001 Access = 3.48; p<0.001 

Patient Assessment of Chronic Illness Care (PACIC) survey* -Patient involvement in care -Degree teams helped set and refine goals 

PACIC (Adjusted mean difference in scores): Patient Activation/Involvement= 3.30; p<0.01 Goal Setting/Tailoring= 3.10; p<0.05 

ganizational culture components: 

Home (PCMH) 

ed odds ratio, CI = confidence interval, GP = general practitioner, NR = not reported, OR = odds ratio, RN = registered nurse 

65
ns on Patient Care: A Systematic Review Evidence-based Synthesis Program 

r Outcomes – Organizational Culture Studies 

Job Stress Job Satisfaction Burnout 

Main Finding Measured as: Main Finding Measured as: Main Finding 

NR Physician questionnaire on 

5 point scale 

8/10 doctors reported improved office atmosphere and quality of life at work 7/10 reported improved workload 

NR NR 

c 

 

on: 

 

a) -0.34 (-0.48 to -0.20) b) -0.25 (-0.37 to -0.13) c) -0.31 (-0.43 to -0.19) d) -0.25 (-0.39 to -0.11) e) -0.34 (-0.46 to -0.22) 

Association with physician rated clinic values: a) Quality emphasis: b) Information and comm. Emphasis: c) Trust in organization: d) Workplace cohesiveness: e) Values alignment: 

a) 0.51 (0.41 to 0.61) b) 0.32 (0.21 to 0.42) c) 0.55 (0.45 to 0.65) d) 0.43 (0.30 to 0.59) e) 0.48 (0.37 to 0.59) 

Association with physician rated clinic values: a) Quality emphasis: b) Information and comm. Emphasis: c) Trust in organization: d) Workplace cohesiveness: e) Values alignment: 

a) -0.57 (-0.76 to -0.37) b) -0.33 (-0.51 to -0.14) c) -0.51 (-0.69 to -0.34) d) -0.33 (-0.50 to -0.15) e) -0.49 (-0.66 to -0.33) 

NR NR NR Maslach Burnout 

Inventory 

10% of PCMH staff reported emotional exhaustion vs. 30% of control clinics p<0.01 

ganizational culture components: 

Home (PCMH) 

orted 

66
ns on Patient Care: A Systematic Review Evidence-based Synthesis Program 

tion of Physical Environment Studies 

Sample 

Study Design 

Working 

Conditions 

Studied 

Patient/Provider 

Outcomes Studied Study Quality Patients Providers/Clinics 

ge students Slides of 35 waiting rooms Case series Environment Perceived quality of 0/3 relevant criteria 

(analyzed data from 34 (waiting areas) care 

ority Caucasian” 

ange 18-24 years) 

): NR 

slides) Comfort in 

environment 

r citizens 

arily Caucasian” 

ange 59-90 years) 

): NR 

=1118 n=19 with data from Phase 1 Before and after Environment Patient anxiety, 1/5 relevant criteria 

: 48.8 

=954 

% 

: 47.8 

matched patients (Phase 1 

) 

and twice in Phase 2 (4 and 11 months after move); 13 administrative/ reception staff, 6 health professionals Male: 21% Race: NR Age (years): NR 

Patient questionnaire completion rate 80% in both phases 

(lighting, sound, space, privacy, furnishings, art) 

satisfaction Staff well-being, job satisfaction Patient-doctor communication 

orted 

67