Changes in working to top of competency 

and burnout from 2012 to 2013: 

Results of the PACT Primary Care Personnel Survey and 

All Employee Survey

Christian D. Helfrich, PhD

Organizational Function Working Group 

PACT Demonstration Lab Coordinating Center 

Survey fielded by the VHA National Center 

for Organization Development (NCOD)

Christopher Orszak – Project Manager

Katerine Osatuke – Research Supervisor
VETERANS HEALTH ADMINISTRATION

What’s your role? 
VETERANS HEALTH ADMINISTRATION

How many of you have role with PACT?

• Member of PACT teamlet

• Member of broader PACT team (working with multiple teamlets)

• Providing other support to PACT (e.g. training)

• PACT/Primary Care Administrators

• Studying PACT
VETERANS HEALTH ADMINISTRATION

Patient Aligned Care Team (PACT) initiative 

background 

• VHA’s patient-centered medical home model

• Launched April 2010

• Multiple components

– Emphasis on team-based care

• PCP (MD, Nurse Practitioner or Physician Assistant), RN care 

manager, clinical assoc. & clerical assistant 

• Share responsibility for defined panel of patients 

– Scheduling & alternatives to in-person visits; 

– Use of nurse care managers and additional health 

promotion support 
VETERANS HEALTH ADMINISTRATION

Patient Aligned Care Team (PACT) initiative 

background 

• Resources to support PACT implementation 

– Funding to support the expanded staffing model

– Training, e.g., Regional Learning Collaboratives w/ VA 

Systems Redesign 

• 5 PACT Demonstration Laboratories: VISNs 4, 11, 20, 

22 & 23

• National Demo Lab Coordinating Center
VETERANS HEALTH ADMINISTRATION

Patient Aligned Care Team (PACT) initiative 

background 

• Goals included

– Increase time PC employees work to top of 

competency 

– Reduce burnout
VETERANS HEALTH ADMINISTRATION

Findings on effects on employees are mixed

• 2012, elements of PACT team-based care associated with 

lower odds of burnout1 & higher job satisfaction2; clinics w/ 

greater PACT implementation reported lower burnout3 

• 2009-2012 trend in overall declining job satisfaction, 

increasing intent to quit among Primary Care employees

• Overall PCP turnover has significantly increased after PACT, w/ 

greater turnover for older & more experienced PCPs4 

1Helfrich et al, JGIM 2014, 2Randall et al, AcademyHealth, 2014, 3Nelson et al, JAMA Internal 

Medicine under revision, 4Sylling et al, SGIM 2014.
VETERANS HEALTH ADMINISTRATION

Survey Methods

• 3 surveys:

– 2012 PACT Primary Care Personnel Survey

• All employees in Primary Care

– 2012 All Employee Survey (AES)

• Employees reporting main type of service is PC

– 2013 All Employee Survey with PACT module

• Employees reporting main type of service is PC

• Members of a PACT also received PACT module 

• Web-based surveys

• Anonymous
VETERANS HEALTH ADMINISTRATION

Measures

• Top of competency measured with self-authored item

– % of time spent each week on work well-suited to training

– 4 categories reflecting quartiles of time 

– Only available in 2012 PACT Survey & 2013 PACT module

• Burnout measured w/ single-item from Physician Worklife

Study 

– Scored on 5 point scale; >=3 indicates burnout

– Validated against Maslach Burnout Inventory1

1Dolan et al, under review
VETERANS HEALTH ADMINISTRATION

Analyses

• Descriptive stats

• Logistic regression to test for difference in burnout rate in 

2013 vs. 2012

– Cluster-adjusted by facility, adjusted for respondent 

demographics and workload and staffing.

– Surveys anonymous so not possible to adjust for withinrespondent correlations

• For 2013 AES Primary Care Cohort, non-PACT respondents did 

not receive single-item measure of burnout

– Imputed burnout from Maslach Burnout Inventory item and 

respondent characteristics
VETERANS HEALTH ADMINISTRATION

FINDINGS 

Findings
VETERANS HEALTH ADMINISTRATION

Respondent demographics similar across 

survey groups

2012 AES 

PC cohort

(n=10,143)

2013 AES 

PC cohort

(n=9,312)

2012 PACT

PC survey

(n=4,819)

2013 AES

PACT 

module

(n=7,076)

Estimated response rates 62% 55% 25% 55%

Age 30-49 years 45% 45% 47% 45%

>= 50 years 55% 55% 53% 55%

Tenure with VA <6 months 4% 6% 3% 5%

6 months – 2 years 15% 14% 17% 14%

2 – 10 years 45% 45% 53% 46%

>10 years 37% 36% 28% 34%

Supervisors 7% 7% 7% 5%
VETERANS HEALTH ADMINISTRATION

Working to top of competency improved for PCPs, 

worse for clinical associates

Respondents reporting

spending highest category of 

time on work well-matched to 

training 

PACT survey

2012 

(n=4,819) 

AES PACT 

Module

2013 

(n=7,076) 

Provider (MD, NP, & PA) 51% 57% 

Care Manager (RN & NP) 48% 47% 

Clinical Associate 67% 61% 

Administrative Associate 65% 65% 
VETERANS HEALTH ADMINISTRATION

Unadjusted burnout by occupation 2012-

2013 –PACT PC survey & PACT Module

45.4%

(n = 1769)

39.9%

(n = 1135)

30.7%

(n = 1358)

37.3%

(n = 557)

43.4%

(n = 1644)

33.9%

(n = 2000)

28.0%

(n = 1355)

37.8%

(n = 438)

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

Provider Nurse Care Manager Clinical Associate Administrative Associate

2012 PPCS 2013 AES
VETERANS HEALTH ADMINISTRATION

Unadjusted burnout by occupation 2012-

2013 –AES Primary Care main service 

40.3%

(n = 2565)

25.1%

(n = 2934)

33.9%

(n = 218)

46.7%

(n = 918)

36.1%

(n = 840)

30.4%

(n = 578)

39.1%

(n = 165)

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

Provider Nurse Care Manager Clinical Associate Administrative Associate

2012 AES 2013 AES
VETERANS HEALTH ADMINISTRATION

Regression analysis: No change in burnout 

from ‘12 to ‘13

Prob Odds Ratio Confidence Limits

2013 (vs. 2012) 0.34 1.07 0.93 – 1.21 

Physician -- -- --

Clinical Associate <0.0001 0.54 0.40 – 0.69

Clerk 0.11 0.85 0.65 – 1.05

Nurse Care Mngr <0.0001 0.70 0.56 – 0.84

Nurse Practitioner/PA 0.52 1.06 0.88 – 1.23

Supervisor 0.20 0.89 0.71 – 1.07

Tenure < 1 year -- -- --

1-5 years <0.0001 2.29 2.10 – 2.49

5-10 years <0.0001 3.12 2.92 – 3.32

10-20 years <0.0001 3.25 3.03 – 3.46

>20 years <0.0001 2.55 2.34 – 2.77

Adjusted for staffing, average panel size, % of PCPs w/ panels over-capacity, patient 

complexity (assessed as average DCG score)
VETERANS HEALTH ADMINISTRATION

Limitations

• Serial cross-sectional data

• Working to top of competency was assessed with 

self-report

• Differences in survey sample and response rate make 

confident comparisons difficult
VETERANS HEALTH ADMINISTRATION

Conclusions

• Burnout remains high

• Burnout not significantly different in 2013

• There may be changes in working to top of 

competency, with improvements for PCPs but 

declines for nurses
VETERANS HEALTH ADMINISTRATION

Thoughts? Questions? Suggestions?

We welcome your input now and later:

• Christian.Helfrich@va.gov 206-277-1655

• John.Messina@va.gov 206-768-5473
Assessing Reactions to PACT 

Implementation: The Team and Individual 

Role Perceptions Survey (TIRPS)

VISN 23 

Patient Aligned Care Team Demonstration Lab

Greg L Stewart, PhD

Stacy Lolkus, BS

1 
The Team and Individual Role 

Perceptions Survey (TIRPS)

• Created to capture perceptions of individual job 

roles and teamwork within Patient Aligned Care 

Teams (PACT)

• Time Waves

• Baseline (Summer 2010; Individual Role Perceptions)

• 1 Year (Summer 2011; Team and Individual Role 

Perceptions)

• 2 Year (Summer 2012; Team and Individual Role 

Perceptions)

• 3 Year (Summer 2013; Team and Individual Role 

Perceptions) TIRPS

Team & Individual Role Perception Survey
The Team and Individual Role 

Perceptions Survey (TIRPS)

• Measures

• Team Characteristics (2011 – 2013) 

» Group-level attributes of high performing teams

• Individual Role Perceptions (2010 – 2013) 

» Perceptions held by individuals about their roles within teams

• Responses from Four Core Roles

• Provider (n = 30, 30, 12, 31)

• RN Care Manager (n = 20, 28, 15, 23)

• LPN (n = 29, 25, 18, 22) 

• Clerk (n = 19, 15, 7, 12) 

TIRPS

Team & Individual Role Perception Survey
2013 Sample Reports


Value For Highest Scoring Team

Top 25% Of Teams Score Have Scores Above This Value

Average Score For 54 Teams

Bottom 25% Of Teams Score Have Scores Below This Value

Value For Lowest Scoring Team

1 

5 

4 

3 

2 
5 

4 

3 

2 

Performance

Monitoring

Member

Backup

Team

Coordination

Conflict

Management

Psychological

Safety

Low 1 

High Score Distribution 

2013 Team Characteristics

TIRPS
5 

4 

3 

2 

Role

Capacity

Role 

Harmony

Role

Clarity 

Skill 

Variety

Personal

Empowerment

Low 1 

High Score Distribution 

2013 Individual Role Perceptions

TIRPS
2010 – 2013 Individual Perceptions: 

Role Capacity

Significant Drop (p<.05)

 LPN: between 2010 & 2012

1

1.5

2

2.5

3

3.5

4

4.5

5

2010 2011 2012 2013

Provider

RN Care Mgr

LPN

Clerk

TIRPS

Team & Individual Role Perception Survey
2010 – 2013 Individual Role Perceptions: 

Role Harmony

Significant Drop (p<.05)

 LPN: between 2010 & 2012

1

1.5

2

2.5

3

3.5

4

4.5

5

2010 2011 2012 2013

Provider

RN Care Mgr

LPN

Clerk

TIRPS

Team & Individual Role Perception Survey
2010 – 2013 Individual Role Perceptions: 

Role Clarity

1

1.5

2

2.5

3

3.5

4

4.5

5

2010 2011 2012 2013

Provider

RN Care Mgr

LPN

Clerk

TIRPS

Team & Individual Role Perception Survey
2010 – 2013 Individual Role Perceptions: 

Skill Variety

1

1.5

2

2.5

3

3.5

4

4.5

5

2010 2011 2012 2013

Provider

RN Care Mgr

LPN

Clerk

TIRPS

Team & Individual Role Perception Survey

Significant Drop (p<.05)

Clerk: between 2010 & 2011
2010 – 2013 Individual Role Perceptions: 

Personal Empowerment

Significant Drops (p<.05)

 RN Care Manager: between 2010 & 2011, between 2010 & 2013

 

1

1.5

2

2.5

3

3.5

4

4.5

5

2010 2011 2012 2013

Provider

RN Care Mgr

LPN

Clerk

TIRPS

Team & Individual Role Perception Survey
Summary

• TIRPS was used to provide team-level 

feedback to 54 groups in VISN 23

• Included information about both 

coordination and individual perceptions of 

their roles in teams

• Team coordination was generally high; teams 

need to improve tracking of progress toward 

goals

• Individual team members feel a sense of 

being asked to do too many tasks; 

empowerment and skill variety are generally 

high 

TIRPS

Team & Individual Role Perception Survey
Acknowledgements 

Gary Rosenthal, MD; Bonnie Wakefield, PhD; David Katz, MD; 

Samantha Solimeo, MPH, PhD; Michelle Lampman, MA; Cody 

Reeves, BS; Eean Crawford, PhD; the VISN 23 Patient Aligned 

Care Team Demonstration Laboratory, Department of Veterans 

Affairs, Iowa City VA Health Care System, Iowa City, IA

The VISN 23 Demonstration Laboratory is funded by the VA 

Office of Patient Care Services. 

Solimeo and True 2013 16
Disclaimer

• The views expressed in this presentation are 

those of the authors and do not necessarily 

reflect the position or policy of the 

Department of Veterans Affairs or the United 

States government.