PACT Implementation and Provider Job Turnover

Philip Sylling, MA, Edwin Wong, PhD, Chuan-Fen Liu, MPH, PhD, Susan Hernandez, MPA, Adam J. Batten, BA, 

Christian Helfrich, MPH, PhD, Karin Nelson, MD, MSHS, Stephan D. Fihn, MD, MPH, Paul L. Hebert, PhD

PACT Demo Labs Coordinating Center,

VHA Office of Analytics and Business Intelligence

PACT Cyberseminar, February 19th, 2014

1 
VETERANS HEALTH ADMINISTRATION

PACT Implementation and PCP Turnover

Poll Question

What is your involvement with PACT?

1) I am a member of a PACT teamlet

2) I am researching or evaluating PACT

3) I am an administrator implementing PACT

4) I am involved with PACT in another capacity

5) I am unfamiliar with PACT

2 
VETERANS HEALTH ADMINISTRATION

PACT Implementation and PCP Turnover

Agenda

1) Previous PACT cyberseminars

2) Project motivation

3) Data definitions and descriptive statistics

4) PACT/PCP turnover association

5) Other factors associated with turnover

6) PACT/turnover association and PCP characteristics

7) Limitations/Conclusions 

8) Questions 

3 
VETERANS HEALTH ADMINISTRATION

PACT Implementation and PCP Turnover

Some Previous PACT Seminars

3/21/2012 Work Role Transformation during PACT Implementation: Successes and Challenges

• Includes VISN 4 and VISN 23 surveys of PCPs and PACT team members

• Interviews with PCPs and RN Care Managers about PACT and role shifts

8/15/2012 Facilitating the Nursing Role Transformation in PACTs

• Includes VISN 22 survey of PCPs and PACT team members

1/16/2013 PACT Implementation: Findings from primary care surveys

• National PACT survey: Includes reported facilitators & barriers to patient-centered care

• VISN 4 survey: Includes barriers to quality improvements and survey of PCP experiences

2/20/2013 Provider and Staff Experience with PACT: Results and recommendations from 

national and regional primary care surveys 

• Elements of PACT and burnout: Includes national PACT survey and VISN 20 and VISN 22 surveys

6/19/2013 Building the Foundation for PACT in a Large VA Academic Medical Center: Access and 

Continuity for Part-time Providers

• Includes VISN 11 interviews with PACT team members

4 
VETERANS HEALTH ADMINISTRATION

PACT Implementation and PCP Turnover

Previous PACT Seminars: Some Conclusions

1) Inadequate staffing is a frequently reported barrier to PACT 

implementation

2) PCPs report relatively high levels of burnout

3) Role shifts necessary for PACT team-based care are often 

difficult and stressful

4) Support for PACT with effective teamlet functioning and 

dedicated administration support 

5 
VETERANS HEALTH ADMINISTRATION

PACT Implementation and PCP Turnover

Why study PCP turnover?

• Organizational change and role shifts may influence 

PCP turnover, at least in the short-term

Lower PCP turnover may help mitigate PACT staffing 

barriers

•

PACT 

Implementation

• Patient-provider continuity is directly impacted by PCP 

turnover PACT Principles

• Provider and job characteristics may predict turnover

• Turnover leads to significant recruitment costs Administration

• We use VA administrative data to study provider 

behavior

Complements insights gained from surveys and 

interviews 

Methodology •

6 
VETERANS HEALTH ADMINISTRATION

PACT Implementation and PCP Turnover

Hypotheses

• At the overall system level, PCP turnover may be related to 

organizational changes

 We examined the association between PACT 

implementation and PCP turnover at the national level using 

an Interrupted Time Series analysis

• At the individual level, PCP turnover may be influenced by 

economic incentives

 We adjusted the PACT/turnover association for individual 

and local economic factors

7 
VETERANS HEALTH ADMINISTRATION

PACT Implementation and PCP Turnover

Data Sources (2003-2012)

• VA Corporate Data Warehouse (CDW)

o Identified VA PCPs (‘03 to ‘12) from [PCMM].[PatientProviders] table

o ProviderRole = “PC ASSIGNMENT”, TeamPurpose = “PRIMARY CARE”

o Used RelationshipStartDate & RelationshipEndDate to establish PCP’s 

assigned patients during a given quarter

o Providers with no assigned patients during a given quarter defined as 

not in VA primary care workforce in that quarter

• VA Personnel and Accounting Integrated Data (PAID)

o Quarterly PCP/job covariates: FTE status, Start of VA employment, etc.

o Used to indicate whether a PCP worked during a given quarter

o http://www.herc.research.va.gov/publications/guidebooks.asp

8 
VETERANS HEALTH ADMINISTRATION

PACT Implementation and PCP Turnover

Data Sources (2003-2012)

• Medicare Wage Index for VA facilities (from HERC)

o Used to inflation-adjust provider annual FTE salary rates

o If VA PCP salary-growth exceeds(lags) Medicare wage index growth, VA 

providers may have lower(higher) incentive to leave VA primary care

• Market area unemployment (not healthcare specific unemployment)

o Proxy for changes in providers’ non-VA employment opportunities

o 23 VISNs comprise 81 geographic market areas

o http://vaww.pssg.med.va.gov/PSSG/geo_acess/FY2011_Geographic_A

ccess_Report_Final_22MAR13.pdf

o Market area unemployment is an average of included county 

unemployment rates (weighted by Veteran population)

9 
VETERANS HEALTH ADMINISTRATION

PACT Implementation and PCP Turnover

Covariates From Data Sources (2003-2012)

Position Covariates

• End of quarter (EOQ) FTE salary 

o In 2003 dollars

o Deflated using Sta5a 

Medicare wage index

• Tenure group=1 (vs. Tenure=0)

o Reduction in Force priority

• Full-time (vs. Part-time)

o Not hours worked in PC

• VAMC (vs. CBOC)

10

Provider Covariates

• Gender

• Provider type (MD, NP, PA)

• Age (Under 45, 45 to 55, Over 55) 

o At EOQ

• VA experience

o EOQ date – VA start date

Other Covariates

• VA market area unemployment

• Sta3n dummy variables (Fixed 

effects)
VETERANS HEALTH ADMINISTRATION

PACT Implementation and PCP Turnover

Longitudinal Sample Construction

 Observational period: FY2003-Q2 to FY2012-Q4, Quarterly frequency

 Assume PACT begins in April, 2010 (FY2010-Q3) 

 Inclusion criteria: PCP has patients in [PCMM].[PatientProviders] during quarter

 Inclusion criteria: PCP has hours worked in PAID data during quarter (13,739 PCPs)

 Excluded residents identified in [CDWWork].[Dim].[ProviderType]

 Excluded PCPs with Intermittent status in PAID data (1,765 PCPs)

 For PCPs in multiple facilities in same quarter (1,386 PCPs), included only longest-held

 Excluded PCPs with no covariate data (587 PCPs)

11
VETERANS HEALTH ADMINISTRATION

PACT Implementation and PCP Turnover

Definition of Turnover Dependent Variable

• For each quarter, PCP is “In Data” if 

previous inclusion criteria are met

• Otherwise, PCP is “Not In Data” 

• Turnover was defined by providers’ 

dropping out of longitudinal sample for 

two or more consecutive quarters 

• Definition allows for leaves of absence 

(up to about 7 months of absence)

• Transfers between facilities (PCP to PCP) 

are not considered turnover

• Transfers within VA (PCP to Non-PCP) are 

considered turnover

• VA PCPs may turnover and later re-enter 

the sample

• Do not indicate reason for turnover

Example of Dependent Variable

Quarter PCP 1 PCP 2 PCP 3

 . . . . . . . . . . . . . . . .

FY06 Q2 In Data 0 In Data 0 In Data 0 

FY06 Q3 In Data 0 In Data 0 In Data 0 

FY06 Q4 In Data 0 In Data 1 Not In Data

FY07 Q1 In Data 1 Not In Data In Data 0 

FY07 Q2 Not In Data Not In Data In Data 0 

FY07 Q3 Not In Data Not In Data In Data 0 

FY07 Q4 Not In Data Not In Data In Data 0 

FY08 Q1 Not In Data In Data 0 In Data 0 

FY08 Q2 Not In Data In Data 0 In Data 0 

FY08 Q3 Not In Data In Data 0 In Data 0 

 . . . . . . . . . . . . . . . .

Turnover 

in FY07 Q1

Turnover 

in FY06 Q4 

– Re-entry 

in FY08 Q1

No 

Turnover 

(Only 1 Qtr 

Absence)

12
VETERANS HEALTH ADMINISTRATION

PACT Implementation and PCP Turnover

Descriptive Statistics

13

Pre-PACT 

FY03 Q2-FY10 Q2

Post-PACT 

FY10 Q3-FY12 Q2

P-Value* (Pre-Post 

difference) 

Quarterly Turnover Rate

All PCPs 3.06% 3.38% P=0.001

MD (FT, Tenure group=1) 2.60% 3.14% P<0.001

NP (FT, Tenure group=1) 3.33% 3.65% P=0.160

PA (FT, Tenure group=1) 3.89% 3.87% P=0.981

Covariates

Female PCP - %(N) 50.9% 54.0% P=0.001

MD Provider - %(N) 69.3% 69.7% P=0.636

NP Provider - %(N) 20.8% 21.3% P=0.528

PA Provider - %(N) 9.9% 9.1% P=0.111

*P-values determined from 1000 iterations of a block bootstrap process
VETERANS HEALTH ADMINISTRATION

PACT Implementation and PCP Turnover

Descriptive Statistics

14

Pre-PACT 

FY03 Q2-FY10 Q2

Post-PACT 

FY10 Q3-FY12 Q2

P-Value* (Pre-Post 

difference) 

Covariates

PCP age under 45 - %(N) 29.7% 22.8% P<0.001

PCP age 45-55 - %(N) 40.0% 35.3% P<0.001

PCP age over 55 - %(N) 30.4% 41.9% P<0.001

Years of experience Mean=8.94 Mean=9.40 P=0.001

Salary index (Mean=1.0 in ‘03) Mean=1.036 Mean=1.108 P<0.001

Tenure group=1 (vs. 0) - %(N) 88.6% 93.5% P<0.001

Full-time salary (vs. PT) - %(N) 87.6% 88.9% P=0.015

*P-values determined from 1000 iterations of a block bootstrap process
VETERANS HEALTH ADMINISTRATION

PACT Implementation and PCP Turnover

Descriptive Statistics

15

Pre-PACT 

FY03 Q2-FY10 Q2

Post-PACT 

FY10 Q3-FY12 Q2

P-Value* (Pre-Post 

difference) 

Covariates

VAMC (vs. CBOC) - %(N) 59.4% 52.9% P<0.001

Market Unemployment rate Mean=6.12 Mean=9.18 P<0.001

Provider-Quarter Observations N=147,225 N=45,541

Unique Providers 11,387

*P-values determined from 1000 iterations of a block bootstrap process
VETERANS HEALTH ADMINISTRATION

PACT Implementation and PCP Turnover

Methodology: Interrupted Time Series

Allow for secular trend in turnover rate: 

Upward/Downward/No Trend

• Linear time trend explanatory 

variable

Allow for change in turnover rate after 

PACT: Upward/Downward/No Change

• PACT Indicator explanatory variable

• = 0 prior to FY10-Q3 (April, 2010)

• = 1 on and after FY10 Q3

Allow for seasonality in turnover

• Quarter-of-year indicator variable

Turnover Rate
VETERANS HEALTH ADMINISTRATION

PACT Implementation and PCP Turnover

Methodology: Interrupted Time Series

Allow for secular trend in turnover rate: 

Upward/Downward/No Trend

• Linear time trend explanatory 

variable

Allow for change in turnover rate after 

PACT: Upward/Downward/No Change

• PACT Indicator explanatory variable

• = 0 prior to FY10-Q3 (April, 2010)

• = 1 on and after FY10 Q3

Allow for seasonality in turnover

• Quarter-of-year indicator variable

Turnover Rate
VETERANS HEALTH ADMINISTRATION

PACT Implementation and PCP Turnover

Logistic Regression: Statistical Notes

• Modeled the quarterly probability of PCP turnover using 

logistic regression

• Robust standard errors account for clustering at the PCP level 

(repeated observations)

• Sta3n dummy variables included in the model – fixed effects 

(500+ observations for all but 6 Stations)

18
VETERANS HEALTH ADMINISTRATION

PACT Implementation and PCP Turnover

Logistic Regression: Average Marginal Effects

• Average Marginal Effects (AME) 

were calculated to estimate the 

association between explanatory 

variables and PCP turnover

• “Effect” is misleading – cannot be 

interpreted as causal effect

• AMEs reflect the change in

turnover probability associated 

with a one unit change in an 

explanatory variable

19

• ProbNP=0.05 – ProbMD=0.045 

 AMENP=0.005

AME X% increase in risk of turnover 

(Assume 3% baseline turnover)

0.002 6.67%

0.004 13.33%

0.006 20.0%

0.008 26.67%
VETERANS HEALTH ADMINISTRATION

PACT Implementation and PCP Turnover

Association between PACT and Turnover

0

0.01

0.02

0.03

0.04

0.05

Turnover Probability

AME of PACT

Baseline Turnover

 0.0026* 0.0028* 0.004**

20

*, ** denote significance 

at the 0.05, 0.01 levels

Model 1 Model 2 Model 3
VETERANS HEALTH ADMINISTRATION

PACT Implementation and PCP Turnover

Association between PACT and Turnover

0

0.01

0.02

0.03

0.04

0.05

Turnover Probability

AME of PACT

Baseline Turnover

 0.0026* 0.0028* 0.004**

Going Forward with 

Model 3

Model 1 Model 2 Model 3

21
VETERANS HEALTH ADMINISTRATION

PACT Implementation and PCP Turnover

Association between PACT and Turnover

22

Broad economic conditions may influence conclusions drawn 

from policy evaluations

Elevated 

unemployment 

roughly 

coincident with 

PACT

Suppose a small 

increase in 

turnover 

observed after 

PACT

Historical 

unemploymentturnover 

relationship 

predicts 

reduction in 

turnover

Greater 

estimated 

association 

between PACT 

and turnover 
VETERANS HEALTH ADMINISTRATION

PACT Implementation and PCP Turnover

Factors associated with baseline PCP Turnover

-0.0027** 

0.0055** 

0.0084** 

-0.0049** 

0.0055** 

-0.0062** -0.007

-0.005

-0.003

-0.001

0.001

0.003

0.005

0.007

0.009

Average Marginal Effect 

(AME)

23

*, ** denote significance at the 0.05, 0.01 levels
VETERANS HEALTH ADMINISTRATION

PACT Implementation and PCP Turnover

Other Covariates

Covariate Average Marginal Effect 

(AME) Standard Error

Linear time trend 0.0002** (<0.0001) 

VA experience (in Quarters) -0.0001** (<0.0001) 

Salary ($1000s of ‘03 dollars) -0.0001* (<0.0001) 

Unemployment Rate -0.0011** (0.0003) 

Tenure group=0 (vs. Tenure group=1) 0.0108** (0.0014) 

Part-time employee (vs. Full-time) 0.0073** (0.0014) 

FY-Q2 (Jan-Mar) (vs. Q1) -0.0035** (0.0012) 

FY-Q3 (Apr-Jun) (vs. Q1) -0.001 (0.0012) 

FY-Q4 (Jul-Sep) (vs. Q1) 0.0062** (0.0011) 

*, ** denote significance at 0.05, 0.01 levels

24
VETERANS HEALTH ADMINISTRATION

PACT Implementation and PCP Turnover

PACT/Turnover Association and PCP Characteristics

• Included interaction term between PACT indicator and PCP characteristics

• Distinct PACT/turnover association for each PCP group 

25

PCP Group AME of PACT 95% Confidence 

Interval

Significant 

Difference 

Male PCP 0.0061 (0.0024 , 0.0098) No 

Female PCP 0.0024 (-0.0009 , 0.0057)

MD PCP 0.0048 (0.0017 , 0.0079)

NP PCP 0.0030 (-0.0021 , 0.0081) No 

PA PCP 0.0014 (-0.0057 , 0.0085)
VETERANS HEALTH ADMINISTRATION

PACT Implementation and PCP Turnover

PACT/Turnover Association and PCP Characteristics

PCP Group AME of PACT 95% Confidence 

Interval

Significant 

Difference 

Age under 45 -0.0008 (-0.0049 , 0.0033) 45-55 AME > 

Under 45 AME

Over 55 AME > 

Under 45 AME

Age 45-55 0.0046 (0.0011 , 0.0081)

Age over 55 0.0069 (0.0026 , 0.0112)

5 yrs VA experience 0.0019 (-0.0012 , 0.0050)

20 Years AME > 

5 Years AME

10 yrs VA experience 0.0051 (0.0022 , 0.0080)

15 yrs VA experience 0.0080 (0.0047 , 0.0113)

20 yrs VA experience 0.0106 (0.0065 , 0.0147)

26
VETERANS HEALTH ADMINISTRATION

PACT Implementation and PCP Turnover

Post-PACT PCP Turnover: Sta3n-level Variation

13 15 

41 

25 

11 13 11 

0

10

20

30

40

50

Number of Parent

Stations (Sta3n)

Average Quarterly PCP Turnover Rate (FY10-Q3 to FY12-Q2) 

Distribution of Post-PACT Turnover at Sta3n-level 

(CBOCs rolled up to Sta3n-level)

27
VETERANS HEALTH ADMINISTRATION

PACT Implementation and PCP Turnover

Limitations

• No control group of VA facilities since PACT is a system-wide 

initiative

o Analysis does account for pre-existing turnover trend back to ‘03

• Measure of turnover is imperfect (inferred from sample 

dropout) 

• Does not account for turnover reason (resignation, 

retirement, etc.)

• Healthcare specific unemployment would be a better proxy 

for PCP employment opportunities than overall 

unemployment rate

28
VETERANS HEALTH ADMINISTRATION

PACT Implementation and PCP Turnover

Conclusions

• At the national level, PACT was associated with a small but 

statistically significant increase in PCP turnover

o Model prediction: Suppose 7,000 VA PCPs * 0.004 PACT AME * 8 Post-PACT 

quarters  Implies 224 additional PCPs leaving VA primary care

• PACT/turnover association was significantly greater for older 

and more experienced providers

• PCP characteristics were significant predictors of baseline 

turnover (e.g., higher baseline turnover for NPs and PAs 

compared to MDs)

• Variation in Post-PACT PCP turnover rate at the Sta3n-level

29
VETERANS HEALTH ADMINISTRATION

PACT Implementation and PCP Turnover

Questions?

Philip Sylling, MA

Philip.Sylling@va.gov

30