Department of Veterans Affairs

Health Services Research & Development Service Evidence-based Synthesis Program

Rural Healthcare Workforce: 

A Systematic Review

December 2015

Prepared for:

Department of Veterans Affairs

Veterans Health Administration

Quality Enhancement Research Initiative

Health Services Research & Development Service

Washington, DC 20420

Prepared by:

Evidence-based Synthesis Program (ESP) 

West Los Angeles VA Medical Center

Los Angeles, CA

Paul G. Shekelle, MD, PhD, Director

Investigators:

Principal Investigator: 

Susanne Hempel, PhD

Paul Shekelle, MD, PhD

Co-investigators: 

Melinda Maggard Gibbons, MD, MSHS

Jesus G. Ulloa, MD, MBA

Ian T. Macqueen, MD

Research Associates: 

Isomi M. Miake-Lye, PhDc

Jessica M. Beroes, BS

Roberta Shanman, MLS

4
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

i

PREFACE

The VA Evidence-based Synthesis Program (ESP) was established in 2007 to provide timely and 

accurate syntheses of targeted healthcare topics of particular importance to clinicians, managers, and 

policymakers as they work to improve the health and healthcare of Veterans. QUERI provides funding 

for four ESP Centers, and each Center has an active University affiliation. Center Directors are 

recognized leaders in the field of evidence synthesis with close ties to the AHRQ Evidence-based 

Practice Centers. The ESP is governed by a Steering Committee comprised of participants from VHA 

Policy, Program, and Operations Offices, VISN leadership, field-based investigators, and others as 

designated appropriate by QUERI/HSR&D.

The ESP Centers generate evidence syntheses on important clinical practice topics. These reports help: 

Develop clinical policies informed by evidence;

Implement effective services to improve patient outcomes and to support VA clinical practice 

guidelines and performance measures; and 

Set the direction for future research to address gaps in clinical knowledge.

The ESP disseminates these reports throughout VA and in the published literature; some evidence 

syntheses have informed the clinical guidelines of large professional organizations.

The ESP Coordinating Center (ESP CC), located in Portland, Oregon, was created in 2009 to expand the 

capacity of QUERI/HSR&D and is charged with oversight of national ESP program operations, program 

development and evaluation, and dissemination efforts. The ESP CC establishes standard operating 

procedures for the production of evidence synthesis reports; facilitates a national topic nomination, 

prioritization, and selection process; manages the research portfolio of each Center; facilitates editorial 

review processes; ensures methodological consistency and quality of products; produces “rapid response 

evidence briefs” at the request of VHA senior leadership; collaborates with HSR&D Center for 

Information Dissemination and Education Resources (CIDER) to develop a national dissemination 

strategy for all ESP products; and interfaces with stakeholders to effectively engage the program. 

Comments on this evidence report are welcome and can be sent to Nicole Floyd, ESP CC Program 

Manager, at Nicole.Floyd@va.gov.

Recommended citation: Hempel S, Maggard Gibbons M, Ulloa JG, Macqueen I, Miake-Lye I, Beroes 

J, Shekelle P. Rural Healthcare Workforce: A Systematic Review. VA ESP Project #05-226; 2015.

This report is based on research conducted by the Evidence-based Synthesis Program (ESP) Center located at 

the West Los Angeles VA Medical Center, Los Angeles, CA, funded by the Department of Veterans Affairs,

Veterans Health Administration, Office of Research and Development, Quality Enhancement Research Initiative. 

The findings and conclusions in this document are those of the author(s) who are responsible for its contents; the 

findings and conclusions do not necessarily represent the views of the Department of Veterans Affairs or the 

United States government. Therefore, no statement in this article should be construed as an official position of the 

Department of Veterans Affairs. No investigators have any affiliations or financial involvement (eg, employment, 

consultancies, honoraria, stock ownership or options, expert testimony, grants or patents received or pending, or 

royalties) that conflict with material presented in the report.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

ii

TABLE OF CONTENTS

EXECUTIVE SUMMARY ............................................................................................................ 1

Introduction ................................................................................................................................. 1

Methods....................................................................................................................................... 1

Data Sources and Searches ..................................................................................................... 1

Study Selection ....................................................................................................................... 1

Data Abstraction and Quality Assessment .............................................................................. 2

Data Synthesis and Analysis ................................................................................................... 2

Results ......................................................................................................................................... 3

Results of Literature Search .................................................................................................... 3

Summary of Results for Key Questions.................................................................................. 3

Discussion ................................................................................................................................... 4

Conclusions ............................................................................................................................. 5

INTRODUCTION .......................................................................................................................... 6

METHODS ..................................................................................................................................... 8

Topic Development ..................................................................................................................... 8

Search Strategy ........................................................................................................................... 8

Study Selection ........................................................................................................................... 8

Data Abstraction ....................................................................................................................... 12

Quality Assessment ................................................................................................................... 12

Data Synthesis ........................................................................................................................... 13

Rating the Body of Evidence .................................................................................................... 13

Technical Expert Panel ............................................................................................................. 14

Peer Review .............................................................................................................................. 14

RESULTS ..................................................................................................................................... 15

Literature Flow.......................................................................................................................... 15

Key Question 1: What are the current versus projected healthcare provider needs by numbers 

and disciplines in the next 20 years in rural areas? ................................................................... 17

Current Healthcare Provider Need ........................................................................................ 17

Predicted Future Healthcare Provider Need ......................................................................... 17

Summary of Findings and Quality of Evidence for Key Question 1 .................................... 25
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

iii

Key Question 2: What factors influence healthcare providers’ geographic choices 

for practice? .............................................................................................................................. 26

Summary of Findings and Quality of Evidence for Key Question 2 .................................... 46

Key Question 3: What interventions have been shown to increase rural healthcare 

provider recruitment? ................................................................................................................ 50

Summary of Findings and Quality of Evidence for Key Question 3 .................................... 57

Key Question 4: What interventions have been shown to increase rural healthcare 

provider retention? .................................................................................................................... 57

Summary of Findings and Quality of Evidence for Key Question 4 .................................... 57

Key Question 5: What is the efficacy of current rural-specific resident and healthcare 

profession student training and education efforts? ................................................................... 57

Summary of Findings and Quality of Evidence for Key Question 5 .................................... 82

SUMMARY AND DISCUSSION ................................................................................................ 84

Discussion by Key Question ..................................................................................................... 85

Limitations ................................................................................................................................ 87

Research Gaps/Future Research ............................................................................................... 87

Conclusions ............................................................................................................................... 88

REFERENCES ............................................................................................................................. 90

TABLES

Table 1. Evidence Table – KQ1 (Provider Need) ..................................................................... 18

Table 2. Summary of Findings – KQ1 ...................................................................................... 25

Table 3: Evidence Table – KQ2 (Provider Choice Assessments) ............................................ 27

Table 4: Summary of Findings and Quality of Evidence – KQ2 .............................................. 46

Table 5. Evidence Table – KQ3 (Provider Recruitment Interventions) ................................... 52

Table 6. Evidence Table – KQ5 (Provider Training Evaluations) ............................................ 59

FIGURES

Figure 1: Literature Flow Chart ................................................................................................ 16

Figure 2: Effect of Student Interventions on Recruitment to Rural Healthcare ....................... 82

APPENDIX 1. SEARCH STRATEGY ...................................................................................... 101

APPENDIX 2. LIST OF EXCLUDED STUDIES ..................................................................... 105

APPENDIX 3. RISK OF BIAS ASSESSMENT ........................................................................ 126

APPENDIX 4. PEER REVIEW COMMENTS/AUTHOR RESPONSES ................................. 129
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

1

EXECUTIVE SUMMARY

INTRODUCTION

Approximately 20 percent of the US total population lives in rural areas. Patients living in rural 

areas are often underserved with regard to healthcare access. The complexity of rural healthcare 

provision requires careful and systematic evaluation of individual contributing factors. The 

purpose of this review is to examine the current literature quantifying current and projected 

health provider need, to explore geographic provider choices, to synthesize evidence on 

interventions to increase rural provider recruitment and provider retention, and to document the 

efficacy of student training for current rural healthcare in the US. 

This topic was developed in response to a nomination by the Office of Rural Health (ORH).

The Key Questions (KQ) are: 

1. What are the current versus projected healthcare provider needs by numbers and disciplines in 

the next 20 years in rural areas? 

2. What factors influence healthcare providers’ geographic choices for practice? 

3. What interventions have been shown to increase rural healthcare provider recruitment? 

4. What interventions have been shown to increase rural healthcare provider retention? 

5. What is the efficacy of current rural-specific resident and healthcare profession student 

training and education efforts? 

METHODS

The review is registered in PROSPERO: CRD42015025403.

Data Sources and Searches

We searched the electronic databases PubMed, CIN/AHL, Web of Science, SCOPUS, 

PsycINFO, ERIC, WorldCat, and Grey Literature Report for English-language research 

published in the last 10 years (2005-February 2015). In addition, we reference-mined pertinent

reviews, accessed targeted online resources (www.raconline.org,

http://bhpr.hrsa.gov/healthworkforce, www.ruralhealthresearch.org), and consulted with topic 

experts.

Study Selection

Two independent reviewers screened retrieved publications against the eligibility criteria; any 

disagreements were resolved by discussion in the review team.

Population(s): Studies in healthcare providers relevant to rural Community-Based Outpatient 

Clinics, Rural Health Clinics, and Critical Access Hospitals (family medicine, internal medicine,

emergency medicine, and obstetrics/gynecology physicians; general surgeons; pediatricians;

geriatricians; psychiatrists; nurse practitioners; and physician assistants) were eligible for 

inclusion for KQ2-4, providers and patients could provide data on KQ1, and providers in training 

were eligible for inclusion for KQ5.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

2

Intervention(s): Interventions aiming to increase provider recruitment and retention were

eligible for KQ3-5; studies addressing recruitment and retention regardless of the aim of the 

intervention were eligible for KQ3 and KQ4.

Comparator(s): Comparative studies documenting current needs and studies using statistical 

modeling techniques to predict future needs were eligible for KQ1, and no comparator was 

required for KQ2-5. 

Outcome(s): Studies reporting on current and projected needs of healthcare providers and 

patient healthcare access measures were eligible for KQ1; provider-reported and otherwise 

analytically provided factors potentially associated with geographic practice choices were

eligible for KQ2; recruitment and retention measures were eligible for KQ3-5. 

Timing: Studies reporting on demand, provider choices, and effects of provider interventions 

and training in 2005 to 2015 were eligible. Studies making predictions for 2015 and beyond were

also eligible for KQ1.

Setting: Studies addressing US rural healthcare settings were eligible for KQ1, KQ2, and KQ4,

and US healthcare training sites were eligible for KQ5.

Data Abstraction and Quality Assessment

For KQ1, we extracted the geographic region, provider groups covered, the predictive timeframe, 

data source, definition of “rural,” analytic method, results for rural healthcare, and the authors’ 

conclusion. The quality assessment concentrated on the data source reporting (detection bias)

and whether predictions exceed the sample (external validity).

For KQ2, we extracted the geographic region; targeted provider groups; number of participants;

study design; definition of “rural”; results for provider and setting characteristics, financial 

aspects and other predictors; and the authors’ conclusion. The quality assessment targeted the 

risk of bias due to response rate limitations (selection bias) and confounding variables (detection 

bias).

For KQ3 and KQ4, we extracted the geographic region, targeted provider groups, number of 

participants, intervention category and description, study design and comparator, recruitment and 

retention results, and the authors’ conclusion. The quality assessment considered selection, 

performance, attrition, detection, and reporting bias.

For KQ5, we extracted the geographic region of the school and the rural placement, the provider

groups in training, number of trainees, training content, school capacity, study design and 

comparator, definition of “rural,” data source for outcome elicitation, recruitment and retention

results, and the authors’ conclusion. The quality assessment focused on the completeness of 

follow-up (attrition bias).

Data Synthesis and Analysis

Each key question was summarized in a narrative synthesis. KQ1 differentiated current and 

future need. The synthesis of KQ2 determined the evaluated factors and the associated strength 

of evidence. KQ3 summarized all interventions that aim to increase provider recruitment. KQ4 
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

3

summarized all interventions that target provider retention. KQ5 summarized programs aimed at 

providers in training. The summary of findings tables were organized by outcome and the quality 

of evidence assessment followed the GRADE approach. VA provider samples were a preplanned 

subgroup analysis.

RESULTS

Results of Literature Search

The literature review identified 5,756 citations. In total, 446 publications were obtained as full 

text. Of these, 59 publications met inclusion criteria, contributing to one or more key question.

Summary of Results for Key Questions

KQ1: Rural Healthcare Provider Needs

We identified a small number of studies quantifying current healthcare provider needs for rural 

areas. Estimates were for specific geographic regions and time periods, and studies 

operationalized provider need differently. Hence it is difficult to make concrete or generalizable 

evidence statements regarding the number of healthcare providers needed across studies. 

However, all included studies reported current unmet healthcare provider needs that worsen with 

increasing rurality. There is a healthcare provider shortage in particular for primary care 

providers, mental health professionals, and general surgeons. 

We also identified a very small number of published studies estimating future provider needs in 

rural healthcare. The 3 studies made predictions for primary care physicians, emergency 

physicians, and surgeons. We did not identify more than one study reporting on the same 

provider group or studies reporting on other provider groups of interest. All studies concluded 

that the supply is not likely to meet demand.

KQ2: Healthcare Providers’ Geographic Choices for Practice

A large number of studies exploring the determinants of practicing in rural care has been 

published. Growing up in a rural community was the most consistent factor associated with 

practice location choice across the 24 identified studies. Education efforts for physicians, such as 

rural tracks, also seem to increase the likelihood of practicing in a rural community. Associations 

were also shown for the family of providers, exposure to rural communities, a primary care and 

family medicine focus, osteopathic education, recreation activities, and rural lifestyle; however, 

the evidence base is very limited. Although a large number of studies explored potential factors

associated with practicing in a rural setting, studies are needed to determine the relative 

importance of the predictor variables.

KQ3: Rural Healthcare Provider Recruitment Interventions

We only identified 5 evaluations aimed at practicing providers. Studies assessed the J-1 visa 

waiver program and loan forgiveness programs. A J-1 visa evaluation reported that 53% of 

physicians did not complete their obligations, but of the respondents who had completed their 

commitment, 84% remained a median of 23 months longer than required. A state-wide 

evaluation of various recruiting programs reported that 80% of placed providers have remained 

at their initial placement site upon completion of their obligation. The loan repayment programs 
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

4

reported on different outcomes and success measures. One study reported that 86% of surveyed 

rural physicians indicated that they continued medical practice at their sponsoring Minnesota 

healthcare facility after completing their service obligation. One evaluation reported that of those 

recipients whose obligated practice location was rural, 84% were practicing in rural areas and 

28% of program completers were practicing in rural areas. One evaluation highlighted that 74% 

of recipients were already working in or intended to work in an eligible community when they 

were made aware of the program.

KQ4: Rural Healthcare Provider Retention Interventions

We did not identify any study evaluating an intervention specifically aimed at improving 

retention for healthcare providers in US rural healthcare facilities. 

KQ5: Rural Student and Resident Training Programs

We identified 23 program evaluations focusing on providers in training. All reported on medical 

students and residents. Programs varied in their approach and recruitment success estimates. 

Results varied across datasets and programs but most estimates ranged between 35 and 65%. We 

did not identify factors that appeared to systematically affect success rates. Across approaches, 

studies reported a median success rate for rural healthcare recruitment of 53%.

DISCUSSION

The review demonstrates rural healthcare workforce needs, determinants of providers’

geographic choices, the lack of intervention studies aimed at US providers, and a multitude of 

recent provider in training efforts. 

The included studies quantified current and predicted shortages for rural areas for specific 

provider groups and for a defined period of time. The identified estimates of unmet needs 

underpin the perceived shortage of healthcare providers for rural areas as highlighted in 

numerous publications on the topic. However, the variability in the metrics used to define unmet 

need illustrate the difficulty in quantifying provider needs and comparing unmet needs across 

geographic regions and provider disciplines.

A large number of studies has addressed determinants of providers’ geographic choices and 

showed growing up in a rural community as the most consistent factor associated with rural 

practice. This factor has also been identified in earlier datasets and it continues to play an 

important role in the current healthcare system. A second key variable that emerged in the 

literature was education efforts such as rural tracks for physicians. Although the choice of 

selecting a rural track may be in part determined by a personal affinity preceding the choice of 

school, multivariate analyses suggest that the effect cannot be entirely explained by the variable 

of growing up in a rural community. However, there is a lack of studies helping to differentiate 

the relative importance of a personal affinity for rural communities, motivation through rural 

training, and effects of interventions attracting trained healthcare providers into rural care 

settings on provider choices.

We only identified a very small number of studies aimed at the recruitment of healthcare 

providers for rural communities. This is consistent with existing reviews on the topic, indicating 

a need for more evaluations to support policy makers in adopting effective interventions to 
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

5

increase the availability of healthcare providers in underserved areas. Programs such as the J-1 

visa program need to be evaluated carefully. While half of physicians did not complete their 

obligations, of the respondents who had completed their commitments, over 80 percent remain 

longer than required. Research on loan forgiveness programs is sparse and the identified 

evaluations assessed different outcomes and results. Careful evaluation is needed given that 

existing research highlighted that a large proportion of participants were working in or intended 

to work in an eligible community when they were made aware of the program. Hence, programs

may influence retention but may not necessarily be key to entering rural healthcare. 

We did not identify any recent study specifically aimed at improving retention for fully trained 

healthcare providers in US rural healthcare facilities. The international literature on this topic 

also highlights that more research is needed to empirically evaluate suggested strategies. Our 

review focused on specific empirical evidence (ie, recruitment success), leaving out studies that 

concentrated on provider satisfaction with programs or other outcomes.

In the last 10 years a large number of studies has been published that evaluates programs for 

healthcare providers in training; however, the literature is dominated by studies focusing on 

medical students and residents. Across all approaches, about half the students trained for rural 

healthcare enter rural settings. Individual training programs vary widely in format and duration. 

More research is needed to parse out the specific aspects of the training experience that influence 

success and to determine the comparative effectiveness of different raining programs.

We did not identify published studies reporting on VA settings, and the definition of “rural” was 

operationalized differently across identified evaluations which added heterogeneity across 

studies. Future research should in particular concentrate on evaluating strategies to improve 

healthcare provider recruitment and retention.

Conclusions

All included studies reported current unmet healthcare provider needs that worsen with 

increasing rurality. The small number of studies estimating future need also predicted 

unmet provider needs for rural healthcare. 

Growing up in a rural community is the most consistent factor associated with practice 

location. Education efforts for physicians, such as rural tracks, also seem to increase the 

likelihood of practicing in a rural community. More research on the relative importance 

of factors is needed.

More research is needed to evaluate existing healthcare provider recruitment 

interventions for rural healthcare. 

There is a lack of evidence regarding interventions to support healthcare provider 

retention in rural healthcare.

Current evaluations of rural training programs for medical students and residents suggest 

a median success rate of 53%.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

6

EVIDENCE REPORT

INTRODUCTION

Approximately 20 percent of the US total population lives in rural areas;

1 exact estimates vary 

depending on the specific definition of rural.2 Compared to their urban counterparts, rural 

communities tend to have older residents and report higher rates of chronic diseases.3 In order to 

obtain healthcare services, members of these geographically dispersed rural populations must 

often travel great distances and incur significant costs.4 Rural Veterans represent a third of the 

total VA enrolled population.

Surveys indicate that rural communities struggle with recruiting as well as retaining healthcare 

providers and experience a healthcare provider shortage with ongoing, long-term vacancies.

5-9

Estimates differ by provider group but, for example, less than 12 percent of US physicians 

practice in rural areas.1 Hence one fifth of the nation’s population resides outside metropolitan 

areas, but only about a tenth of the nation’s physicians are to be found there. Prior publications 

have frequently concluded that the shortage will increase over time and that shortages of specific 

provider groups are more pronounced in rural areas compared to their urban counterparts.10-14

In order to ensure access to rural healthcare services, it is essential to reliably determine and 

quantify the current demand and shortage of healthcare providers. Furthermore, given the many 

years of training required for key healthcare personnel such as physicians, predictive studies are 

crucial to anticipate future provider workforce needs. Professional bodies, including state and 

national agencies, may then use study results and take adequate steps to ensure healthcare 

delivery in rural communities. Prediction models need to take a variety of variables into account, 

including current workforce status and anticipated workforce,

15,16 an aging population,17 changes 

in patient case mix, federal and state programs that address already perceived shortages,

4,12,18,19

and changes in models of care with implications for demand of individual provider groups.20-24

In recent years, more research has concentrated on exploring determinants of geographic practice 

choices of healthcare providers. Insight into the relative importance of demographic 

characteristics and motivational factors may provide indications which groups of providers 

should be targeted to maximize return on recruitment efforts.25 Systematically exploring the 

strength of associations may also contribute to selecting strategies that address demand, such as 

those that seem predominately normative (ie, matching inclinations and background of 

candidates), coercive (eg, a mandatory period of practice in rural healthcare), or utilitarian (eg,

continuous education programs to support practitioners in rural healthcare) policy approaches.26

Many approaches have been suggested and resources spent aiming to increase the number of 

providers practicing in rural healthcare. The international literature has pointed out emerging 

evidence,27,28 but it is not clear whether interventions can be applied to the unique US healthcare 

system and infrastructure. Our knowledge about which recruitment strategies are evidence-based 

and have indeed been shown to be successful and cost-effective in recruiting healthcare 

providers is limited. Furthermore, evaluating the effects of federal, state, or local programs is a 

complex undertaking and the relative importance and success rate of programs has to be 

evaluated within the context of the current healthcare system. 
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

7

Rural providers commonly treat a great diversity of conditions and perform a wide variety of 

procedures, often without specialized training and with limited access to professional support 

such as colleagues and educational opportunities.29-31 In addition, the intersection of rural living 

and healthcare challenges may create additional barriers to care that providers are not trained to 

navigate, as for example shown in the following quote: “I think when you work in a rural setting 

you can find yourself on call 24 hours a day. People will come to you in ways that are unseeing, 

unbelievably naive or intrusive, or completely innocent, almost with the expectation that you can 

do much more than you can.”32 It is crucial that healthcare provider organizations support 

providers and identify appropriate efforts that ensure workplace retention of recruited 

providers.33,34 However, evidence syntheses on the effectiveness and comparative effectiveness 

of interventions aiming to increase healthcare provider retention are sparse and frequently 

outdated, as the care environment has changed in the last decade through the increased use of 

internet applications and advanced communication technologies.

35-38 Telehealth and access to 

specialist input through online, real time exchanges, and easy access to high-quality 

videoconferencing technology can now support providers in remote locations. However, whether 

these innovations are successful in increasing retention of healthcare providers is an open 

question.

Finding ways to get physicians to practice in underserved areas has been an ongoing priority for 

organizations such as the Association of American Medical Colleges (AAMC). In 2006, AAMC 

called for a 30 percent increase in MD-granting medical school enrollment by 2015.

39 A 

systematic review with literature searches to 2006 highlighted efforts to target healthcare 

providers in training, for example by adding rural tracks to medical schools.40 Since the review, a 

substantial amount of new research has been published and results of implemented programs and 

reforms to address shortcomings41-44 should also have become apparent.

The complexity of rural healthcare provision requires careful and systematic evaluation of 

individual contributing factors. The purpose of this systematic review is to examine the research 

describing healthcare provider need, exploring geographic provider choices, synthesizing 

evidence on interventions to increase provider recruitment and provider retention, and 

documenting the efficacy of student and resident training for rural healthcare in the US in the last 

decade. The review concentrates on key healthcare provider groups with long training periods,

requiring workforce planning ahead of time, and that are critical to rural Community-Based 

Outpatient Clinics, Rural Health Clinics, and Critical Access Hospitals. The review does not 

provide a historic overview but concentrates on research that addresses the review questions in a 

contemporary context, applicable to the current healthcare environment. We did not restrict to a 

particular definition of “rural” but limited to US studies to maximize relevance and applicability 

to the VA. The review aims to systematically document current published, empirical evidence 

reporting on the outcomes of interest for rural healthcare.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

8

METHODS

TOPIC DEVELOPMENT

This topic was developed in response to a nomination by the Office of Rural Health (ORH), for 

an evidence review examining the literature describing access needs and limitations for Veterans 

in rural settings and the interventions that have been shown to improve the recruitment and 

retention of healthcare providers in rural settings. Key questions were developed with input from 

the topic nominator, the ESP Coordinating Center, the review team, and the technical expert 

panel (TEP).

The Key Questions were: 

1. What are the current versus projected healthcare provider needs by numbers and disciplines in 

the next 20 years in rural areas? 

2. What factors influence healthcare providers’ geographic choices for practice? 

3. What interventions have been shown to increase rural healthcare provider recruitment? 

4. What interventions have been shown to increase rural healthcare provider retention? 

5. What is the efficacy of current rural specific resident and healthcare profession student 

training and education efforts?

The review was registered in PROSPERO: CRD42015025403.

SEARCH STRATEGY

We searched the electronic databases PubMed, CIN/AHL, Web of Science, SCOPUS, 

PsycINFO, ERIC, WorldCat, and Grey Literature Report for English-language research 

published in the last 10 years (2005-2015). In addition, we reference-mined pertinent reviews,

accessed targeted online resources including www.raconline.org,

http://bhpr.hrsa.gov/healthworkforce, and www.ruralhealthresearch.org, and consulted with topic 

experts to identify pertinent US studies.

STUDY SELECTION

Two independent reviewers screened the titles and abstracts of retrieved citations. Citations 

deemed relevant by at least one reviewer were obtained as full text. Full-text publications were

screened against prespecified eligibility criteria. Any disagreements were resolved by consensus 

decision after discussion by all investigators.

To be included in the systematic review, studies had to meet the following criteria, organized in 

the PICOTS framework:

KQ1 PICOTS: What are the current versus projected healthcare provider needs by numbers and 

disciplines in the next 20 years in rural areas?
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

9

Population(s): Studies on healthcare providers relevant to rural Community-Based 

Outpatient Clinics, Rural Health Clinics, and Critical Access Hospitals (family medicine, 

internal medicine, emergency medicine, and obstetrics/gynecology physicians; general 

surgeons; pediatricians; geriatricians; psychiatrists; nurse practitioners; and physician 

assistants) were eligible for inclusion. In addition, studies using patient demographics

relevant to adults to determine objective healthcare provider needs were eligible to

provide data. Studies exclusively providing data for other professions and specialties 

were not eligible.

Intervention(s): n/a

Comparator / Study design: Studies eligible to demonstrate the current healthcare needs 

required a comparator (eg, rural vs urban settings). Studies eligible to predict future 

provider needs required the use of statistical modeling techniques; vague estimates (eg,

“need is likely to increase”) were not eligible.

Outcome(s): Studies reporting on current and projected healthcare provider needs in 

rural areas were eligible. Studies only reporting on provider supply without needs or 

demand assessment were excluded. We accepted the author’s definition of need, demand, 

shortage, or benchmarks (eg, existing guidelines for patient-provider ratio), and patient 

healthcare access measures (eg, projected number of required physicians to maintain 

patient-provider ratio). Studies addressing demand without specific numbers for the 

healthcare provider groups of interest and studies not providing data specifically to rural 

areas were not eligible.

Timing: Studies reporting on provider need since 2005 regardless of the start of the 

evaluation period, and studies making predictions beyond 2015 were eligible. 

Setting: Studies had to report on US rural healthcare practice settings, using the authors' 

definition (remote, not urban, non-metropolitan), to be eligible.

KQ2 PICOTS: What factors influence healthcare providers’ geographic choices for practice?

Population(s): Studies in healthcare providers relevant to rural Community-Based 

Outpatient Clinics, Rural Health Clinics, and Critical Access Hospitals (family medicine, 

internal medicine, emergency medicine, and obstetrics/gynecology physicians; general 

surgeons; pediatricians; geriatricians; psychiatrists; nurse practitioners; and physician 

assistants) were eligible for inclusion. Studies exclusively providing data for other 

professions and specialties were not eligible. Studies presenting analytic data predicting 

healthcare providers’ choices for practice were included but studies exclusively reporting 

data from other participant groups (eg, hospital administrators) reporting on perceived 

healthcare providers’ choices were not eligible.

Intervention(s) / Independent variables: Studies of provider-reported or other 

analytically derived factors potentially associated with geographic choices for practicing 

in rural care were eligible. Analytically derived factors were limited to variables 

preceding practicing in rural care such as demographic or provider training characteristics 

(eg, gender, growing up in rural community, rural track training). Studies exclusively 

associating training programs with practicing in rural healthcare were documented in 

KQ5.

Comparator / Study design: Surveys and interviews with healthcare providers as well 

as analytic studies identifying predictors of practice in rural settings published in journal 
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

10

articles or reports were eligible for inclusion. Studies only in abbreviated or other formats

(eg, conference abstracts dissertations) were excluded.

Outcome(s): Studies reporting associations and predictions for practicing in rural care 

were eligible. Studies only assessing factors associated with the intent of practicing in 

rural care and case studies reporting experiences of a single provider were not eligible.

Timing: Studies reporting on practicing in rural care since 2005 were eligible regardless 

of the timing of the predictor variables (eg, growing up in a rural area), start of the 

evaluation period, exposure duration, or length of follow-up. 

Setting: Studies had to report on US rural (as defined by the author) healthcare practice 

settings to be eligible. Studies predicting practicing in rural versus very remote US areas 

were eligible but studies comparing choices between US and international settings were

excluded.

KQ3 PICOTS: What interventions have been shown to increase rural healthcare provider 

recruitment? 

Population(s): Studies in healthcare providers relevant to rural Community-Based 

Outpatient Clinics, Rural Health Clinics, and Critical Access Hospitals (family medicine, 

internal medicine, emergency medicine, and obstetrics/gynecology physicians; general 

surgeons; pediatricians; geriatricians; psychiatrists; nurse practitioners; and physician 

assistants) were eligible for inclusion in the review. Studies exclusively providing data 

for other professions and specialties were not eligible. Interventions targeting providers in 

training were eligible for KQ5.

Intervention(s): Interventions aiming to increase provider recruitment and studies 

addressing recruitment regardless of the aim of the provider intervention were eligible

(eg, financial incentives, visa programs, educational interventions, improving the practice 

environment initiatives).

Comparator / Study design: Studies with concurrent (eg, randomized controlled trials 

[RCT]) or historic (eg, pre-post) comparators were eligible, and post-only studies were

eligible when they reported on a distinct cohort of participants.

Outcome(s): Studies reporting on recruitment success measures and studies reporting on 

retention measures, such as extended practice in rural areas after a mandatory period of 

time required by the intervention, were eligible. Post-only studies were eligible only if 

they reported numerical recruitment or retention data together with a denominator (eg,

number of participants in study group); publications making general statements such as 

“program was successful” were excluded.

Timing: Studies reporting data on practicing in rural care since 2005 were eligible 

regardless of the timing of the intervention, start of the evaluation period, intervention 

duration, or length of follow-up. 

Setting: Studies had to report on US rural (as defined by the author) healthcare practice 

settings to be eligible.

KQ4 PICOTS: What interventions have been shown to increase rural healthcare provider 

retention? 

Population(s): Studies in healthcare providers relevant to rural Community-Based 

Outpatient Clinics, Rural Health Clinics, and Critical Access Hospitals (family medicine, 
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

11

internal medicine, emergency medicine, and obstetrics/gynecology physicians; general 

surgeons; pediatricians; geriatricians; psychiatrists; nurse practitioners; and physician 

assistants) were eligible for inclusion. Studies exclusively providing data for other 

professions and specialties were not eligible.

Intervention(s): Interventions aiming to increase provider recruitment and retention, and 

studies addressing recruitment and retention regardless of the aim of the provider 

intervention were eligible.

Comparator / Study design: Studies with concurrent (eg, RCT) or historic (eg, pre-post) 

comparators were eligible. Post-only studies were only included if they reported on a 

distinct cohort of participants. 

Outcome(s): Studies reporting on provider retention measures (eg, staff turnover, 

employment duration, provider loss rate, moving to rural area) were eligible. Post-only 

studies were only included if they reported numerical recruitment or retention data 

together with a denominator (eg, number of participants in study group); publications 

making general statements such as “program was successful” were excluded.

Timing: Studies reporting data on practicing in rural care since 2005 were eligible 

regardless of the timing of the intervention, start of the evaluation period, intervention 

duration, or length of follow-up. 

Setting: Studies had to report on US rural (as defined by the author) healthcare practice 

settings to be eligible.

KQ5 PICOTS: What is the efficacy of current rural-specific resident and healthcare profession

student training and education efforts? 

Population(s): Studies in healthcare providers in training relevant to rural CommunityBased Outpatient Clinics, Rural Health Clinics, and Critical Access Hospitals (family 

medicine, internal medicine, emergency medicine, and obstetrics/gynecology physicians; 

general surgeons; pediatricians; geriatricians; psychiatrists; nurse practitioners; and 

physician assistants) were eligible for inclusion. Studies exclusively providing data for 

students of other professions and specialties were not eligible.

Intervention(s): Training and educational programs specific to rural healthcare and 

programs explicitly aiming to increase provider recruitment for rural areas were eligible.

Comparator / Study design: Studies published in peer-reviewed journal articles were

eligible regardless of the presence or the type of comparator. Studies not peer-reviewed

or documented only in abbreviated form (eg, dissertations, conference abstracts were 

excluded).

Outcome(s): Studies reporting on recruitment success measures were eligible. Studies 

only reporting on the intention to practice in rural healthcare or other effectiveness 

measures of programs and intervention-specific outcomes were excluded.

Timing: Studies reporting data on practicing in rural care since 2005 were eligible 

regardless of the timing of the intervention, start of the evaluation period, intervention 

duration, or length of follow-up. 

Setting: Studies reporting on US training sites and reporting on US rural healthcare 

practice settings (as defined by the author) were eligible. Training sites were defined as 

formally recognized and accredited locations for healthcare student and resident clinical 

education.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

12

Publications reporting on the same dataset were abstracted as one study. 

DATA ABSTRACTION

For KQ1 studies (current and future provider need), we abstracted the geographic region the 

study aimed to cover, the targeted provider groups, the predictive timeframe category (current vs

future demand) and the specified timeframe the study addressed, the data source, the definition 

of “rural” or other relevant case definition variables, the analytic method, the results for rural 

healthcare, and the authors’ conclusion.

The evidence table for KQ2 studies (provider choices to practice in rural healthcare) documents

the geographic region, the provider groups covered, the number of participants together with the 

response rate of all invited and eligible participants, the study design and comparators (where 

applicable), the results associated with predictions based on participant characteristics, results 

with regard to rural healthcare setting characteristics, predictive results regarding financial 

aspects, results for other potential predictors, and the authors’ conclusions.

Studies reporting on interventions relevant to KQ3 (provider recruitment interventions) or KQ4 

(provider retention interventions) documented the geographic regions included in the study, the 

targeted provider groups, the number of participants, the study design and the comparator 

(concurrent or historic), results regarding recruitment success, results regarding retention, and the 

authors’ conclusion.

The evidence table for KQ5 (programs for healthcare providers in training) studies documents

the geographic region of the school and the training / rural placement location, the discipline of 

the targeted providers in training, the number of trainees evaluated in the research study, the 

content of the training including duration of rural placement (where applicable), the capacity of 

the school, the study design (post-only, pre-post, comparative study with concurrent comparator, 

and random participant assignment), source of outcome data, definition of “rural,” rural 

healthcare recruitment results, information on retention in rural healthcare, other notable 

outcomes, and the authors’ conclusion.

QUALITY ASSESSMENT

Due to the diversity of included studies, the quality assessment primarily targeted study designassociated characteristics and inherent limitations in addition to key risk of bias dimensions 

appropriate for the type of question the study aimed to answer.

45

For KQ1, the quality assessment concentrated on the data source reporting (detection bias) and 

whether predictions exceeded the sample (external validity).

For KQ2, the quality assessment targeted the risk of bias due to response rate limitations 

(selection bias) and confounding variables (detection bias).

For KQ3 and KQ4 (provider intervention studies), the quality assessment considered selection, 

performance, attrition, detection, reporting, and other key sources of bias. Selection bias assessed

whether the study was based on a highly selective and not representative sample of the target 

population (eg, depending on survey respondents). Performance bias assessed whether fidelity to 

the intervention protocol was maintained. Attrition bias assessed the loss to follow-up. Detection 
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

13

bias evaluated whether the data were based on a reliable source. Reporting bias assessed whether 

key outcomes were apparently missing from the analysis and results were based on surrogate 

measures of recruitment success (eg, number of offers made, not number of successfully 

recruited providers). The other source of bias category determined whether the study reported on 

a related concept rather than rural healthcare (eg, practicing in very rural areas).

For KQ5, the quality assessment focused on the completeness of follow-up (attrition bias).

DATA SYNTHESIS

The identified research is presented in a narrative synthesis for each of the 5 key questions. 

Evidence tables summarize each study meeting inclusion criteria.

Some publications were relevant to more than one key question, such as studies evaluating the 

success of a recruitment intervention and analyzing predictors of practicing in rural healthcare 

within intervention groups. All studies investigating geographic choices of practitioners were

extracted for KQ2, regardless of whether the study also contributed to other evidence tables. 

Given the conceptual overlap, studies addressing programs aimed at students as well as 

practitioners were abstracted for KQ3 and KQ4 (provider recruitment and retention). Given the 

conceptual overlap, studies reporting on recruitment and retention measures were abstracted for 

KQ3 and discussed in the KQ4 section. KQ4 was reserved for studies that evaluated

interventions that exclusively focused on the retention of practicing healthcare professionals. 

Studies evaluating programs exclusively aimed at students or residents, and not fully trained 

healthcare providers, were summarized in the KQ5 section.

RATING THE BODY OF EVIDENCE

Where possible a summary of findings and quality of evidence table was used to summarize the 

existing evidence. Based on the GRADE working group,

46 the quality of the evidence was

categorized as follows:

High: We are very confident that the true effect lies close to that of the estimate of the effect.

Moderate: We are moderately confident in the effect estimate: The true effect is likely to be 

close to the estimate of the effect, but there is a possibility that it is substantially different.

Low: Our confidence in the effect estimate is limited: The true effect may be substantially 

different from the estimate of the effect.

Very low: We have very little confidence in the effect estimate: The true effect is likely to be 

substantially different from the estimate of effect.

GRADE evaluates the quality of the evidence across all identified studies contributing to the 

outcome of interest. We took the standard criteria Study limitation and risk of bias, 

Inconsistency, Indirectness, Imprecision, Publication bias, Large effect, and Dose response, and 

All plausible residual confounding would reduce a demonstrated effect and/or would suggest a 

spurious effect if no effect was observed into account when grading the evidence. The starting 

point for all key questions was moderate evidence (not high evidence) because the data were not 

based on RCTs or equivalently strong research design studies. Risk of bias, Inconsistency, 
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

14

Indirectness, Imprecision, and Publication bias can lower the quality, Large effect, Dose 

response, and All plausible residual confounding can upgrade the quality of the body of evidence

(where applicable). Neither of the key questions could be answered by a standard intervention 

study such as an RCT, hence the grading rested primarily on study limitation, risk of bias, and 

inconsistency across studies assessing the same factor. 

TECHNICAL EXPERT PANEL

The TEP guiding the project included: Nancy Maher, PhD, Program Analyst, Office of Rural 

Health, VHA Office of Rural Health; Stephanie Kondrick, VHA National Workforce Planner, 

Healthcare Talent Management Office; Ray Lash, MD, Director, ORH Rural Health Training 

Initiative, VA Maine Healthcare System; Dan Mareck, MD, Chief Medical Officer, Federal 

Office of Rural Health Policy/HRSA; George Zangaro, PhD, RN, Director, National Center for 

Health Workforce Analysis, HHS Bureau of Health Workforce; Randy Longenecker, MD, 

Assistant Dean, Rural and Underserved Programs and Professor of Family Medicine, Ohio 

University; Judy Howe, PhD, Director, ORH Rural Health Training Initiative, Bronx VAMC; 

Peter Kaboli, MD, Director of the Veterans Rural Health Resource Center — Central Region, 

Associate Professor at the University of Iowa Carver College of Medicine, Iowa City VA 

Medical Center; Thomas Klobucar, PhD, Deputy Director, Office of Rural Health, VHA Office 

of Rural Health (10P1R); and Janice Garland, MPH, Health Systems Specialist, Office of Rural 

Health, VHA Office of Rural Health (10P1R).

PEER REVIEW

A draft version of the report was reviewed by technical experts and clinical leadership. Reviewer 

comments and our responses are documented in Appendix 4.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

15

RESULTS

The identified research was summarized in a narrative synthesis. 

LITERATURE FLOW

The literature review identified 5,756 potentially relevant citations. In total, 446 citations were

obtained as full text. Of these, 59 publications met inclusion criteria, contributing to one or more 

key question. The flow diagram shows the number of identified studies through the literature 

search, studies meeting inclusion criteria, and the number of excluded studies with the reason for

exclusion. 

The publications not meeting inclusion criteria are listed in Appendix 2, ordered by reason for 

exclusion.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

16

Figure 1: Literature Flow Chart 

Search results: 

5,756 references

Pulled for full text review: 

446 references

Included studies: 59* 

Excluded = 5,310 references

Not US, not empirical study, provider group 

not eligible 

Excluded = 279 references

Exclude-Participants: 30

Exclude-Intervention: 4

Exclude-Study design: 86

Exclude-Outcome: 117

Exclude-Timing: 14

Exclude: Setting: 27

Duplicate: 1

Background = 108 

(more information on included studies or 

source of potential includes)

KQ 1 

(demand):

N = 11

KQ 2 

(motivation):

N = 24

KQ 3

(recruitment 

interventions):

N = 5

KQ 5 

(training):

N = 23

*Some studies address more than one key question.

KQ 4

(retention 

interventions): 

N = 0
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

17

KEY QUESTION 1: What are the current versus projected healthcare

provider needs by numbers and disciplines in the next 20 years in 

rural areas? 

The following evidence table (Table 1) summarizes all identified studies contributing to this key 

question with associated predictions of provider needs and study conclusions. Studies estimating 

current need are listed first, followed by 3 predictive studies.

Current Healthcare Provider Need

We systematically identified published studies investigating current healthcare provider demand

and reporting specific estimates for the provider groups of interest. Identified studies used 

employer surveys,

47 state or national provider-specific files,48-51 specialty-specific national 

provider membership rosters,

50,52 and existing data sets1,52 to estimate current rural healthcare 

provider needs. Most studies reported on physicians1,47-50,52 with one broadening to include allied 

health providers, advanced practice nurses, physician assistants or dentists,

47 and 2 reported on 

mental health professionals.

50,51 Provider groups consisted of family physicians and general 

practitioners,

1,47 psychiatrists,47,50 psychiatric mental health advanced practice registered nurses

(PMH-APRNs),

51 pediatricians,47 emergency physicians,47 obstetrics and gynecology or 

women’s health providers,

47,52 and general surgeons48,49

When reporting provider need, results were presented as national unweighted or populationweighted estimates,

1,51,52 state-specific need,47-49 county-level need50 or reported vacancy rates.47

Rurality was designated according to labor market regions,

47 state designation,48 the US Office 

of Management and Budget’s statistical area definition,

52 the US Department of Agriculture 

Rural-Urban Continuum Code (RUCC),

53,54 the Rural-Urban Commuting Area (RUCA)

codes,

1,50 or were based on the National Center for Health Statistics (NCHS).51 One study did not 

define rural.49 

Predicted Future Healthcare Provider Need

We systematically identified published studies predicting future healthcare provider demand. 

Three identified studies used either state-specific physician provider files,

55 or existing national 

data sets.

53,56 Provider groups consisted of primary care physicians,55 surgeons,53 or emergency 

medicine physicians.

56 Predictive models reported tiered scenarios (best-case, worse-case, 

intermediate scenario),56 population analysis algorithms,

53 or previously described models (ie,

Health Resources and Services Administration [HRSA] model).55 Metrics for assessing need 

were not standardized across studies; 2 studies reported national need in comparison to expected 

provider-to-population-based ratio53,56 and one study reported county-based need for a state.

55

Predictive time frames were 2005 to 204056, 2011 to 2030,

53 and 2005 to 2020.55 Rurality was 

designated according to location outside of a county metropolitan statistical area,

56 RUCC,53,54 or 

RUCA classification.55 
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

18

Table 1. Evidence Table – KQ1 (Provider Need) 

ID Region

Provider

Data

Timeframe

Analysis Definition of need

Definition of rural

Results Conclusion

Current Need

Branch, 

201447

Alaska

Allied health 

and ancillary 

services, 

counselors 

and behavioral 

therapists, 

nurses, 

physicians, 

and dentists

608 

respondents 

(67%) out of 

906 surveyed 

employers

Timeframe: 

2012

For each occupation, 

sample (unweighted) 

and population 

(weighted) estimates 

were calculated

Need: Estimated rural 

vacancy rates

Rural: Labor market 

regions in Alaska, as 

defined by the 

Department of Labor: 

Anchorage (urban), 

Fairbanks (urban), 

Juneau (rural), Gulf 

Coast – Rural South 

Central (rural), North 

(rural), Rural Interior 

(rural), Rural Southeast 

(rural), and Southwest 

(rural).

Rural areas suffer from 

extreme shortages in 

traditional primary care 

occupations. Estimated rural 

vacancy rates were: family 

physicians (21%), family nurse 

practitioners (17%), physician 

assistants (19%), women’s 

healthcare nurse practitioners 

(44%), psychiatric nurse 

practitioners (16%); 

psychiatrists (15%), 

emergency physicians (21%), 

general practitioners and 

family physicians (21%), 

pediatricians (16%). Estimated 

vacancy rates for general 

practitioners and family 

physicians in North (26%), 

Southwest (25%), Gulf Coast 

Rural Southcentral (18%), and

Gulf Coast Rural Southeast 

(16%); all are rural regions

Alaska must address 

challenges inherent in 

developing and sustaining a 

high-quality and stable 

health workforce if it wants 

to maintain access to 

healthcare services and 

care for its residents far into 

the future. This can be 

accomplished by: 1) making 

long-term investments to 

prepare students in middle 

and high school for healthfocused post-secondary 

programs, 2) developing 

and sustaining postsecondary programs to 

keep Alaskans here to study 

and practice, 3) examining 

laws and restrictions related 

to barrier crimes, alerting 

Alaska’s youth to be aware 

of the career consequences 

of their actions, and 4) 

continuing to invest in statebased loan repayment and 

incentive programs for 

health providers to come to 

Alaska’s rural communities 

to practice.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

19

ID Region

Provider

Data

Timeframe

Analysis Definition of need

Definition of rural

Results Conclusion

Ghosh, 

201151

US

Psychiatric 

mental healthadvanced 

practice 

registered 

nurses (PMHAPRNs)

American 

Nurses 

Credentialing 

Center, listing of 

the employment 

ZIP codes of 

certified PMHAPRNs (N = 

10,452)

Timeframe: 

2007

Geographical analysis 

of the distribution of 

PMH-APRNs

Need: Lower than 

expected number of 

providers, populationweighted

Rural: Counties divided 

into 6 urban-rural 

categories on the basis 

of the 2006 National 

Center for Health 

Statistics (NCHS) 

classification scheme

A significant number of 

counties with low or very low 

concentration of PMH-APRNs 

were rural counties (N = 150). 

Among counties with very high 

cluster types (high 

concentration of PMJ-APRNs), 

a higher numbers of counties 

were from large central 

metropolitan (N = 35) and 

large fringe metropolitan areas 

(N = 80), emphasizing an 

uneven distribution of PMHAPRNs among urban and rural 

counties

The interdisciplinary 

approach, including both 

mapping and statistical 

analyses, identified 

shortage areas and 

provided the groundwork for 

directing future education, 

clinical practice, and public 

policy initiatives.

Hendryx, 

200854

Appalachian 

region in 13 

states

Mental health 

professionals: 

psychiatrists, 

clinical 

psychologists, 

clinical social 

workers, 

psychiatric 

nurse 

specialists, 

and marriage 

and family 

therapists

2005 Area 

Resource File 

merged with 

data from 

Bureau of 

Health 

Professions, US 

DHHS, HRSA. 

(N = 618 

counties)

Timeframe: 

2005

Descriptive and 

bivariate analyses and 

a series of maximum 

likelihood logistic 

regression analyses 

Need: HRSA-designated 

mental health 

professional shortage 

area

Rural: Appalachian 

counties that were 

designated as 

nonmetropolitan 

according to US 

Department of 

Agriculture urban 

influence codes were 

selected, and 

metropolitan counties 

(code 1 or 2) were 

excluded.

Of the 268 non-metropolitan 

Appalachian counties, 69.8% 

were designated as mental 

health professional shortage 

areas, compared to 57.7% of 

non-metropolitan, nonAppalachian counties within 

the same states (p<.002)

Appalachian location is 

associated with mental 

health professional 

shortages, but this effect is 

driven by underlying social 

differences, in particular by 

lower education. This 

method of identifying 

Appalachia for comparative 

purposes may be applied to 

many other health services 

research questions and to 

other defined geographic 

regions.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

20

ID Region

Provider

Data

Timeframe

Analysis Definition of need

Definition of rural

Results Conclusion

Maizel, 

200948

Maryland

Surgeons

Maryland Board 

of Physicians 

licensing files 

for 2006-2007 

and interviews 

with medical 

directors of 52 

acute care 

hospitals

Timeframe: 

2006 to 2007

Clinical FTE per 

100,000 residents

Need: Benchmark 6.4 

practicing general 

surgeons per 100,000 

residents

Rural: western (4 

counties), southern (3 

counties), and eastern (9

counties) regions.

Metropolitan: central (5 

counties and Baltimore).

Suburban: capital region 

(2 counties)

The number of surgeons 

providing care to patients per 

100,000 residents was below 

reported requirements in 

general surgery (western: 6.5, 

southern: 3.28, eastern: 6.26; 

vs central: 6.15, capital: 4.02). 

Surgeons in rural areas spent 

86.3% of their time on patient 

care, as compared to 70.3% of 

surgeons in urban, suburban, 

or teaching settings 

Critical shortages of 

qualified surgeons currently 

exist in many regions of 

Maryland, especially in rural 

regions. Administrative, 

teaching, and research 

activities significantly reduce 

the amount of time 

surgeons are able to devote 

to patient care, particularly 

in academic and suburban 

settings. Fewer surgeons 

are available to care for 

patients in Maryland, and 

they are significantly older 

than assumed in manpower 

databases. Access to 

surgical care in Maryland 

will be jeopardized if these 

issues are not considered in 

future healthcare workforce 

discussions.

Rayburn, 

201252

US

Obstetricians 

and 

gynecologists 

(ob-gyn) 

fellows and jr. 

fellows

2010 US 

County Census 

File for women 

of reproductive 

age and the 

2010 ACOG 

membership 

roster

Timeframe: 

2010

Ob-gyn distribution was 

divided into 2 county 

groups and state data 

were categorized as 

density of ob-gyns by 

state and district 

Need: Absence of obgyn; benchmark: 1 per 

10,146 general 

population or 2.5 per 

10,000 women

Rural: US Office of 

Management and 

Budget's statistical area 

definitions

Density of ob-gyns declined 

from metropolitan to 

micropolitan and to rural 

counties. The mean population 

in counties with no ob-gyns 

was much lower than in 

counties with 1 or more obgyns 

An uneven distribution of 

ACOG Fellows and Junior 

Fellows in practice exists 

throughout the United 

States and may worsen if 

resident graduates continue 

to cluster in metropolitan 

areas. Meeting the needs of 

women in underserved 

areas requires creative 

innovations in enhancing a 

more uniform geographic 

distribution of providers.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

21

ID Region

Provider

Data

Timeframe

Analysis Definition of need

Definition of rural

Results Conclusion

Rosenblatt, 

20101

US

Medical school 

graduates 

1988-1997

2005 AMA and 

AOA Masterfile

Timeframe: 

2005

Cross-sectional 

analysis

Need: Relative shortage 

of rural compared to 

urban physicians

Rural: ZIP code-derived 

RUCA code; urban, large 

rural, small rural, and 

isolated small rural 

categories

Relative shortage of 

physicians in rural areas 

remains: urban areas have 

210 physicians per 100,000 

people, isolated small rural 

areas have 52. Generalists 

represent 35.9% of all 

physicians in the US but 

account for almost half of all 

physicians in large rural areas, 

indicating rural areas rely on 

physicians for their healthcare. 

Specialty supply diminishes as 

areas become smaller and 

more remote 

The precipitous decline in 

the number of US medical 

graduates choosing family 

medicine residencies, and 

the decline in the number of 

graduates from these 

residencies despite the 

importation of large 

numbers of international 

medical graduates, has led 

to increasing shortages of 

rural physicians and 

threatened the integrity of 

the rural healthcare system. 

Future projections of 

population growth suggest 

that the shortages will 

worsen unless the private 

and public sectors work 

together to change the 

dynamics that affect the 

choice of medical career 

and practice location.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

22

ID Region

Provider

Data

Timeframe

Analysis Definition of need

Definition of rural

Results Conclusion

Stewart, 

201349

Texas

General 

surgeons

Texas Medical 

Board, the US 

Census Bureau/ 

Texas State 

Library and 

Archives 

Commission, 

and the Texas 

Department of 

State Health 

Services

Timeframe: 

2002 to 2012

Unadjusted data and 

data normalized per 

100,000 population for 

2002 and 2012

Need: Benchmark 

standard: 7 general 

surgeons per 100,000 

population

Rural: N/A

From 2002-2012, the Texas 

population increased 21%, 

actively practicing physicians 

44%,and general surgeons 

increased 4%. The number of 

general surgeons per 100,000 

population decreased 14% 

(P<.01). In 2012, 329 

additional general surgeons 

were needed by benchmark 

standards. When analyzed by 

county, 449 additional general 

surgeons were needed in the 

individual counties. Effects 

were greater in 

nonmetropolitan areas (per 

capita general surgeons 

decreased by 21%)

The absolute increase in 

Texas general surgeons 

over the past decade has 

not kept pace with an 

increase in the Texas 

population. The general 

surgery workforce deficit 

based on the Texas state 

population underestimates 

the local workforce 

shortage, particularly in the 

nonmetropolitan areas of 

Texas.

Thomas, 

200950

US, counties

Psychiatrists, 

psychologists, 

advanced 

practice 

psychiatric 

nurses, social 

workers, 

licensed 

professional 

counselors, 

marriage and 

family 

therapists

Supply data 

compiled from 

professional 

associations, 

state licensure 

boards, and 

national 

certification 

boards

Timeframe: 

2006

County-level need 

measured by 

estimating prevalence 

of serious mental 

illness, combining 

separate estimates of 

provider time needed 

derived from National 

Comorbidity Survey

Replication, US 

Census, and Medical 

Panel Expenditure 

Survey data; shortage 

measured for 

prescribers, 

nonprescribers, and a 

combination; ordinary 

least-squares 

regression identified 

county characteristics 

associated with 

shortage

Need: Shortage = % 

unmet need for mental 

health visits within 

county

Rural: 9-point rural-urban 

continuum code

77% of counties had a severe 

shortage of mental health 

prescribers or non-prescribers, 

with over half their need 

unmet. 96% had at least some 

unmet need for prescribers. 

Rurality and per capita income 

were the best predictors of 

unmet need. A 1-point 

increase in rurality on the 

Rural-Urban Continuum Code 

corresponded to an increase 

in unmet need of 3.3% points

These findings identified 

widespread prescriber 

shortage and poor 

distribution of 

nonprescribers. A caveat is 

that these estimates of need 

were extrapolated from 

current provider treatment 

patterns rather than from a 

normative standard of how 

much care should be 

provided and by whom. 

Better data would improve 

these estimates, but future 

work needs to move beyond 

simply describing shortages 

to resolving them.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

23

ID Region

Provider

Data

Timeframe

Analysis Definition of need

Definition of rural

Results Conclusion

Predicted Need

Camargo, 

200856

US

Emergency 

medicine 

physicians

2005 NEDI-USA

Timeframe: 

2005 to 2040

3 models (best-case, 

worst-case, 

intermediate scenario), 

Emergency Medicine 

Workforce Calculator 

(http://www.emnetusa.org/nedi/workforce.

html)

Need: Demand estimate 

accounted for attrition, 

demand = # ED visits in 

2005 per 3,548 visits, 

assuming 1 emergency 

physician 24/7 coverage

= 5.35 FTE

Rural: Located in a 

county not in a 

metropolitan statistical 

area

A total of 40,030 emergency 

physicians were needed to 

staff all EDs (55% of demand 

met); 6,450 (16%) were 

needed in rural EDs and 

33,580 (84%) in non-rural EDs

Supply of EM residencytrained, board-certified 

emergency physicians is not 

likely to meet demand in the 

near future; alternative 

emergency physicians 

staffing arrangements merit 

further consideration.

Williams, 

201153

US

Surgeons (obgyn and 

general 

surgeons)

AHA’s Fast 

Facts survey in 

2008

Timeframe: 

2011 to 2030

Population analysis 

algorithm; Census

figure of 309 million in 

2010 as a baseline for 

US population, 3,012 

urban hospitals, and 

1,998 rural hospitals

Need: Ratio: 7.5 general 

surgeons per 100,000

Rural: RUCA 

classification

From 2011-2030, rural 

hospitals will have to recruit an 

average of 3.4 ob-gyns, 1.6 

orthopedics, and 2.0 general 

surgeons for a total of 7 FTEs. 

Urban hospitals have to recruit 

10 ob-gyns, 6 general 

surgeons, 5 ear, nose, throat 

surgeons, 2.5 urologists,1 

neurosurgeon, and 1 thoracic 

surgeon

Rural hospitals will be in 

competition with urban 

hospitals for hiring from a 

limited pool of surgeons. As 

urban hospitals have a 

socioeconomic advantage in 

hiring, surgical care in rural 

areas may be at risk. It is 

imperative that each rural 

hospital analyze local future 

healthcare needs and 

devise strategies that will 

enhance hiring and 

retention to optimize access 

to surgical care.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

24

ID Region

Provider

Data

Timeframe

Analysis Definition of need

Definition of rural

Results Conclusion

Wilson, 

201155

Kentucky

Primary care 

physicians

Kentucky Board 

of Medical 

Licensure, Area 

Resource File, 

US Census 

Small Area 

Health 

Insurance 

Estimates, 

Kentucky State 

Data Center, 

and National 

Resident 

Matching 

Program

Timeframe: 

2005 to 2020

HRSA model applied to 

Kentucky's 120 

counties

Need: Physician Supply 

Model and Physician 

Requirements Model 

(HRSA) used to estimate 

need (preventive, acute, 

and chronic care) and 

demand (all healthcare 

services wanted)

Rural: US Department of 

Agriculture’s RUCC

1,527 additional primary care 

physicians are needed to meet 

projected needs and 1,888 

additional primary care 

physicians are required to 

meet projected demands by 

2020 in Kentucky. 43% of the 

population resides in rural 

communities

No single policy can solve 

the shortage of primary care 

physicians; therefore, 

multiple approaches must 

be used at the local, state, 

and national levels; a new 

system of care, patient 

centering, to reform the 

healthcare system is also 

suggested.

Note: ACOG = American Congress of Obstetricians and Gynecologists, AHA = American Hospital Association, AMA = American Medical Association, AOA = American 

Osteopathic Association, 95% CI = cconfidence interval, DHHS = Department of Health and Human Services, FTE = full-time equivalent, HRSA = Health Resources and Services 

Administration, NCHS = National Center for Health Statistics, NEDI = National Emergency Department Inventories, PA = physician assistant, PMH-APRNs = psychiatric mental 

health-advanced practice registered nurses, RUCA = Rural-Urban Commuting Area, RUCC = Rural-Urban Continuum Codes
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

25

Summary of Findings and Quality of Evidence for Key Question 1

The summary of findings table summarizes the results for current and future provider demand 

across the included studies. 

Table 2. Summary of Findings – KQ1

Provider need # 

studies

Results for rural health

Current demand

Primary care providers 2 Isolated rural areas have one-quarter the ratio of physicians per 

capita1

Rural areas in Alaska continue to suffer from extreme shortages in 

traditional primary care occupations47

Psychiatrists and mental 

health professionals 

2 77% of counties in US had severe shortage of psychiatrists.50

Rural areas of Appalachia more likely to be designated mental 

health professional shortage areas 54

General surgeons 2 Critical shortage in rural areas of Maryland 48

Per capita general surgeons in Texas decrease 21% from 2002-

2012; 449 additional general surgeons required in individual 

counties49

Ob-Gyn in rural areas 1 Rural areas currently underserved for ob-gyn based on national 

data52

PMH-APRNs 1 Analysis of lower than expected numbers of PMH-APRNs highlights 

the uneven distribution among urban and rural counties51

Future demand

Primary care physicians 1 Approximately 1700 additional primary care physicians needed in 

Kentucky by 2020 55

Emergency physicians 1 16% of emergency physicians need to be recruited to rural 

locations; demand is unlikely to be met based on national data56

Surgeons (ob-gyn and 

general surgeons)

1 Rural hospitals need to hire 7 physicians (on average) by 2030 to

compete with urban hospitals, based on national data53

Although we identified 2 studies reporting on primary care provider needs, mental health 

professionals, and general surgeons, the included studies quantified healthcare provider needs for 

specific regions and specific years, and operationalized provider need differently. Hence it is 

difficult to make concrete evidence statements for the number of healthcare providers needed 

across the studies. However, all included studies reported current unmet healthcare provider 

needs that worsen with increasing rurality. Based on the existing evidence base, our confidence 

is high that there is a shortage for primary care providers, mental health professionals, and 

general surgeons. 

We only identified one study each reporting on obstetrics and gynecology physicians and 

psychiatric mental health advanced practice registered nurses. Both studies did not provide an 

estimate of how many providers are needed to meet demand but pointed to the absence of 

providers in rural areas and reported a decreasing density from metropolitan to rural counties. 

We identified 3 studies estimating future provider demand. All included studies predicted unmet 

provider needs that worsen with increasing rurality. However, the identified studies reported on 

individual healthcare provider groups: primary care physicians, emergency physicians, and 
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

26

obstetrics/gynecology and general surgeons. We did not identify more than one study reporting 

on the same provider group. The studies reported specific estimates of how many additional 

physicians are needed, but it is difficult to judge our confidence in the estimates in the absence of 

replication, in particular given the complexity of future demand estimates. However, all 3 studies 

concluded that the supply is not likely to meet demand, regardless of the individual provider 

group studied.

We did not identify any published estimates from VA datasets.

KEY QUESTION 2: What factors influence healthcare providers’ 

geographic choices for practice? 

We systematically identified studies investigating factors that may influence providers’ 

geographic choices for practice with regard to rural healthcare. A large number of studies 

contributed to this key question. Some identified studies used provider surveys and self-reported 

reasons for selecting rural healthcare; other studies were qualitative analyses of in-depth 

interviews with a small group of healthcare providers; some studies used existing datasets to 

identify predictors of practicing in rural healthcare. 

The large majority (19) of identified studies evaluated physicians, 3 studies explored the choices 

for practice in physician assistants, and one included a range of healthcare providers eligible for 

a loan repayment program. Some studies differentiated allopathic and osteopathic degrees, and 

several limited to primary care physicians, emergency department physicians, and general 

surgeons. All studies addressed reasons for choosing to practice in rural areas in the last 10 years, 

that is from 2005 onwards. Of those studies that limited their study to a particular year, 4 used a 

2005 dataset,57-60 2 a 2007 dataset,61,62 and 2 a 2009 dataset.

63,64 The other studies used other 

years or a range of years with data for practicing in rural care. The number of participants ranged 

from 865 to datasets with information on over 175,00057 healthcare providers.

The majority of studies reported on practicing in rural care. Others investigated specific 

outcomes such as practicing in a rural satellite clinic, or practicing in a small town. Although a 

number of studies used an urban-rural continuum to differentiate practice locations, outcomes 

were analyzed dichotomously (eg, rural versus not). Definitions of “rural” referred to federal 

taxonomies such as RUCA codes, US Department of Agriculture county-based Urban Influence 

Codes, RUCC, and the Rural Office of Management and Budget designation, and state 

definitions, including the Illinois Department of Public Health designations of rural areas and the 

California Office of Statewide Planning and Development (OSHPD) definition of Rural Medical 

Service Study Areas. One study used the Rural-Urban Density Typology (RUDT) based on 

population density and 2 studies referred to a population of less than 50,000 people. Other 

studies used local definitions (eg, the Hawaiian island O’ahu was considered urban, all other 

areas rural) or did not define “rural.”

The evidence table summarizes all identified studies contributing to this key question. We 

differentiated the dependent variables (ie, the predictors or named choices for choosing rural 

care) by demographic background factors, variables associated with the training of the provider, 

financial aspects and incentives, aspects of the rural area, and other relevant results. The 

evidence table also shows the authors’ conclusions drawn from the study. 
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

27

Table 3: Evidence Table – KQ2 (Provider Choice Assessments) 

ID Participants N, study design, 

outcome

Results –

demographic 

background

Results – training Results –

financial 

aspects

Results – rural 

care 

characteristics

Other results Authors’ 

conclusions

Chen, 

201057

Clinically 

active MDs, 

DOs, and 

international 

medical 

graduates 

(IMGs) who 

graduated 

from medical 

school 

between 1987-

1997

Practicing 

physicians

N = 175,649

Analytic study

Rural: RUCA and 

county 

designations

Outcome: 

Practicing in rural 

location in 2005

In 2005, 18% of DOs, 

11% of MDs, and 

13% of IMGs are 

practicing in a rural 

location. 31% of rural 

physicians were 

women (37% of MDs 

and 31% of DOs 

female). An 

increasing proportion 

of female physicians 

entered rural practice 

over the study period 

(7.8% to 9.8% female 

MDs, 12.2% to 17.7% 

female DOs). Overall, 

94% of physicians 

were MDs and 6% 

DOs, but 18% of DOs 

and 11% of MDs 

practice in a rural 

care

Of the 1.4% MDs 

trained in rural 

residency, 36% were 

in rural practice. Of 

the 3.6% DOs trained 

in a rural residency, 

50% were in rural 

practice. Rural 

residents were 3x 

more likely to practice 

in rural areas (RR 

3.4, P<.001). Rural 

residents account for 

5% of MDs and 10% 

of DOs in rural areas. 

60% of rural family 

medicine residents 

were in rural practice 

and were 3x more 

likely to practice in 

rural care (RR 2.8, 

P<.001). Only 9% of 

rural family providers 

trained in a rural 

residency 

N/A N/A N/A The proportion 

and number of 

physicians 

entering rural 

practice has 

remained stable 

compared with 

earlier analyses. 

However, recent 

trends such as 

declining 

primary care 

interest are not 

yet reflected in 

these data and 

may portend 

worsening 

shortages of 

rural physicians.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

28

ID Participants N, study design, 

outcome

Results –

demographic 

background

Results – training Results –

financial 

aspects

Results – rural 

care 

characteristics

Other results Authors’ 

conclusions

DHHS, 

200666

US

18 specialties

N = N/A

Analytic study

Rural: N/A

Outcome: 

Working in rural 

area

Female physicians 

are less likely to work 

in rural areas

N/A N/A N/A N/A The growth and 

aging of the US 

population will 

cause a surge 

in demand for 

physician 

services; if 

current 

healthcare 

utilization and 

delivery 

patterns 

continue, the 

overall supply of 

physicians 

should be 

sufficient to 

meet the 

expected 

demand through 

the next 10 

years.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

29

ID Participants N, study design, 

outcome

Results –

demographic 

background

Results – training Results –

financial 

aspects

Results – rural 

care 

characteristics

Other results Authors’ 

conclusions

Duffrin, 

201467

Current

members of 

the North 

Carolina 

Medical Board 

who are listed 

as primary 

care 

physicians

Primary care 

physicians 

practicing in 

family 

medicine, 

internal 

medicine, obgyn, general 

practice, and 

pediatrics

N = 975

Survey

Rural: County of 

<50,000 people

Outcome: 

Practicing in rural 

area and 

practicing 

physicians in 

2012

Population of 

hometown =<11,000 

was associated with 

working in non-metro 

area (p = .007, 1st 

subgroup)

N/A Pay as a factor in 

choosing a work 

site, financial 

support from a 

hospital, and 

medical school 

loan repayment 

were correlated 

with rural practice 

(effect size not 

reported)

N/A Federal and 

state incentives 

should continue; 

having been 

raised in an 

area of 11,000 

or less was 

highly predictive 

of future rural 

medical practice 

and could be 

used in the 

recruitment of 

physicians and 

residents to 

increase the 

ultimate yield for 

rural areas.

Fordyce, 

201258

MDs and DOs 

from 2005 

AMA and AOA 

Masterfiles

Practicing 

physicians, 

non-federally 

employed, 

aged 70 years 

or younger

N = 231,660

Analytic study

Rural: RUCA 

classification 

(urban, large 

rural, small rural, 

or isolated small 

rural)

Outcome: 

Practicing in rural 

areas in 2005

IMGs comprised 

22.2% of total 

clinically active 

workforce, but 

contributed 19.3% to 

the rural PCP 

workforce, some 

geographic variation. 

IMG PCPs were more 

likely than other PCPs 

to practice in rural 

persistent poverty 

locations (12.4% vs

9.1%). The proportion 

of rural PCP 

workforce 

represented by IMGs 

decreased with 

increasing rurality.

DOs comprised 4.9% 

of clinically active 

workforce but 

contributed 10.4% to 

rural PCP workforce, 

some geographic 

variation. DO PCPs 

were more likely than 

allopathic PCPs to 

practice in rural 

places (20.5% vs

14.9%). Proportion of 

rural PCP workforce 

represented by DOs 

increased with 

increasing rurality

N/A N/A N/A DO and IMG 

PCPs constitute 

a vital portion of 

the rural 

healthcare

workforce; their 

ongoing 

participation is 

necessary in 

addressing 

existing rural 

PCP shortages 

and handling 

the influx of 

newly insured 

residents as the 

ACA comes into 

effect.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

30

ID Participants N, study design, 

outcome

Results –

demographic 

background

Results – training Results –

financial 

aspects

Results – rural 

care 

characteristics

Other results Authors’ 

conclusions

Glasser, 

201068

Recently

located rural 

physicians and 

graduates of a 

rural medical 

education 

program in 

rural Illinois

Interviewees (N = 

20), graduates (N

= 107) 

Other design

Rural: Illinois 

Department of 

Public Health 

designations of 

rural areas

Outcome: 

Practicing in rural 

county 

Major reason for 

practicing in a rural 

location was family 

ties to the community 

(50%).

N/A 2nd major reason 

for practicing in a 

rural location was 

a loan or 

scholarship 

obligation (30%).

Top attributes 

potentially 

affecting 

retention were 

hard work/long 

hours (25%) 

and patients not 

paying enough 

or being 

uninsured 

(20%). Most 

positive factors 

were patients’ 

appreciation 

(45%), lifestyle/ 

work-life 

balance (20%), 

and meeting 

individual and 

community 

needs (20%)

N/A Keys to success 

in rural 

physician 

retention seem 

to include 

identifying and 

recruiting 

medical 

students of rural 

origin and 

focusing on a 

healthy practice 

environment; 

policy makers 

need to work 

with local 

government, 

schools and 

employers to 

offer programs 

to identify local 

youth for 

induction in 

rural healthcare.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

31

ID Participants N, study design, 

outcome

Results –

demographic 

background

Results – training Results –

financial 

aspects

Results – rural 

care 

characteristics

Other results Authors’ 

conclusions

Hancock, 

200969

PCPs 

practicing in 

rural 

northeastern 

California and 

northwestern 

Nevada

PCPs

N = 22 

interviewees

Other design

California 

OSHPD Rural 

Medical Service 

Study Areas 

(density <250 

persons per 

square mile, no 

census-defined 

place >50,000)

Outcome: 

practicing in rural 

areas in 

northeastern 

California and 

northwestern 

Nevada (2006-

2007)

Rural exposure via 

upbringing and 

recreation, and a 

history of strong 

community or 

geographic ties 

facilitates future rural 

practice.

Rural exposure via 

education facilitates 

future rural practice

N/A N/A Exposure 

facilitates 

through 

desires for 

familiarity, 

sense of 

place, 

community 

involvement, 

and selfactualization

Results support 

a focus on 

recruitment of 

rural-raised and 

communityoriented 

applicants to 

medical school, 

residency, and 

rural practice. 

Local 

mentorship and 

‘‘place-specific 

education’’ can 

support the 

integration of 

new rural 

physicians by 

promoting selfactualization, 

community 

integration, 

sense of place, 

and resilience.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

32

ID Participants N, study design, 

outcome

Results –

demographic 

background

Results – training Results –

financial 

aspects

Results – rural 

care 

characteristics

Other results Authors’ 

conclusions

Helland, 

201070

Emergency 

medicine 

residents who 

graduated 

from 2006 to 

2008 and 

practice in 

rural EDs, and 

a random 

sample 

practicing in 

urban EDs.

Emergency 

medicine 

physicians

N = 197

Survey

Rural: Based on 

US Department of 

Agriculture 

county-based 

Urban Influence 

Codes

Outcome: 

Practicing in rural 

EDs (graduated 

2006-2008)

Rural practice 

location was 

associated with 18 

childhood years in a 

rural area (42% rural 

vs 24% urban), but 

was not associated 

with 1 to 17 years or 0 

years of childhood 

spent in a rural area. 

Important factors 

reported for choosing 

practice location 

included 

family/spouse (81% 

rural vs 72% urban) 

and previous time 

spent in similar area 

(61% rural vs 53% 

urban) 

Emergency medicine 

board certification 

was associated with 

rural practice. 

Practice location was 

not significantly 

correlated with rural 

residency rotation. No 

difference was 

observed between 

rural vs urban 

providers reporting 

residency rotation as 

an important for 

choosing practice 

location. 25% of 

urban physicians 

considered practicing 

in a rural area 

immediately after 

graduation

Cost of living, 

salary signing 

bonus, and loan 

repayment were 

not rated as 

important. There 

was a significant 

difference in 

ratings between 

urban and rural 

providers for the 

importance of 

loan repayment

Important 

factors for 

choosing 

practice 

location 

included 

lifestyle (78%), 

but not access 

to CME, service 

to the 

underserved, 

autonomy/scop

e of practice, or 

access to 

specialists. 43% 

vs 56% of rural 

vs urban 

providers rated 

ED volume as 

very important; 

53% vs 68% 

access to 

amenities/recre

ation

Rural and 

urban 

physicians 

reported 

similar plans 

for duration of 

practicing in 

their type of 

area for 

greater than 

10 years 

(57% vs 60%) 

and for less 

than 2 years 

(6% vs 5%) 

Promising 

strategies for 

recruiting new 

residency 

graduates to 

rural EDs are 

selection of 

individuals with 

a rural 

upbringing and 

higher salaries; 

increasing the 

availability of 

rural rotations 

during 

emergency 

medicine 

residency also 

may help to 

motivate and 

prepare some 

new graduates 

to practice in 

rural EDs.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

33

ID Participants N, study design, 

outcome

Results –

demographic 

background

Results – training Results –

financial 

aspects

Results – rural 

care 

characteristics

Other results Authors’ 

conclusions

Heneghan, 

200571

General 

surgeons 

practicing in 

the US

General 

surgeons

N = 421

Survey

Rural: OMB 

designation, 

Goldsmith 

modification

Outcome: 

Difference in 

ratings between 

rural and urban 

surgeons

N/A N/A Reporting income 

as having a high 

impact on 

practice location 

was lower among 

rural surgeons 

(19.8% rural vs

36.1% urban, p = 

.0002)

The impact of 

potential for 

professional 

growth, 

availability of 

hospital 

facilities, quality 

of surgical 

community, and 

quality of 

medical 

community (all 

factors p<.001) 

on location 

preference 

were rated 

differently 

between rural 

and urban 

groups. Quality 

of life was not 

rated as an 

important factor

N/A Although rural 

and urban 

surgeons do not 

differ in age or 

the importance 

of lifestyle in 

deciding career 

location, 

different factors 

do impact their 

choice of 

location; 

practice pattern 

and educational 

needs varied 

markedly 

between rural 

and urban 

general 

surgeons.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

34

ID Participants N, study design, 

outcome

Results –

demographic 

background

Results – training Results –

financial 

aspects

Results – rural 

care 

characteristics

Other results Authors’ 

conclusions

Henry, 

200765

Physicians 

assistants 

(PAs) in Texas 

who work 

autonomously 

in a rural 

health clinic, 

sole PCP in 

community for 

>24 months. 

Town with 

<5,000 

persons, no 

other primary 

care within 25 

miles.

PAs

N = 8

Survey

Rural: Town with 

< 5,000 people

Outcome: 

Important factor 

influencing to 

work in a rural 

satellite clinic in 

2005

Majority (7/8) did not 

grow up in small 

town. Confidence to 

practice without 

physician was an 

important factor 

influencing to work in 

a rural clinic

N/A N/A Desire for 

small-town life, 

importance of 

knowing 

patients on a 

personal level, 

and spouse 

value of smalltown life 

influenced work 

in rural clinic

N/A In order to 

increase 

retention rates, 

PAs committed 

to autonomous, 

rural primary 

care would 

benefit from 

additional 

training, 

particularly in 

emergency 

medicine, the 

benefits of 

community 

involvement, 

and adaptation 

to the local 

culture.

Hughes, 

200572

Family 

practice 

graduates 

from UCSFFresno 

residency from 

1970-2000 

with US high 

school 

addresses

Family 

practice 

graduates

N = 178

Analytic study

Rural: RUCA 7-9 

(small towns), 10 

(rural) 

Outcome: 

Practicing in rural 

areas

Rural high school 

location was 

significantly 

associated with 

practice in rural areas 

(OR 5.7, CI 2.0 to 

16.4), controlling for 

high school in high 

minority areas, 

medically

underserved areas, 

rural training track, 

age, gender, minority, 

and Hispanic 

ethnicity. 32% of 

graduates practicing 

in a rural location 

graduated from a 

rural high school, 

compared with 11% in 

a non-rural practice

Rural training during 

residency was 

associated with rural 

practice (OR 2.7, CI 

1.2 to 6.4) when 

controlling for rural 

high school, minority 

% population of high 

school, high school in 

medically 

underserved area, 

age, gender, minority, 

Hispanic 

N/A N/A N/A Census data 

from the 

residency 

graduate’s high 

school predicted 

rural practice 

and practice in 

a proportionally 

high-minority 

community, but 

not in a 

federally 

designated 

medically 

underserved 

area.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

35

ID Participants N, study design, 

outcome

Results –

demographic 

background

Results – training Results –

financial 

aspects

Results – rural 

care 

characteristics

Other results Authors’ 

conclusions

Jarman, 

200973

Surgery 

residents 

graduates of 

all 4 Wisconsin 

programs from 

1994-2008

General 

surgeons and 

subspecialty 

surgeons who 

completed 

general 

surgery 

residency

N = 45

Survey

Rural: Rural = 

population of 

<50,000 people

Outcome: 

General surgery 

practice in rural 

location 1994-

2008

Factors associated 

with rural vs urban 

practice included 

attending a nonurban 

high school (p = .001) 

or college (p = .001), 

location spouse/ 

partner grew up (p = 

.022), and having a 

child before/during 

medical school (p = 

.043). Graduates in 

an urban setting were 

more likely to have a 

parent with a medical 

occupation (p = .03). 

Practice location was 

not associated with 

sex, birthplace (USonly, rural vs nonrural), parental 

occupation, having a 

parent who grew up 

on a farm, 

participation in high 

school or college 

sports, participation in 

high school academic 

club, playing a 

musical instrument, 

listening to all types of 

music, being married, 

being married during 

college or before, or 

having children

Factors associated 

with rural practice 

included completing a 

rural clerkship (p = 

.001) and having 

chosen a surgical 

residency program 

committed to rural 

training (p = .046). 

Factors negatively 

associated with rural 

practice included 

completion of a 

fellowship (p<.001) 

and teaching surgical 

residents (p<.001). 

Practice location was 

not associated with 

clinical research 

during residency or 

bench research 

during residency

N/A Factors 

positively 

associated with 

rural practice 

included 

interest in 

hunting birds (p

= .010) or large 

game (p = 

.001). 

Graduates in 

rural practice 

more often cited 

"broad scope of 

practice" as an 

important 

reason. 

Practice 

location was not 

associated with 

current hobbies, 

fishing, hunting 

small game, 

happiness with 

location, 

spouse's 

happiness with 

location, or 

satisfaction with 

scope of 

practice

General surgery 

residency 

graduates and 

their spouses 

who choose 

rural practices 

are more likely 

than those 

selecting urban 

practices to 

have rural 

backgrounds 

and interests; 

completing a

rural clerkship 

during medical 

school and 

choosing a 

residency 

program 

committed to 

rural general 

surgery 

preparation are 

strongly 

correlated with 

rural practice.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

36

ID Participants N, study design, 

outcome

Results –

demographic 

background

Results – training Results –

financial 

aspects

Results – rural 

care 

characteristics

Other results Authors’ 

conclusions

Kimball, 

200774

Female 

physicians 

practicing in 

rural 

Wisconsin

Physicians

N = 10

Other design

Rural: Wisconsin 

communities 

<16,000 people

Outcome: 

Motivation to 

enter rural 

practice, reasons 

for choosing 

practice location

70% had a rural 

background. 60% 

stated they were 

always interested in 

rural practice. 

Reasons for choosing 

location included 

proximity to family 

(60%) or personal 

connection to area 

(60%). 60% stated 

that family obligations 

did not influence 

decision to practice in 

current community. 

Family obligations 

(10%) and spouse's 

job location (30%) 

influenced some 

participants 

20% stated that their 

medical school had 

encouraged rural 

practice, 40% stated 

that it was 

discouraged, 40% 

stated that no specific 

practice location had 

been emphasized

N/A Reasons for 

choosing 

practice 

included liking 

the community 

(60%), good 

access to 

specialist 

backup (20%), 

and full scope 

of practice 

(10%)

N/A The participants 

provided insight 

into motivating 

woman to enter 

rural practice, 

finding a 

balance 

between the 

challenges and 

benefits of rural 

medicine, and 

promoting the 

future of rural 

healthcare. 

Mason, 

201263

1990-1999 

UMC 

graduates 

practicing in 

Mississippi 

(MS) from 

2004 MS 

Board of 

Medical 

Licensure

Physicians

N = 927

Analytic study

Rural: N/A

Outcome: 

Practicing in 

small town in 

2009

Factors not 

associated: attended 

high school in MS, 

attending college in 

MS, internship in MS, 

began practice in MS, 

moved practice to 

MS, age, sex, race, or 

marital status

UMC graduates were 

not more likely to 

practice in rural areas 

in MS than physicians 

who graduated 

elsewhere. PCPs 

were 2.4 times 

(P<.001) more likely 

to practice in small 

town areas than 

specialists (controlling 

for all other factors)

Salary or student 

loan debt were 

not predictors of 

practicing in 

small towns 

(multivariate 

analysis).

N/A N/A Health 

educators and 

policy makers 

should consider 

broadening the 

enrollment 

policies and 

greater 

emphasis 

should be 

placed on 

recruiting 

physicians.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

37

ID Participants N, study design, 

outcome

Results –

demographic 

background

Results – training Results –

financial 

aspects

Results – rural 

care 

characteristics

Other results Authors’ 

conclusions

Pepper, 

201075

Physicians in 

Wyoming

All MDs and 

DOs

N = 693

Survey

Rural: 2003 

Department of 

Agriculture RUCC 

codes 3 to 9

Outcome: 

Practicing in lesspopulated county 

in 2007

Being raised in a rural 

area was associated 

with practicing in a 

less populated county 

(p < .05) in a 

multivariate analysis. 

Living in Wyoming as 

a child (p < .099) and 

attending medical 

school in a bordering 

state (p < .01) was 

not significant. There 

was no association 

with gender, being 

raised in a bordering 

state, completing an 

internship or 

residency in a 

bordering state, or 

plans to move out of 

state

There was no 

association with 

medical school 

location

N/A N/A N/A Rural 

backgrounds 

and training 

independently 

predict practice 

location 

decisions.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

38

ID Participants N, study design, 

outcome

Results –

demographic 

background

Results – training Results –

financial 

aspects

Results – rural 

care 

characteristics

Other results Authors’ 

conclusions

Phillips, 

200960

US allopathic 

medical 

students

Physicians

N = 322,131

Analytic study

Rural: RUCA 

codes

Outcome: Rural 

practice in Rural 

Health Clinic 

(2001-2005)

Rural practice was 

strongly associated 

with being born in 

rural county (OR 

2.35). Rural practice 

was associated with 

being male (OR 1.49), 

married (OR 1.47), 

age at graduation 

(OR 1.03). Rural 

practice was 

associated with plans 

to serve in 

underserved areas 

(RR 3.40) 

Rural practice was 

associated with

attending medical 

school in rural area 

(OR 2.93), career in 

family medicine (OR 

2.65), as well as 

attending a public 

medical school (OR 

1.66), community 

related medical 

school (OR 1.20), and 

experience in Title VII 

funded school (OR 

1.11). Rural practice 

was associated with 

practice taking a rural 

(RR 1.9) or 

community health 

(RR 1.63) elective, 

family medicine 

clerkship (RR 1.44), 

experience with a 

Title VII school (RR 

1.31), primary care 

residency (RR 1.22) 

Rural practice 

was associated 

with NHSC loan 

repayment (OR 

2.06), NHSC 

scholarship (OR 

1.88), medical 

school debt 

$200-250K (OR 

1.34), medical 

school debt 

$150-200K (OR 

1.24), medical 

school debt 

$100-150K (OR 

1.29), medical 

school debt $50-

100K (OR 1.19),

and medical 

school debt $1-

50K (OR 1.06) 

N/A N/A If rural–born 

students 

interested in 

serving the 

underserved 

also have rural 

training 

experience, it 

may have 

“multiples of 

effect”. Schools, 

residency 

programs, and 

medical 

education 

funders should 

consider this. 

Schools should 

institute a series 

of interview 

questions about 

rural and other 

underserved 

patients and 

should give 

these weight in 

acceptance.

They could also 

become 

markers for 

targeted 

mentoring and 

training 

experiences.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

39

ID Participants N, study design, 

outcome

Results –

demographic 

background

Results – training Results –

financial 

aspects

Results – rural 

care 

characteristics

Other results Authors’ 

conclusions

Rabinowitz, 

201261,76

MD graduates 

from Jefferson 

Medical 

College

Physicians

N = 3006

Analytic study

Rural: Rural 

county = 2007 

Rural-Urban 

Density Typology 

(RUDT)

Outcome: 

Practicing in rural 

area in 2007

3 predictors of rural 

practice (p<.001): 

growing up in a rural 

area, entering 

medical school with 

plans for rural 

practice, and entering

medical school with 

plans to be a family 

physician. Of 

graduates with all 

predictors, 45% 

practiced in rural 

areas; of those with 2, 

33%; of those with 1, 

1%; and of those with 

0, 12% practiced in 

rural areas. The RR 

for rural practice was 

3.9 (CI 2.7-5.7, p < 

.001) for those with 3 

predictors, 2.9 (CI 

2.0-4.2, p < .001) for 

those with 2 

predictors, and 1.8 

(CI 1.2-2.8, p < .01) 

for those with 1 

predictor. Medical 

students' specialty 

plans were strongly 

related to rural 

practice (p < .001)

N/A N/A N/A N/A Three factors 

known at the 

time of medical 

school 

matriculation 

have a powerful 

relationship with 

rural practice 3

decades later; 

relatively few 

students without 

predictors 

practice in rural 

areas, which is 

particularly 

significant given 

subsequent 

factors known to 

be related to 

rural practice – 

for instance, 

rural curriculum, 

residency 

location or 

spouse.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

40

ID Participants N, study design, 

outcome

Results –

demographic 

background

Results – training Results –

financial 

aspects

Results – rural 

care 

characteristics

Other results Authors’ 

conclusions

Renner, 

201062

Colorado

Healthcare

professionals 

that 

participated in 

loan 

repayment 

program

N = 93

Survey

Rural: ZIP code 

with RUCA 

designation 

above and 

including 4.0

Outcome: 

Practicing in rural 

care in 2007

Rural providers were 

more likely to have 

gone to a rural high 

school than urban 

providers (38% vs

9%, p = .007) 

N/A 58% reported 

that salary was 

an important 

factor. Signing 

bonus, amount of 

loan repayment, 

and other 

incentives were 

less important. 

42% reported the 

loan program had 

an important 

influence on the 

specific 

community they 

chose to practice

Rural providers 

rated location 

(83%), scope of 

practice (79%),

and family fit 

with community 

(73%) as the 

most important 

factors. School 

opportunity for 

children were 

rated less 

important for 

rural providers

N/A Loan repayment 

programs 

targeting rural 

Colorado 

usually enroll 

providers who 

would have 

worked in a 

rural area 

regardless of 

loan repayment 

opportunities, 

but are likely to 

play a role in 

provider's 

choice of 

specific rural 

community for 

practice. 

Schiff, 

201277

Physicians 

practicing in 

Hawaii who 

graduated 

from the 

University of 

Hawaii School 

of Medicine 

from 1993-

2006.

Physicians (all 

specialties)

N = 177

Analytic study

Rural: O‘ahu 

considered urban; 

all other islands 

rural

Outcome: 

Practicing in rural 

settings in Hawaii 

(1993-2006)

Hawaii-schooled 

physicians who 

attended rural high 

schools were 9x more 

likely to practice in a 

rural location than 

those who went to 

high school on a 

neighbor island 

(p<.0001)

No significant 

association between 

rural practice and 

primary care specialty 

(p = .09)

N/A N/A N/A If the State of 

Hawaii wants to 

expand the 

physician 

workforce in the 

rural areas of 

Hawaii, 

recruiting more 

students from 

rural areas is an 

excellent path to 

take.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

41

ID Participants N, study design, 

outcome

Results –

demographic 

background

Results – training Results –

financial 

aspects

Results – rural 

care 

characteristics

Other results Authors’ 

conclusions

Shannon, 

201178

Physician 

assistants in 

West Virginia 

who 

completed a 

rural rotation 

during clinical 

education

Physician 

assistants 

(PAs)

N = 168

Analytic study

Rural: ZIP code 

approximation of 

the RUCA code 

classification

Outcome: Rural 

practice in West 

Virginia in 2005-

2010

Using gender, school, 

student evaluation of 

rural field experience, 

change in interest in 

rural health, 

confidence in 

community activities, 

confidence in meeting 

the needs of rural 

populations, rural high 

school hometown, 

rural practice intent, 

likelihood of West 

Virginia (WV)

practice, only the 

variables rural high 

school hometown and 

likelihood of WV

practice correctly 

predicted rural 

practice; 77% of 

students who 

predicted rural 

practice were in rural 

practice, 63% who did 

not predict it were in 

rural practice (p<.04)

N/A N/A Confidence in 

community 

activities, and 

confidence in 

meeting the 

needs of rural 

populations, did 

not predict rural 

practice

N/A This study 

suggests 

moderate 

predictive 

validity of PA 

student 

reporting on 

rural practice 

and on West 

Virginia practice 

intent; such 

methods may 

have potential in 

prediction of the 

future rural PA 

workforce.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

42

ID Participants N, study design, 

outcome

Results –

demographic 

background

Results – training Results –

financial 

aspects

Results – rural 

care 

characteristics

Other results Authors’ 

conclusions

Smith, 

201264

US

PAs

N = 312

Survey

Rural: < = 50,000 

people

Outcome: Rural 

setting for first 

practice in 2009

Respondents who 

graduated from a 

rural high school and 

singles were 

significantly more 

likely to practice in a 

rural setting (both 

p<.05). No significant 

difference between 

rural practice and 

degree, race, gender, 

and age at 

graduation. Support 

of and for significant 

other was the most 

important factor for 

first practice location

Specialty distribution 

(primary care, 

specialty, other) was 

significantly different 

between urban and 

rural groups (p<.05)

N/A N/A Six factors 

emerged from 

factor 

analysis: 

hours of 

work/ 

compensation

, support 

of/for 

significant 

other, 

community 

and job 

amenities, 

educational 

resources 

/access to 

care, practice 

opportunities, 

and location

Respondents 

felt that support 

of and for the 

significant other 

was the most 

important factor 

in their first 

practice-location 

choice; 

recruiters may 

wish to pay 

closer attention 

to spousal 

opportunities 

and should not 

underestimate 

the impact of 

family in the 

decision about 

work location.

Snyder, 

201479

PAs actively 

practicing in 

Indiana who 

graduated 

from 2000-

2010, and had 

email 

addresses 

available.

PAs

N = 157

Survey

Rural: 

Respondentdefined

Outcome: 

Location of initial 

job, location of 

current job (2000-

2010)

N/A 71% indicated 

educational dept had 

no influence on 

location of initial job 

Males were more 

likely to perceive 

debt as 

influencing initial 

job location (χ2 = 

11.65, p<.05).

N/A 34% of urban 

PAs would 

have 

practiced in a 

rural area if 

they had 

received 

federal or 

state loan 

forgiveness 

for 

educational 

debt, 30% 

would have 

reconsidered.

This study 

provides 

evidence that 

debt may 

influence 

practice 

specialty and 

location choice. 

Further studies 

are needed to 

determine how 

gender might 

account for 

decisions to 

practice in 

certain 

specialties and 

location.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

43

ID Participants N, study design, 

outcome

Results –

demographic 

background

Results – training Results –

financial 

aspects

Results – rural 

care 

characteristics

Other results Authors’ 

conclusions

Stenger, 

200880

Massachusetts 

physicians 

practicing in 

areas of the 

state 

designated as 

rural from 

2004-2005

Practicing 

physicians

N = 160

Survey

Rural: Nonmetropolitan 

county 

designated nonurban by RUCA 

or population < 

10,000 and 

population 

density < 500 

people per 

square mile

Outcome: 

Practicing in rural 

area in 2004-

2005

Most rural physicians 

(73.2%) had grown up 

in larger towns with 

populations of 10,000 

or greater

N/A N/A Responses to 

why remaining 

in rural practice 

included feeling 

established with 

a strong sense 

of connection to 

patients and 

place, overall 

satisfaction with 

practice, and 

being in a great 

place to live

Factors 

associated 

with higher 

satisfaction 

included not 

feeling 

overworked 

(p = .043), or 

professionally 

isolated (p = 

.004), and 

being 

involved in 

practice (p = 

.045) and 

community (p 

= .036).

The findings 

reaffirm the 

importance of 

rural medical 

education 

opportunities in 

physician 

recruitment, 

retention, and 

practice 

satisfaction and 

indicate that a 

major source of 

physicians for 

rural and small 

town 

communities is 

physicians who 

have been 

raised in 

urban/suburban 

communities 

and who were 

trained outside 

of the region but 

who were 

prepared to live 

and to practice 

in rural and 

small town 

communities.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

44

ID Participants N, study design, 

outcome

Results –

demographic 

background

Results – training Results –

financial 

aspects

Results – rural 

care 

characteristics

Other results Authors’ 

conclusions

Whitacre, 

201081

Medical 

students at 

Oklahoma 

State 

University 

College of 

Osteopathic 

Medicine

Medical 

students 

between 1997-

2002

N = 190

Analytic study

Rural: population 

< 50,000

Outcome: Rural 

practice location

Being raised in a rural 

area was associated 

with rural practice 

(p<.05)

Rural practice was 

associated with 

summer rural 

externship (p<.05), 

but not with all early 

clinical experiences

N/A N/A N/A Programs 

implemented by 

an osteopathic 

school can 

influence where 

graduates 

choose to 

practice; 

programs taking 

place in both 

the 1st and 4th 

year of training 

have an impact 

on rural practice 

location, 

implying that 

students can be 

influenced both 

early and late in 

their medical 

school careers.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

45

ID Participants N, study design, 

outcome

Results –

demographic 

background

Results – training Results –

financial 

aspects

Results – rural 

care 

characteristics

Other results Authors’ 

conclusions

Zink, 

201082

Medical 

students at 2 

University of 

Minnesota 

campuses that 

did or did not 

participate in a 

rural training 

program

Medical 

students

N = 3,365

Analytic study

Rural: OMB’s 

definitions; 

communities not 

listed were 

considered 

metropolitan if 

within 50 miles of 

an urban center, 

otherwise 

designated as 

rural

Outcome: Rural 

practice in 2008

In multivariate 

analysis, rural 

practice was 

associated with being 

raised in a rural 

community (OR 2.82, 

CI 2.1-3.79), but not 

with being raised in 

rural community and 

training in a rural 

program (OR.63, CI 

0.35-1.15), or being 

raised in a rural 

community and 

Duluth campus (OR 

0.56, CI 0.33-0.96)

Rural practice was 

associated with 

participating in a rural 

program (OR 4.62, CI 

3.01, 7.07) and 

training at Duluth 

campus (rural focus) 

(OR 4.09, CI 2.81, 

5.96)

N/A N/A N/A RPAP and 

UMN-Duluth 

provide 

significant, 

complementary 

educational 

programs that 

lead more 

graduates to 

choose rural 

and primary 

care practices; 

efforts across 

the nation to 

address the 

crisis in rural 

primary care 

should build on 

these 

successful 

efforts.

Note: AMA = American Medical Association, AOA = American Osteopathic Association, CI = 95% Confidence interval, CME = Continuing medical education, DHHS = 

Department of Health and Human Services, DO = Doctor of osteopathic medicine, ED = emergency department, IMG = International medical graduates, MD = allopathic medical 

doctor, NHSC = National Health Service Corps, Ob-gyn = obstetricians and gynecologists, OR = odds ratio, OSHPD = Office of Statewide Health Planning and Development, PA

= physician’s assistant, PCP = primary care provider, OMB = Office of Management and Budget, RR = risk ratio, RUCA = Rural-Urban Commuting Area, RUCC = Rural-Urban 

Continuum Codes, UCSF = University of California-San Francisco
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

46

Summary of Findings and Quality of Evidence for Key Question 2

This section summarizes the relative importance of individual factors that have been assessed in 

more than one study. The predictors are documented in the order of provider characteristics, 

provider training, financial aspects, and setting characteristics. Most studies asked or obtained 

responses regarding the demographic background of healthcare providers currently practicing in 

rural care. Half the studies addressed the training of healthcare providers (eg, graduating from a 

rural track school). A third of studies addressed financial aspects such as debt. About as many 

studies addressed aspects of rural communities or aspects of rural healthcare (eg, access to 

support from specialists). 

The number of studies shown in Table 4 reflects how many studies have addressed the predictor 

variable regardless of the direction and strength of association with the outcome variable 

(entering rural healthcare practice). The table differentiates positive associations and results 

suggesting no associations. Results from multivariate analyses that control for confounding 

variables are listed first, followed by other results. 

The table also documents the quality of evidence (QoE) which influences how confident we are 

in making evidence statements based on the identified research. Key criteria were the number of 

available studies reporting on the variable of interest, confirmation in multivariate analyses, and 

conflicting results across studies.

Table 4: Summary of Findings and Quality of Evidence – KQ2 

Predictor 

variable

# 

studies

Results and consistency across studies GRADE

Provider Characteristics

Proxy for rural 

background

16 Association

Significant association with rural high school in multivariate analysis72,78

Significant association with being raised in rural area in multivariate 

model81,82

Significant correlation with nonurban high school or college73

Significant difference between groups due to rural high school64

Significant association with population of hometown67

Qualitative analysis suggests rural exposure via upbringing69

Significant difference due to rural childhood70,77

70% of rural providers had a rural background74

Born in rural county increased odds (OR 2.65)60

Being raised in a rural area was associated with practicing in a very rural 

area vs rural area in a multivariate analysis75

A combination of growing up in a rural area, plans to practice in rural 

area, and plans for family medicine showed a positive association61

Higher proportion of attending rural high school in rural vs urban 

providers62

No association

Majority of rural providers did not grow up in small town65,80

High

Gender 8 Association

Being male increased odds (OR 1.49)60

Slightly smaller number of female rural practitioners than in overall 

population57

Conclusion female physicians are less likely to practice in rural areas66

No association

No association with gender in multivariate analysis63,75,78

No difference by gender groups64,70,73

Very low
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

47

Predictor 

variable

# 

studies

Results and consistency across studies GRADE

Family 7 Association

Family ties reported as major reason68

Family/spouse reported to be a very important factor70

Significant association with location partner grew up in73

Proximity to family listed as motivation74

Significant association with having a child during or before med school73

Conclusion that support of and for significant other is most important 

factor64

No association

Having children not associated with practice location73

Family obligation did not influence decision74

Job of spouse was rated as very important only by 28% of participants62

Spouse’s job location was cited by only 30%74

Low

Age 3 No association

Age not associated with practicing in rural area 72

Age not associated with practicing in small town63

Age at graduation not associated with rural setting for first practice64

Age at graduation OR 1.0360

Low

Marital status 3 Association

Singles were significantly more likely to practice in a rural setting as first 

employment64

Being married increased odds (OR 1.47)60

No association

Being married was not associated 63,73

Very low

Race, ethnicity 2 No association

Practicing in small town not associated with race63

Rural setting for first practice not associated with race64

Low

International 

medical graduate

(IMG)

2 Association

IMGs comprise 22% of the clinically active workforce but contribute 19% 

to rural PCP workforce58

 

No association

13% of IMG compared to 18% DOs and 11% MDs are practicing in a 

rural location57

Very low

Exposure 2 Association

Qualitative analysis suggests exposure via recreation facilitates future 

rural practice69

Previous time spent in similar area was an important factor70

Low

Training

Rural rotation in 

training or 

residency

10 Association

Rural residency training showed an association in multivariate analysis 

controlling for rural upbringing72

Rural programs increase odds in addition to being raised in a rural 

community82

Rural residency trainees are 3x more likely to practice in rural areas57

Interviews suggest that exposure via education facilitates rural practice69

Rural clerkship and rural residency training were associated73

Optional summer rural externship increases probability81

Medical school in rural area OR 2.65, rural elective RR 1.53-1.9360

No association

UMC graduates were not more likely to practice in rural areas than 

physicians who graduated elsewhere63

Medical school had discouraged rural practice for 40% of practitioners74

No association with medical school location75

No difference in rural rotation between rural and urban practitioners70

Moderate
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

48

Predictor 

variable

# 

studies

Results and consistency across studies GRADE

Primary care and 

family medicine 

focus

4 Association

Primary care physicians are 2.4x more likely to practice in small towns 

than specialists in a multivariate analysis63

Rural family medicine residency graduates were 3x more likely to practice 

in rural care57

Specialty distribution (primary care, specialty) was significantly different 

between rural and urban groups64

Career in family medicine OR 2.65, family medicine clerkship RR 1.26-

1.4460

Career in primary care OR 1.06, primary care residency RR 1.22-1.7960

No association

No significant association with primary care specialty77

Career in primary care OR 1.0660

Low

Osteopathic

degree

2 Positive association

6% of workforce were DOs but 18% practice in rural care57

4.9% of the workforce but contribute 10.4% to rural primary care58

Low

Financial Aspects

Student loan or 

scholarship

6 Association

2nd major reason was a loan or scholarship obligation68

Medical school loan repayment was correlated with rural practice67

NHSC loan repayment, NHSC scholarship, and debt increased odds60

Significant difference in ratings between urban and rural providers for 

importance of loan repayment70

Loan repayment program had an important influence on the community 

providers chose to practice for 42%62

No association

Student loan debt was not a predictor of practicing in small towns63

The amount of loan debt was a less important factor62

For 71%, educational debt had no influence on initial job79

Very low

Salary 5 Association

Importance of income as a factor in practice location was different 

between rural and urban groups71

58% found the salary an important factor62

Pay was correlated with selecting rural care67

No association

Salary was not a predictor of practicing in small towns in a multivariate 

analysis63

Salary / signing bonus was rated as very important by only 24-28%70

Very low

Setting Characteristics

Scope of practice 4 Association

Broad scope of practice was cited as an important reason for general 

surgeons73

Scope of practice was important to 71% for healthcare providers62

No association

Scope of practice was rated very important only by 30% of emergency 

department physicians70

Full scope of practice was important to 10% female physicians74

Very low

Recreation 

activities

3 Association

Access to amenities/recreation was rated as important for choosing 

practice location70

Recreation activities were rated as important by 58%62

Hunting birds and large game was associated with rural practice73

No association

Currently hunting or fishing, fishing, and hunting small game showed no 

Low
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

49

Predictor 

variable

# 

studies

Results and consistency across studies GRADE

difference73

Lifestyle, small 

town life

2 Association

Lifestyle was rated as very important70

Qualitative interviews identify desire for small town life as important65

Low

Provider Characteristics

Regarding provider characteristics, a large number of studies have investigated a rural 

background (17 studies, high QoE) and have overwhelmingly shown a positive association with 

this factor. The result was also found in 2 multivariate analyses which control for confounders, 

suggesting that the finding is not better explained by other, confounding variables. However, 2

studies pointed out that of the identified providers in rural healthcare, most did not grow up in a 

rural community.65,80 We graded the evidence as high, despite the study design limitations and 

given that the factor cannot be analyzed in a strong research design such as RCTs.

Gender has been addressed in a number of studies but it remains unclear whether female 

healthcare providers are less likely to choose rural healthcare (9 studies, very low QoE). Study 

results were conflicting, may be confounded by changes over time, or may be provider groupspecific. 

Preferences of family, spouse, and children may be an important factor for choosing the 

geographic location (7 studies, low QoE). However, the existing evidence is somewhat 

conflicting, and the factor has not been addressed in multivariate analyses. Consequently, the 

relative importance of this factor, compared to rural upbringing for example, is unclear. There 

were conflicting results for marital status, with studies reporting increased odds, decreased odds, 

and no association (4 studies, very low QoE). 

The association of provider race and ethnicity has been reported in 2 studies and both did not 

find an association. However, this is a common characteristic and a large amount of data may be 

available to answer this question which could not be considered (2 studies, low QoE of no 

association). Two studies investigated whether the country of origin of the healthcare provider 

can predict practice location but the evidence is unclear (very low QoE). 

Two studies reported on the effect of exposure to rural areas, not specific to childhood 

experiences or provider training. Both suggested an association but the studies were not designed 

to quantify the strength of association and size of effect (low QoE). 

Training

Regarding the effect of healthcare provider training on the geographic choice of practice, a large 

number of studies assessed the effect of rural tracks or a rural rotation as part of the healthcare 

provider training or medical residency (11 studies, moderate QoE). The choice to select a school 

with a rural track is likely to be influenced by an affinity to rural healthcare; however, 2 studies 

reported an association in addition to rural upbringing, lending support to the importance of this 

factor.72,82 But it should be noted that not all studies documented an association. We only 

considered evidence here from studies that looked at more than one predictor of rural practice; 

the success of training approaches is documented in KQ5. 
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

50

Other characteristics addressed in more than one study were a primary care and family medicine 

focus which was associated with greater odds of practicing in rural care (5 studies, low QoE). 

The direction of causation for this association is unknown. Finally, 2 studies reported on 

differences between allopathic and osteopathic degrees in medicine, highlighting that osteopathic 

providers represent a smaller proportion of the workforce but contribute proportionally more to 

rural healthcare, but the statistical significance was not reported (2 studies, low QoE).

Financial Aspects

Regarding financial aspects, student loans have been investigated in several studies; however, the 

results indicate that it may depend on who was asked (8 studies, very low QoE). Student loans 

may be important factors for some healthcare providers, but even a large proportion of loan 

recipients indicate that they would have chosen rural care regardless of the program. The effect 

of loan forgiveness programs on rural recruitment and retention are documented in the next 

section (ie, KQ3).

Salary has been addressed as a factor in a number of studies but it remains unclear whether pay is 

a deciding factor for choosing rural healthcare (5 studies, very low QoE). Existing research 

shows conflicting results and the relative importance of the effect is not known.

Setting Characteristics

As mentioned above, there were relatively few studies addressing rural healthcare setting 

characteristics and their influence on the choice of the practice location (4 studies). Most studies 

that addressed these aspects reported on the scope of practice. Results were conflicting across 

studies and it is unclear whether an expected broader scope affects the geographic choice (very 

low QoE). Identified studies addressed different healthcare provider groups and results may vary 

by provider group. Unfortunately, there is not more than one study available that reported on the 

same group. Hence, the effects are not replicated and we could not evaluate any systematic 

differences between provider groups. 

Three studies reported on recreation activities and there was some evidence suggesting that this 

factor affects the choice of practice (low QoE). However, one of the contributing studies reported 

conflicting results within the study, with associations depending on the exact predictor variable.73

Finally, the lifestyle in rural communities has been investigated in 2 studies. Both reported a 

positive association with the choice for practice. The variable has not been assessed in 

multivariate analyses and the strength of association in not known (low QoE).

We did not identify any VA datasets.

KEY QUESTION 3: What interventions have been shown to increase 

rural healthcare provider recruitment? 

This section describes the identified intervention evaluations directed at healthcare providers in 

rural communities. This includes programs for which healthcare providers that have completed 

their initial training period are eligible, rather than interventions exclusively designed for 

students and residents still in the process of completing their training. The evidence table lists

evaluations of programs directed at recruitment as well as studies of programs directed at 

recruitment or provider retention in rural healthcare. 
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

51

The included studies addressed effects of the J-1 visa waiver program and state-wide loan 

forgiveness programs. One of the evaluated programs addressed physicians,83 one physicians and 

nurses,84 and 3 a range of healthcare professionals including physicians, dentists, physician 

assistants, nurse providers, midwives, and mental health specialists.62,85,86 All studies were postonly study designs with no historic or concurrent comparator. Three relied, at least in parts, on 

the responses of surveyed participants; the response rate ranged from 55 to 80%.

The J-1 Visa Waiver Program has been in place since 1994 and was expanded in 2002; it allows

states visa waiver slots on an annual basis. The evaluation assessed the effects of the program on 

the state of Washington where selected physicians are obligated to work for an approved 

employer for 3 years for primary care physicians and 5 years for specialists.83 The loan 

forgiveness programs were established to increase the number of healthcare providers in rural85

or rural or underserved areas.62,84 One study reported on all state-wide recruiting interventions 

for West Virginia.86

All evaluations assessed state-wide effects. Studies reported on the uptake of the program, the 

percentage of physicians fulfilling their obligation (eg, 3 years of practice at a designated site), 

and other results relevant to recruitment and retention.

The evidence table summarizes the studies and evaluated programs.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

52

Table 5. Evidence Table – KQ3 (Provider Recruitment Interventions) 

ID Region

Providers

Participants Intervention Study design

Definition of 

rural

Recruitment 

results

Retention 

results

Authors’ 

conclusions

Kahn, 

201083 Washington state

J-1 visa 

waiver 

physicians

77 of 141 

returned survey 

(response rate 

55%); 155 

physicians 

began J-1 

waivers in 

Washington 

state between 

1995 and 2003; 

addresses of 

141 were 

located

Under the 1994 Conrad J-1 Visa Waiver 

Program (updated and expanded in 

2002), each participating state is 

allocated 30 visa waiver slots annually to 

administer through its state health 

department. After states recommend 

physicians for visa waivers, these 

doctors are obligated to work for an 

approved J-1 waiver employer for the 

duration of their commitment period, 

which in Washington state is 3 years for 

primary care physicians and 5 years for 

specialists

Post-only study

Rural: ZIP code 

Version 1.11 of 

RUCA Codes 

(based on 1998 

Census 

commuting data 

and 1988 ZIP 

codes) to classify 

addresses and 

Version 2.0 

(based on 2000 

Census 

commuting data 

and 2004 ZIP 

codes) to classify 

the current work 

addresses of the 

127 physicians 

we located; 10-

point scale

155 began 

program, 68% 

practiced primary 

care; 37% 

completed their 

obligations in rural 

areas. Of these, 

47% practiced in 

large rural 

cities/towns, 32% 

in small rural 

towns, and 21% in 

isolated small rural 

towns

Of 141 tracked 

physicians, 23% 

are still working 

for their 

assigned 

employer; of 

respondents 

who had 

completed 

commitments 

84% remained 

longer than 

required 

(median 23, 

mean 25 

months, range 

0-120); the 

average RUCA 

rating for original 

J-1 waiver 

locations was 

3.02 (higher 

numbers 

indicate more 

rurality), 

whereas the 

average current 

employment 

RUCA category 

was 1.51; 

physicians 

appear to have 

moved toward 

more urban 

areas (p<.001)

In Washington 

state, the Conrad 

Program has 

increased the 

number of 

physicians in 

underserved 

areas who 

frequently stay 

beyond their 

obligations. The 

significant 

movement away 

from rural areas 

for postobligation 

employment, 

however, 

highlights the 

long-term need 

to continue state 

efforts to recruit 

physicians to 

these areas.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

53

No 

Author, 

200785

Minnesota

Physicians, 

NPs, nurse 

midwives, 

nurse 

anesthetists, 

advanced 

clinical nurse 

specialists, 

physician 

assistants, 

dentists, 

pharmacists, 

allied health 

and nursing 

faculty, 

nurses

564 participants 

from 1990 to 

2007, 405 

surveyed, 73% 

response rate

In 1990, the Minnesota State Legislature 

created and funded a program to recruit 

physicians to practice in rural Minnesota. 

Since then the program has expanded to 

assist 564 practitioners in rural 

Minnesota or other high-need locations. 

After almost 17 years of operation and 

growing from an annual state 

appropriation of $320,000 to $1.295 

million in 2007, the Minnesota Loan 

Forgiveness Programs have served over 

300 healthcare facilities and educational 

institutions throughout the state. In the 

past 7 years, Minnesota has invested 

$7.789 million in the Loan Forgiveness 

Programs

Post-only study

Rural: 

Designated rural 

sites for areas 

outside 

metropolitan 

counties 

(Hennepin, 

Anoka, Ramsey, 

Dakota, 

Washington, 

Carver, and 

Scott) and 

excluding 

Rochester, 

Moorhead, St. 

Cloud, Duluth, 

and Mankato

>240 responders 

have chosen rural 

or high-need 

settings in the past 

8 years as a result 

of the program; 

44% of 

respondents said it 

was important to 

very important in 

influencing their 

decision to choose 

primary medical 

practice (70% in 

1999). Distinct 

decline in physician 

applications in 

2007 (1/3rd fewer 

applications than 

2006, > 50% 

reduction since 

2003

86% of 

respondents 

continued 

medical practice 

at their 

sponsoring 

facility 

(placement site) 

after completing 

their service 

obligation

The Loan 

Forgiveness 

Programs are 

successfully 

meeting their 

program goals 

and increasing 

the number of 

healthcare

providers and 

educators in 

rural Minnesota 

and specialty 

locations.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

54

Renner, 

201062

Colorado

Healthcare

professionals 

(39% 

physicians)

97 respondents 

(response rate 

80%)

Colorado health professional loan 

repayment program (CHPLRP): began in 

1992, rewards physicians, dentists, PAs, 

NPs, certified midwifes and mental 

health specialists with up to $35,000 per 

year of educational loan repayment for 

working in any rural or urban HPSA in 

Colorado with minimum 2-year 

commitment. Colorado Rural Outreach 

Program (CROP): created in 1998, 

rewards all types of healthcare 

professionals (including but not limited to 

physicians, NPs, PAs, nurses, mental 

health services, dental health services, 

and allied health professionals up to 

$10,000 per year for educational loan for 

up to 3 years. Provider must work in a 

rural community in Colorado to be 

eligible and program requires 1 year 

service commitment. Dental Loan 

repayment program: began in 2002, 

provider must work with underserved 

populations in either rural or urban areas 

to be eligible

Post-only study

Rural: RUCA ZIP

code designation 

above and 

including 4.0

57% of 

respondents 

worked in rural 

communities. Of 

rural participants, 

74% were working 

in or intended to 

work in an eligible 

community when 

they were made 

aware of the loan 

repayment 

program. Of those 

planning to work in 

a rural community 

regardless of loan 

repayment option, 

42% reported the 

program had an 

important influence 

on the specific 

community they 

chose

Of the 

participants 

already working

in rural 

communities, 

38% reported 

loan repayment 

as being 

important to 

retention. Of 

rural 

participants, 

22% cited the 

desire for a 

higher income 

as an important 

reason to leave 

their 

communities 

regardless of 

loan repayment

Loan repayment 

programs enroll 

providers who 

would have 

worked in a rural 

area regardless 

of loan 

repayment 

opportunities, but 

are likely to play 

a role in 

provider's choice 

of specific rural 

community for 

practice; they 

also appear to 

influence rural 

provider 

retention, though 

financial 

concerns are 

generally less 

influential for 

non-retained 

rural providers 

than are family 

preferences and 

professional 

dissatisfaction.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

55

Wheeler, 

200984

Oklahoma

Graduates of 

Oklahoma 

State 

University 

Osteopathic 

College

333 The Physician Manpower Training 

Commission (PMTC) was established by 

the Oklahoma state legislature in 1975 

with the primary mission of increasing 

the number of physicians and nurses in 

rural and underserved areas of the state

Post-only study

Rural: N/A

PMTC has 

provided financial 

assistance to 333 

graduates, 30 

graduates opted to 

repay the loan and 

terminated their 

contract, resulting 

in 303 program 

completers since 

1978. Of all 

program 

completers, 83% 

continue to practice 

medicine in 

Oklahoma; 28% of 

these are 

practicing in rural 

areas of the state. 

62% of alumni who 

participated in 

PMTC’s funding 

programs are 

practicing in rural 

areas

51 program 

completers 

completed their 

contractual 

practice 

obligation but 

are either no 

longer in active 

practice or have 

left the state. 

For those whose 

obligated 

practice location 

was in a rural 

area, 84% are 

currently 

practicing in a 

rural location

Clearly, PMTC 

programs have 

been successful 

in the 

recruitment and 

retention of 

graduates to 

rural practice in 

Oklahoma.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

56

Wheeler, 

201386

West Virginia

Primary care 

physicians, 

subspecialists, 

specialists, 

physician 

assistants, 

and family 

nurse 

practitioners

not defined The Division of Rural Health and 

Recruitment administers several 

financial incentive programs: the 

National Health Service Corps loan 

repayment program (primary care 

physicians, physician assistants, nurse 

practitioners, certified nurse midwives, 

general and pediatric dentists, dental 

hygienists, behavioral healthcare

providers, nursing faculty, and practicing 

nurses; 2-year service obligation); State 

Loan Repayment (10 providers per year; 

physicians, nurse practitioners, dentists, 

physician assistants, pharmacists; 

obligation to work 2 years in rural West 

Virginia; recipients eligible to reapply for 

additional 2 years); Recruitment & 

Retention Community Project (for 

facilities located in a federally 

designated MUA/MUP; one-year 

commitment, with the employer matching 

the award, for a total of $20,000.00); the 

J-1 Visa Waiver Program (communities 

unable to recruit a US citizen to provide 

healthcare are allowed to recruit a 

foreign physician); Recruitable 

Community Program (promoting 

volunteerism).

Post-only study

Rural: West 

Virginia is the 

second most 

rural state in the 

nation, and the 

only state that is 

located entirely 

within Appalachia

Through these 

various programs, 

the Division of 

Rural Health and 

Recruitment has 

placed a total of 

154 NHSC 

providers; 52 SLRP 

providers; 47 

RRCP providers; 

and 94 J-Visa 

physicians since 

2008

Currently, 80% 

of these 

providers have 

remained at 

their initial 

placement site 

upon completion 

of their 

obligation

The Division of 

Rural Health and 

Recruitment 

works diligently 

to alleviate some 

of those 

shortages and to 

strengthen the 

healthcare safety 

net in West 

Virginia, and 

utilizes the most 

up-to-date and 

relevant provider 

recruitment and 

retention 

strategies 

available.

RUCC = Rural-Urban Continuum Codes, RUDT = Rural-Urban Density Typology, RR = relative risk, OR = odds ratio 
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

57

Summary of Findings and Quality of Evidence for Key Question 3

In addition to the small number of identified evaluations we did not identify studies reporting on 

the same outcome within the recruitment or the retention outcome category. Hence, no summary 

of findings table was completed and results are narratively described. 

One of the identified evaluations was a J-1 visa waiver program evaluation on state level. Of all 

J-1 visa waiver placements in the study, about one-third were in rural areas. The J-1 visa 

evaluation reported that 53% of physicians did not complete their obligations. But of the 

respondents who had completed commitments, 84% remained a median of 23 months longer 

than required.83

The Minnesota loan forgiveness program highlighted that 86% of surveyed physicians continued 

medical practice at their sponsoring healthcare facility.85 The Colorado program reported that 

57% of respondents worked in rural communities. The evaluation also highlighted that 74% of J1 visa waiver recipients were already working in or intended to work in an eligible community 

when they were made aware of the program. Furthermore, 38% of those working in rural 

communities reported loan repayment as being important to retention.62 The Oklahoma 

assistance program reported that of those recipients whose obligated practice location was rural, 

84% were practicing in rural areas and 28% of those who fully completed their service obligation 

were practicing in rural areas.

84 An evaluation of Virginia Division of Rural Health and 

Recruitment programs stated that 80% of placed providers have remained at their initial 

placement site upon completion of their obligation.86

We did not identify any VA datasets.

KEY QUESTION 4: What interventions have been shown to increase 

rural healthcare provider retention? 

We did not identify any evaluation of a provider intervention that was exclusively directed at 

retaining fully-trained providers practicing in rural healthcare. All other intervention studies 

aimed at providers are included in the evidence table for KQ3 (interventions to improve 

recruitment, alone or in combination with improving retention). Studies examining interventions 

for providers in training are discussed in the KQ5 section.

Summary of Findings and Quality of Evidence for Key Question 4

Of the small number of identified provider intervention studies (see evidence table 5), some 

reported on the outcome retention. The J-1 visa evaluation and loan forgiveness programs

reported that 80% or more of respondents remained longer than their required obligations, 

although only one study reported the duration.83,85,86

KEY QUESTION 5: What is the efficacy of current rural-specific 

resident and healthcare profession student training and education 

efforts? 

This section documents the identified evaluations aimed at resident and healthcare profession

student training. All identified studies are documented in the evidence table; the summary of 

findings section synthesizes results across studies. 
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

58

Thirteen studies assessed the training of medical students, while 9 examined the training of 

residents, and one a combination. The most common resident specialties were family medicine 

and general surgery, with one study each examining residents in emergency medicine,

87

psychiatry,

88 and across all specialties.89

The majority of the included studies evaluated a single training program at one institution. 

Rabinowitz et al reported on multiple cohorts at the Jefferson Medical College, Pennsylvania.90,91

One study reported on different programs at the University of Minnesota,

82 while another study 

reported the effect of different components of a program.92 Total numbers of participants 

(including controls) ranged from 793 to 3,365.

82,94,95 The studies utilized internal records or 

internally distributed surveys for determining practice locations after graduation. Four studies 

used external records such as the AMA Masterfile or state licensure records.

The majority of training programs evaluated consisted of embedding a student or trainee in a 

rural community for all or part of their medical training. Experiences ranged in duration from 4 

weeks93 to 5 years.

96 Studies of individual training programs reported capacities ranging from 

229,97 to 6098 trainees per year.

Studies evaluating multiple training programs reported on sample sizes ranging from 12399 to 

322,13160 participants. Few of these multiple-institution studies included details for the year of 

graduation or completion of training. The majority of these studies discerned practice locations 

from publicly available data such as the AMA Masterfile, while one study used data reported by 

the individual training programs examined 98 and one used a survey distributed by the authors.

87

Studies evaluated rural tracks across medical schools,

98,99 assessed resident training in Critical 

Access Hospitals or Rural Health Clinics, compared 3 different rural programs that combine 

preferential admissions of students likely to practice in rural areas with a rural curriculum,

91

assessed the effect of the expansion of medical schools on rural healthcare or Title VII 

funding,

100 evaluated rural rotations for emergency medicine residents,87 and assessed which US 

medical schools provide physicians for the Appalachian region of the US.

101

Most studies reported on the outcome of recruitment of graduated trainees to a rural area, and 5

studies reported on retention in these areas. One study reported only on matriculation of medical 

students from rural backgrounds.

100 In studies that formally defined “rural,” the designation 

referred to federal taxonomies such as RUCA codes or RUCC. In 7 studies, the designation of an 

area as rural was determined by the training program or was not specifically discussed. Only 6

studies included a comparison or control group against which to compare outcomes, with the rest 

of the studies reporting only on the group of trainees undergoing the rural-specific training.

The evidence table summarizes all identified studies contributing to this key question. The table 

includes information about the training programs, outcomes, and the authors’ conclusion from 

the study. The table lists single institution studies first, followed by evaluations across multiple 

training institutions.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

59

Table 6. Evidence Table – KQ5 (Provider Training Evaluations) 

ID Location

Capacity

Students

N 

Data 

source

Program description Study design

Definition of 

rural

Recruitment results Retention 

results

Other 

results

Authors’ 

conclusion

Single-institution Program Evaluations

Antonenko, 

200996

North Dakota

University of 

North Dakota 

(UND) School 

of Medicine 

and Health 

Sciences

Capacity: 

1986-2008 = 

44 residents

General 

surgery 

residents

N = 44

Internal 

records

General surgery 

residency started in 

1982, first graduates in 

1986. ACGME-required 

rotations include 

anesthesiology, 

pathology, plastics, 

orthopedics, ears, nose 

and throat, and urology, 

pediatric surgery and 

trauma surgery as 4thyear residents; 1 month 

in each of first 2 years at 

Belcourt Indian Health 

Services Hospital on on 

the Turtle Mountain 

Indian reservation in 

central North Dakota

Post-only

Rural: N/A

47% of graduates 

continue to practice in 

rural areas; of those 

pursuing fellowship 

training, 16% practice 

in rural areas (58% 

general surgeons who 

did not take fellowship 

training practice in rural 

sites)

N/A N/A The program 

continues to provide 

the best training 

consistent with the 

practice 

requirements of its 

graduates – 

particularly those 

who chose to 

practice in smaller 

communities and 

rural sites.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

60

Bonham, 

201488

University of 

New Mexico

(UNM)

New Mexico

Capacity: 60 

residents 

from 1991-

2010

Psychiatry 

residents

N = 60, 37 

responded 

to survey 

(62 % 

response 

rate)

Internal 

database

The UNM Rural 

Psychiatry Residency 

Program (UNM RPRP) 

was developed in 1991 

to improve residency 

education about rural 

mental health and to 

increase the number of 

psychiatrists in rural and 

underserved 

communities. All trainees 

participate in an 8-week 

seminar about rural 

mental health, cultural 

competence, and public 

mental health systems. 

1/3 of residents elect to 

participate in community 

site visits in PGY-2 and 

PGY-3 years. During the 

PGY-4 year, residents 

work in community 

settings for 2 days a 

week throughout the 

year. Residents return 

each week to the 

university for didactics 

and to see their 

psychotherapy patients 

and supervisors. 

Participants receive 

sponsorship to attend 

the meeting of the Rural 

and Community 

Psychiatry Network of 

New Mexico. This 

network is designed to 

support efficacy and 

retention of providers in 

rural communities by 

reducing the experience 

of provider isolation. At 

the meeting, residents 

meet psychiatrists who 

Post-only

Rural: 

Metropolitan 

(continuously 

built-up areas of 

50,000 or more), 

large rural town 

(10,000–

49,999), small 

rural town 

(2,500–9,999), 

or isolated small 

rural area 

(outside of urban 

areas or urban 

clusters).

37% of program 

graduates were 

practicing psychiatry in 

rural communities 

compared to 10 % of 

the graduates in the 

traditional program. Of 

all graduates who were 

currently practicing in 

rural communities, 75% 

were practicing in large 

rural towns, 20% in 

small rural towns, 6% in 

isolated small rural 

areas

N/A N/A Purposeful and 

well-coordinated 

educational 

opportunities 

situated in rural 

community health 

clinics can address 

some of the barriers 

for recruiting and 

retaining 

psychiatrists in rural 

areas. Practical 

skill-building at the 

individual, agency, 

and system level is 

integral in training 

psychiatrists for 

work in these 

communities. In 

particular, the use 

of telepsychiatry 

emerged as an 

important practical 

application for the 

provision of rural 

mental healthcare. 
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

61

practice in rural 

communities and begin 

to build the personal and 

professional networks on 

which they will rely as 

professionals.

Crane, 

2014102

North 

Carolina

Hendersonvill

e Family 

Medicine 

Residency 

Program

Capacity: 37 

graduates 

from 1999-

2010

Family 

medicine 

residents

N = 37

Internal 

database

Rural-track training 

program to address the 

growing shortage of 

primary care physicians 

in rural North Carolina. 

Designed to develop 

successful rural health 

physicians

Post-only

Rural: N/A

65% graduates practice 

in rural communities

N/A 60% work 

in a full or 

partial 

health 

professiona

l shortage 

area

Rural-track training 

programs have 

generally been 

more successful 

than traditional 

family medicine 

residencies in 

placing graduates in 

rural or underserved 

practice settings. 

Our program 

contributes to

improving access to 

care and new 

models of care and

warrants further 

investigation in how 

training can be 

scaled to address 

North Carolina’s 

continuing 

shortages of rural 

primary care 

physicians.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

62

Crump, 

2013103

University of 

Louisville, 

Kentucky

Trover 

campus in 

Madisonville 

(ULTC) 

versus main 

campus in 

Louisville

Capacity: 6-

10 medical 

students per 

year

Medical 

students

N = 1391

AMA 

Masterfile 

linked to 

medical 

school 

graduation 

list

After completing their 

first 2 years of basic 

sciences, students move 

to Madisonville and 

complete all required 

clinical rotations and 

most electives there or in 

the smaller surrounding 

communities. The small 

class size provides a 

personal connection 

between faculty and 

student. Students are 

based within a rural 

integrated health system 

with a 400-bed hospital 

staffed by 80 primary 

and secondary care 

physicians. The system 

includes 10 satellite 

clinics within a 30-min 

drive that are in towns of 

4,000-8,000 that host 

portions of clinical

rotations. Trover 

students participate in 

the same lectures as the 

Louisville students via

live video connection

and all curriculum 

elements, teaching 

materials, and evaluation 

systems are identical. 

Clinical rotations on the 

ULTC provide the 

opportunity for 1-on-1 

learning with an 

experienced clinician 

preceptor. On ~1/3 of 

rotations, a family 

medicine resident is on 

the teaching service as 

well. All 3rd-year 

students regardless of 

Comparative

study

Rural: RUCC,

nonmetropolitan 

RUCC codes as 

a surrogate to 

identify 

rural/small-town 

practice

55% of ULTC 

graduates chose 

nonmetropolitan 

practices compared 

with to 9% of Louisville 

graduates (p<.001)

N/A N/A These data support 

the value of a small 

regional rural 

clinical campus in 

optimizing the 

affinity model to 

place rural students 

into rural practice. 

The ULTC students 

showed equivalent 

adjusted test scores 

and slightly 

narrowed the gap in 

unadjusted USMLE 

scores compared 

with the main 

campus students.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

63

rotation meet for smallgroup, problem-based 

learning sessions using 

the iterative process 

2x/month. A longitudinal 

teaching skills program 

assists the communitybased faculty in guiding 

the students through 

their required clerkships

Deveney, 

200929,97

Oregon 

Health 

Sciences 

University

Grants Pass, 

Oregon

Capacity: 2 

4th-year 

residents per 

year

General 

surgery 

residents

N = 70

Internal 

results

Grants Pass has a 

population of 25,000 with 

a 125-bed hospital, 

residents are placed for 

a year-long 4th-year 

rotation

Post-only

Rural: Site with 

population 

<50,000

Graduates are more 

likely to practice in a 

site of population 

<50,000 (p = .02) than 

graduates before the 

program

N/A N/A The need for more 

general surgeons 

who are prepared 

and willing to serve 

rural communities is 

well-recognized. 

Based on our 

experience over the 

past 7 years, we

believe that 

residents will benefit 

from a training 

program that 

provides extensive 

exposure to 

procedures unique 

to a rural practice.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

64

Glasser, 

200868,104,105

Rockford, 

Illinois

Rockford, 

Illinois

Capacity: 159 

graduates 

from 1993-

2007, 

typically 

admits 15-20 

students

Medical 

students

N = 216

Internal 

database

The curriculum is based 

on observations of other 

rural medical education 

programs. A key concept 

is the integration of 

clinical medicine and 

population health. PY1

topics: population-based 

approaches to rural 

healthcare, agricultural 

hazards and farm safety, 

health resources in rural 

communities, rural 

mental health issues, 

and community-based 

rural health research. 

PY2: chronic disease 

management, rural 

clinical cases discussion, 

complementary and 

alternative medicine, 

rural health issues, and 

community-oriented 

primary care (COPC). 

PY3: introduction to 

Community Project 

Resource Book and 

Community Structure 

Project, perspectives on 

practice location and 

practice arrangement 

decisions, ethical 

dilemmas, coding and 

optimized 

reimbursement, the IRB 

process, and COPC 

work sessions. In years 

1 and 2, there are also 

field trips and special 

events (eg, “Southern 

Exposure;” the “No Harm 

on the Farm” tour; and 

the Illinois Rural Health 

Association meeting) to 

Post-only

Rural: RUCA

67% of graduates 

practice primary care in 

towns of 20,000 people 

or fewer or practice in 

communities classified 

as RUCA codes 4 and 

higher. 2010 

publication reports 70% 

practice in rural area. 

2013 publication 

reports 61% practice in 

towns of 20,000 people 

or less, with 23% 

practicing in towns of 

5,000 or less, 56% 

practice in RUCA code 

4 or higher areas

82% have 

stayed at their 

original 

practice site, 

8 graduates 

with 3 or more 

years in 

primary care 

practice have 

relocated to 

rural 

communities

N/A Our program can 

serve as a model at 

many levels, 

including 

recruitment, 

collaboration, 

curriculum, and 

retention. Future 

challenges include 

recruiting students 

from the growing 

number of rural 

minority 

populations, 

expanding the 

number of program 

slots, and 

integrating the 

program with other 

health professions 

to address the 

needs of rural 

populations.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

65

provide students with the 

opportunity to observe 

and participate in rural 

health communities and 

organizations. The 

course grades are part 

of the final medical 

school transcripts. The 

3rd-year curriculum 

serves as preparation for 

the 4th-year, 16-week 

preceptorship in a rural 

community. Students 

select 1/25 rural 

communities. For each 

site, the local hospital 

has agreed to provide 

room and board for the 

student as well as a 

primary care physician to 

serve as preceptor.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

66

Kallail, 

2010106

Kansas

N/A

Capacity: 14 

students per 

year

Medical 

students

N = 104

Internal 

database

Applicants must be 

Kansas residents with 

significant experience 

living in a rural Kansas 

community. They must 

demonstrate intellectual 

promise, the intention to 

practice medicine in rural 

areas of Kansas, and a 

commitment to service 

by exhibiting the 

dedication and 

compassion necessary 

to be a competent and 

caring physician. The 

application and selection 

process is similar to 

regular medical school 

admissions. The primary 

difference is that 

Scholars applicants 

interview after their 

undergraduate 

sophomore year and 

enter medical school 2 

years later after 

successful completion of 

program requirements. 

Each program 

participant must 

demonstrate 

achievements in 

academic programs and 

a significant, informed 

interest in rural 

healthcare. Scholars are 

assigned a rural primary 

care physician mentor 

beginning in the junior 

year of college. The 

mentors are usually from 

Scholars’ hometowns or 

nearby. Scholars

shadow for a minimum 

Post-only

Rural: Based on 

population 

density (either 

frontier, rural, or 

dense rural)

63% of graduates 

practice in rural, nonmetropolitan 

communities (11/12 in 

Kansas); of those in 

rural communities, 11 

are family physicians, 

and 1 is a general 

surgeon

N/A 84% 

practice in 

either a 

rural or 

urban 

underserved 

community

As one component 

of the effort to 

provide physicians 

for Kansas, our

program showed 

beneficial outcomes 

for attracting 

applicants who want 

to practice in rural 

or other medically 

underserved 

communities and 

who maintain that 

interest over the 

long process of 

medical education. 

The designation of 

medical school 

positions for 14 

Scholars in Rural 

Health enhances 

the likelihood of 

success for 

maintaining a 

pipeline of 

physicians for rural 

Kansas.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

67

of 200 hours over the 2 

years. They write 4 case 

reports and write a paper 

on a population-based, 

community health issue.

Mason, 

201263 Mississippi

N/A

Capacity: N/A

1990-1999 

UMC 

graduates 

practicing in 

Mississippi

N = 927

Mississippi 

State Board 

of Medical 

Licensure 

University of Mississippi 

Medical Center (UMC) is 

the only medical school 

in the state and 

Mississippi is one of the 

most rural states in the 

nation; about half of new 

practicing physicians 

come from UMC’s 

graduating classes

Post-only

Rural: N/A

UMC graduates are not 

more likely to practice 

in rural, small towns, or 

geographically isolated 

areas in Mississippi 

than physicians who 

graduated elsewhere

N/A Primary 

care 

physicians 

are 2.4 

times 

(P<.001) 

more likely 

to practice 

in small 

town areas 

than 

specialists, 

all else 

being equal

Health educators 

and policy makers 

should consider 

broadening UMC’s 

enrollment policies 

and greater 

emphasis should be 

placed on recruiting 

physicians.

Nash, 

200893 University of Texas (UT) 

Medical 

Branch at 

Galveston

Weimer, 

Texas (138 

miles from 

university)

Capacity: 2-3 

residents

Family 

medicine 

residents

N = 7 

Internal 

records

UT Galveston 

implemented a rural 

training track (RTT) in 

1998. Residents spend 4 

weeks living and working 

in a rural Texas 

community supervised 

by clinical family 

physician faculty. In 

2000, the RTT was 

expanded to a 

longitudinal rural 

curriculum (spending 28 

weeks in rural 

communities)

Post-only

Rural: N/A

86% of graduates 

entered practice in rural 

areas

N/A N/A Although the 

program is small, 

the RTT has shown 

proportionally 

impressive success 

with graduates 

choosing rural 

practice in Texas.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

68

Quinn, 

201192

University of 

Missouri 

School of 

Medicine

Rural 

communities 

in Missouri

Capacity: 

Preadmission 

program: up 

to 15 

students/year

. Summer 

Community 

Program: 20 

to 30 

students/year

. Rural Track 

Clerkship: 20 

to 35 

students/year

. Rural Track 

Elective: 5 to 

7 

students/year

. 

Medical 

students in 

their 2nd-4th

years

N = 344

MUSOM 

Alumni 

Association 

database, 

the National 

Residency 

Match 

Program, 

and the 

American 

Board of 

Medical 

Specialties, 

as well as 

information 

gleaned 

from Internet 

searches 

and from 

personal 

corresponde

nce with 

program 

participants

The program has 4 

facets: 

1. Preadmissions 

program for rural 

students (Rural 

Scholars). A cooperative 

effort between 6 

designated institutions 

and the MUSOM, 

designed for students 

from rural backgrounds. 

Rural Scholars are 

accepted into the 

MUSOM, conditional on 

academic achievements, 

professional conduct, 

and participation in 

required activities. 

Program activities 

include biannual 

retreats, mentoring 

relationship with a rural 

physician, and 

community service. 

Rural Scholars also 

participate in the 

Summer Continuity 

Program, the Rural 

Track Clerkship, and the 

Rural Track Elective.

2. Summer Community 

Program for rising 2ndyear students. Students 

work closely with 

physicians practicing in 

smaller communities for 

4-8 weeks.

3. Rural Track Clerkship 

(RTC) Program for 3rdyear students. The RTC 

Program offers all 3rdyear medical students 

the choice of completing 

1-3 of their 7 required 

Post-only

Rural: RUDT, 

population 

density 

thresholds of the 

US Census 

Bureau’s 

classification 

system, Office of 

Management 

and Budget’s 

urban population 

nucleus 

requirements, 

and other criteria 

to classify 

counties as 

“rural,” “mixed 

rural” 

Preadmission Rural 

Scholars program: 50% 

report first practice 

location to be rural or 

mixed rural. 

Only in the Rural Track 

Clerkship: 64% first 

practice location rural 

or mixed rural. 

Rural Track Clerkship + 

Summer Continuity 

Program or Rural Track 

Elective: 56% first 

practice location rural 

or mixed rural;

Any aspect of the Rural 

Track Pipeline 

Program: 57% first 

practice location rural 

or mixed rural

N/A N/A The longitudinal 

program

successfully recruits 

students for rural 

and primary care 

practice to address 

the healthcare

needs of Missouri.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

69

core clinical clerkships in 

rural training sites. 

Students live in the 

community while 

completing their clinical 

rotations.

4. Rural Track Elective 

Program for 4th-year 

students. Medical 

students may choose to 

complete a variety of 

required monthlong 

primary care or specialty 

electives at 1 of 10 

community-based rural 

clerkship sites. 

Beginning with the 

graduating class of 

2013, Rural Scholars are 

required to take at least 

1 elective.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

70

Rabinowitz, 

201190,107

Jefferson 

Medical 

College - 

Pennsylvania

3rd-year 

family 

medicine 

clerkship in 

rural location, 

and senior 

outpatient 

subinternship 

in family 

medicine

Capacity: 104 

PASP 

graduates 

between 

1992 and 

2002

Medical 

students in 

Jefferson 

Physician 

Shortage 

Area 

Program 

(PSAP) 

versus those 

that did not 

participate in 

PSAP

N = 2394

Jefferson 

Longitudinal 

Study (JLS) 

of Medical 

Education 

database 

from 1992-

2002; 2007 

alumni 

database

PSAP initiated 1974 at 

Jefferson Medical 

College. Recruits and 

selectively admits 

medical school 

applicants who have 

grown up in a rural area 

and who are committed 

to practicing family 

medicine in a rural area, 

especially Pennsylvania. 

During medical school 

PASP students have a 

family physician faculty 

mentor, take required 

3rd year family medicine 

clerkships in a rural 

location, take their senior 

outpatient subinternship 

in family medicine 

(usually rural location) 

and receive a small 

amount of additional 

financial aid (usually in 

form of repayable loans).

On completion of 

medical school, PSAP 

graduates are expected 

to complete family 

medicine residency and 

practice family medicine 

in a rural location of their 

choosing, although no 

formal mechanism exists 

to ensure compliance.

Comparative

study

Rural: RuralUrban Density 

Typology

(RUDT), % rural 

or urban, 

population in 

urbanized areas, 

population 

density 

Alternate 

definition: rural 

counties are 

those where less 

than 50% of the 

population live in 

an urbanized 

area (ie, a 

densely settled 

territory with 

50,000 or more 

people)

43% of PSAP 

graduates were 

currently practicing in a 

rural area (compared to 

15.8% of non-PASP 

graduates, RR 2.7, CI: 

2.1-3.5, p < .001). 

PSAP graduates were 

almost 10x more likely 

to combine family 

medicine with practice 

in a rural area (32.0% 

vs non-PSAP peers

3.2% (RR 9.9, CI 6.8-

14.4; p < .001); > 1% of 

PSAP graduates were 

practicing in 18 of the 

state’s 48 rural 

counties, whereas no 

rural county had even 

0.3% of non-PSAP 

graduates

N/A When 

applying 

the 

secondary 

definition of 

rural, the 

pattern of 

results 

were 

similar; but 

the 

absolute % 

were lower.

Women 

PSAP 

graduates 

were >2x

as likely as 

non-PSAP 

women 

graduates 

to practice 

in rural 

areas (RR, 

2.6; CI, 

1.6–4.2), 

similar to 

the PSAP 

impact on 

men

Despite major 

changes in 

healthcare in recent 

decades, 

Jefferson’s PSAP

continues to 

represent a 

successful model 

for substantially 

increasing the 

supply and 

distribution of rural 

family physicians. 

Especially with the 

forthcoming 

expansion in health 

insurance, access 

to care for rural 

residents will 

require an 

increased supply of 

providers. 

graduates.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

71

Rabinowitz, 

2013108,109

Jefferson 

Medical 

College, 

Philadelphia, 

PA

N/A

Capacity: 

1937 

graduates 

from 1978 to 

1986

Jefferson 

medical 

school 

graduates

N = 1937 

graduates 

from 1978 to 

1986

Jefferson 

Longitudinal 

Study

The Physician Shortage 

Area Program (PSAP), 

which began in 1974, 

recruits and selects 

medical school 

applicants that have 

grown up or lived in a 

rural area or small town 

for a substantial portion 

of their life after college 

and who were committed 

to practicing family 

medicine in a similar 

area. During medical 

school, PSAP students 

received faculty 

mentorship and career 

support and they 

completed their required 

3rd-year, 6-week family 

medicine clerkship in a 

small town. During their 

4th year, most PSAP 

students took a 

preceptorship in a rural 

location. Upon 

graduation, PSAP 

students were expected 

to take a family medicine 

residency and practice 

family medicine in a 

small town or rural area, 

although there is no 

mechanism to ensure 

compliance.

Post-only

Rural: County 

not designated 

as a standard 

metropolitan 

statistical area

N/A Of 37 PSAP 

graduates 

who originally 

practiced rural 

family 

medicine, 

70% were still 

practicing 

family 

medicine in 

the same 

rural area 

(compared 

with 46% of 

non-PSAP 

graduates) (p 

= .02)

N/A This study provides 

additional support 

for the substantial 

impact of medical 

school rural 

programs, 

suggesting that 

graduates of rural 

programs are not 

only likely to enter 

rural family 

medicine but to 

remain in rural 

practice for 

decades.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

72

Ross, 

2013110

Klamath 

Falls, Oregon

Cascades 

East Family 

Medicine 

Residency 

Program; Sky 

Lakes 

Medical 

Center

Capacity: 76 

residents 

completed 

training from 

1994 - 2009.

Family 

medicine 

residents

N = 76 

graduates 

sent survey. 

Response 

received 

from 62 (> 

82% 

response 

rate) 

Survey

3-year family residency 

training program in 

Klamath Falls 

(population ~42,000) at 

96-bed, nonprofit 

community hospital (only 

Oregon residency 

program outside 

metropolitan Portland 

area). AAFP notes this 

training site to be the 

most rural and remote 

training site in the nation 

and it is supported by 

the smallest community 

hospital. The program’s 

main goal is to produce 

full scope of practice 

family physicians to 

enter rural practice

Post-only

Rural: Rural 

practice = 

populations 

<25,000, >25 

miles from major 

centers

60% of graduates 

initially enter practice in 

communities of 

<25,000 population and 

45% practice in towns 

of <10,000; 63% of 

graduates first 

practiced in HPSAs

37% 

remained in 

very rural 

locales of 

<10,000 

population; 

50% of all 

graduates 

since 1994 

remain in 

rural settings; 

52% of 

graduates 

that first 

practiced in 

HPSAs 

remain in 

those areas, 

with 45% 

delivering 

care at 

FQHCs or 

RHCs

Of 25 

graduates 

who 

practiced in 

>1 location 

since 

graduation, 

18% moved 

to larger 

centers, 

11% moved 

to towns of 

the same 

size, and 

the 

remaining 

11% moved 

to smaller 

population 

centers

Family medicine 

residency programs 

of 3 years duration, 

located in small 

community 

hospitals, would 

seem to be ideal 

settings for training 

future rural 

physicians. The 

outcomes 

substantiate this 

conclusion and 

show that rural 

residency graduates 

tend to serve small 

communities, often 

the one where they 

first enter practice, 

for an extended 

period.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

73

Whitacre, 

201081

Oklahoma

Oklahoma 

State 

University

Capacity: N/A

Medical 

students

N = 190

Internal data

Osteopathic school 

program designed to 

promote rural location:

Summer rural externship 

(optional) – observe 

local physician 30-35 

hours per week during 

summer following 1st

year of medical school.

Early clinical experience 

(optional) – 5-day tour of 

different physician 

offices across the state. 

Community clinical 

clerkship (mandatory) – 

1-month rotation during 

3rd year in medical 

school. Rural clinic 

clerkship (mandatory) – 

1 month rotation during 

3rd year in medical 

school. 

Community hospital 

rotations (mandatory) – 

2 months during 4th year 

of medical school.

Post-only

Rural: 

Community with 

population 

<50,000

30% of graduates 

chose to practice in a 

rural location. In 

regression models, the 

optional summer 

program increased

probability of locating in 

a rural area by about 

24%

N/A N/A Programs 

implemented by 

osteopathic school 

can influence where 

graduates choose 

to practice. In 

particular, programs 

taking place in both 

the 1st and 4th year 

of training have an 

impact on rural 

practice location.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

74

Zink, 

201082,94,95

University of 

Minnesota; 

Duluth and 

Twin Cities 

campuses

Minnesota

Capacity: 

Duluth 

average class 

size = 60 

students; 

RPAP 

average class 

size = 33 

students

Medical 

students

N = 3365

RPAP and 

Duluth 

internal 

database 

matched to 

AMA 

Masterfile

Duluth Campus: During 

the 1st year of medical 

school, each student is 

assigned to a practicing 

family physician. 

Students meet with their 

preceptors 10x per year. 

Sessions expose 

students to the variety of 

activities (eg, hospital 

rounds, clinic practice, 

nursing home rounds). 

At the end of the 1st year 

and again for 3 sessions 

during the 2nd year, 

students live with a rural 

preceptor and his or her 

family for 3 days and are 

exposed to the 

physician’s everyday 

working environment 

and lifestyle. These 

communities are located 

in Minnesota and 

western Wisconsin, and 

many are contiguous to 

or located in medically 

underserved areas. 

Faculty meet with 

students 3x/year to 

discuss the students’

experiences.

RPAP (Rural Physicians 

Associates Program) 

Students spend 9 

months in a rural 

community under the 

mentorship of a primary 

care preceptor. RPAP 

developed its training 

model in the early 

1970s. Students 

experience the full scope 

of primary care and 

Comparative

study

Rural: Office of 

Management 

and Budget’s 

definitions of 

metropolitan and 

nonmetropolitan 

populations. 

Communities 

that were not 

listed were 

considered 

metropolitan if 

within 50 miles 

of an urban 

center, 

otherwise 

designated as 

rural

43% of RPAP 

graduates are in rural 

settings and 28% of 

Duluth campus without 

RPAP. Logistic 

regression 

demonstrated that the 

RPAP (OR 4.62 (3.01-

7.09, p<.001)) and 

UMN–Duluth (OR 4.09 

(2.81-5.96, p<.001)) 

experiences are 

additive for the 

outcome of graduating 

students who chose 

rural practice

N/A N/A RPAP and UMN–

Duluth provide 

significant, 

complementary 

educational 

programs that lead 

more graduates to 

choose rural and 

primary care 

practices. Efforts 

across the nation to 

address the crisis in 

rural primary care 

should build on 

these successful 

efforts. Both are 

achieving their 

missions, and the 

programs are 

complementary.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

75

become part of the 

small-town community. 

They follow patients over 

time, acquire hands-on 

experience, and 

complete specialty 

rotations. In recent 

years, online learning 

modules and class 

discussions fostering 

connectivity and learning 

across sites were 

incorporated.

Evaluations across Multiple Training Institutions

Baker, 

2012101

West Virginia

N/A

Capacity: N/A

Medical 

school 

graduates 

practicing in 

Appalachia

N = 44,894

2009 AMA 

Masterfile

US medical schools 

providing physicians for 

the Appalachian region 

of the US

Post-only

Rural: rural or 

urban according 

to 1999 federal 

metropolitan and 

nonmetropolitan 

designations

10 schools produced 

50.5% of all US 

medical school 

graduates from these 

years who practiced 

primary care in rural 

Appalachian counties.

The West Virginia 

School of Osteopathic 

Medicine ranked 1st, 

followed by the 

University of Pikeville 

Kentucky College of 

Osteopathic Medicine 

and the University of 

Alabama School of 

Medicine. 6/10 are 

located not just in 

Appalachian states but 

actually in Appalachian 

counties

N/A N/A Physicians 

practicing in 

Appalachia are 

largely graduates of 

medical schools in 

or near the region. 

New schools being 

developed in the 

region may help 

reduce its 

continuing physician 

shortages.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

76

Deutchman, 

201398

35 US 

medical 

schools with 

established 

rural tracks or 

rural tracks in 

development, 

38 rural track 

Capacity: The 

number of 

students 

participating 

in each RT 

ranges from

4-60, with the 

majority 

representing 

5% to 10% of 

each class at 

that 

institution.

MD and DO 

students

N = N/A

Information 

collected 

and 

provided by 

each Rural 

Track 

program

Rural tracks (RT) 

commonly employ 

curriculum elements 

across all years of 

training. These elements 

serve to expose students 

to rural-related 

healthcare topics in 

years 1 and 2, provide 

early and lengthy rural 

clinical experience, and 

form a social network 

with other like-minded 

students and faculty. 

Clinical clerkships in 

rural communities take a 

variety of forms within 

each RT. Some students 

go for a year or more to 

a rural site that serves 

as a branch campus. 

More commonly, 

students spend several 

months in 1 rural 

location either rotating 

among physicians or 

with one main clinical 

faculty member or group. 

These longer 

experiences integrate 

the learning objectives of 

several formerly 

separate clerkship topics 

in a longitudinal model.

Post-only

Rural: Defined 

by presence of 

Rural Track site

The 18 programs that 

have been able to track 

students' practice 

locations report that an 

average of 44% of their 

graduates practice in a 

rural area (range: 20-

73%)

N/A Based on the 

findings and 

lessons learned in 

this report it would 

be helpful for RTs to 

track and report 

standardized 

parameters. This, in 

combination with 

knowledge of 

admissions and 

curriculum 

information, will help 

define and refine 

best practices for 

education of the 

future rural 

physician workforce.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

77

Patterson, 

201399

Multiple 

family 

medicine 

residencies 

with Rural 

Training 

Tracks 

supported by 

the RTT 

Technical 

Assistance 

Program

Multiple

Capacity: The 

18 RTTs in 

this study 

each 

graduated an 

average of 

1.5 

physicians 

per year.

Family 

Medicine 

residents

N = 123

Survey of 

RTT 

Programs, 

AMA

Masterfile, 

American 

Board of 

Family 

Medicine, 

and Centers 

for Medicare 

and 

Medicaid 

Services.

The “1-2” family 

medicine rural training 

track (RTT) model 

combines 1 year of 

urban training with 2 

years of rural training

Comparative

study

Rural: ZIP codes

72% of graduates 

began clinical practice 

in rural areas (2-3x the 

proportion of family 

medicine residency 

graduates overall)

3 years after 

graduation: 

some 

migration 

away from 

rural (60.6% 

of graduates 

in rural areas)

N/A RTT programs 

continue to succeed 

in recruiting and 

preparing family 

physicians for 

practice with rural 

and underserved 

populations. 
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

78

Phillips, 

200960

US Allopathic 

Medical 

Schools

US

Capacity: N/A

Medical 

students

N = 322,131

Medicare 

outpatient 

institutional 

claims filed 

from Rural 

Health 

Centers 

(RHC), and 

rural primary 

care 

hospitals.

Since 1976, DHHS Title 

VII funding has provided 

substantial support for 

Family Medicine 

educational programs at 

medical schools in many 

states. These curricula 

focus on development of 

primary care physicians 

who would care for 

urban and rural 

underserved populations

Post-only

Rural: RUCA 

Code

Title VII-funded school 

experience increases 

likelihood of rural 

practice (OR 1.11, 

sign.) but not Title VIIfunded residency (OR 

0.90, sign.)

Relative likelihood of 

association between 

Title VII funding and 

practice in rural area 

(RR 1.31, sign.) and 

practice in rural health 

center (RR 1.20, sign.). 

Practice in rural area 

associated with predoc 

Title VII training funding 

(RR 1.39, sign.) and 

matriculated in Title VII 

funded school (RR 

1.31, sign.).

N/A N/A Title VII funding has 

languished over the 

last decade and is 

due for 

reauthorization.

There is 

overwhelming 

evidence, confirmed 

in this study, of the 

beneficial effects 

associated with this 

small federal 

program.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

79

Phillips, 

201389

American 

Board of 

Family 

Medicine, 

Washington 

D.C.

N/A

Capacity: N/A

Residents 

who trained 

in Critical 

Access 

Hospital 

(CAH), 

RHC, or 

FQHC 

between 

2001-2005 

and 

compared to 

practice 

location in 

2009

N = 3,430

Medicare 

claims 2001-

2005, 2009, 

2011 AMA 

Masterfile

RHC: nonurbanized 

area, MUA, HPSA, or 

GDSA, NPs, CNMs, or 

PAs at least 50% of the 

time the clinic is open, 

nonprofit or for profit, 

sliding fee schedule 

optional, provider 

productivity standard, 

cost-based Medicare, 

Medicaid prospective 

payment system. 

FQHC: MUA or MUP, 

tax-exempt nonprofit or 

public, Board of 

Directors, majority from 

community served, 

sliding fee schedule 

(must), provider 

productivity standard, 

FTCA malpractice 

coverage, 

Comprehensive Services 

requirement, cost-based 

Medicare, Medicaid, 

eligible for federal grant 

support. 

CAH: rural area, more 

than 35 miles from 

nearest hospital (15 in 

mountainous areas), 24-

hour emergency care, 25 

or fewer beds, average 

stay 96 hours or less, 

cost-based Medicare. 

[MUA: medically 

underserved area;

GDSA: governordesignated shortage 

area; MUP: medically 

underserved population;

FTCA: Federal Tort 

Post-only

Rural: N/A

52.6% of the residents 

who trained in a CAH, 

38.1% who trained in 

an RHC, and 31.2% 

who trained in an 

FQHC practiced in a 

safety net setting in 

2009. Of CAH trainees, 

40.9% remained in 

CAH, of RHC trainees, 

10.4%, and, of FQHC 

trainees, 12.5%

N/A N/A Using Medicare 

claims data, the 

authors identified 

residents who 

trained in safety net 

settings and 

demonstrated that 

many went on to 

practice in these 

settings. They 

recommend that 

graduate medical 

education policy 

support or expand 

training in these 

settings to meet the 

surge in healthcare

demand that will 

occur with the 

enactment of the 

Affordable Care Act

insurance provision 

in 2014.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

80

Claims Act]

Rabinowitz, 

201291

Jefferson 

Medical 

College; 

Pennsylvania

Jefferson 

Medical 

College, 

Pennsylvania

; University of 

MinnesotaDuluth 

campus; 

University of 

Illinois 

Rockford 

campus

Capacity: 

Jefferson 

PSAP 

average 12.1 

grads per 

year; Duluth 

average 42.9 

grads per 

year; RMED 

average 14.7 

per year

Graduates 

of 3 ruralspecific 

training 

programs

N = 1,757 

RP 

graduates 

versus 

6,474 IMGs

2010 AMA 

Masterfile 

information 

on 

graduates 

from 3 rural 

programs

and 

compared to 

practicing 

IMGs 

3 long-standing and 

successful rural 

programs (RPs) that 

include preferential 

admission of students 

likely to practice in rural 

areas as well as having 

a required rural 

curriculum: the Physician 

Shortage Area Program 

(PSAP) of Jefferson 

Medical College of 

Thomas Jefferson 

University; the University 

of Minnesota Medical 

School Duluth (UMD, a 

2-year program where 

students take their final 2

years at the University of 

Minnesota Minneapolis); 

and the Rural Medical 

Education Program 

(RMED) at the University 

of Illinois College of 

Medicine at Rockford

Comparative

study

Rural: RUDT, 

practicing in a 

nonmetropolitan 

county according 

RUCC

63% of graduates 

practicing in a rural 

area (IMGs: 26.5% 

were in rural areas)

N/A Although 

there were 

almost 7x

as many 

IMGs as 

RP 

graduates, 

the 

absolute 

number of 

RP 

graduates 

practicing 

rural family 

medicine 

was 1.5x 

greater 

than IMGs 

Despite their 

relatively small size, 

RPs had a 

significant impact 

on rural family 

physician and 

primary care supply 

compared with the 

much larger cohort 

of IMGs. Wider 

adoption of the RP 

model would 

substantially 

increase access to 

care in rural areas 

compared with 

increasing reliance 

on IMGs or 

unfocused 

expansion of 

traditional medical 

schools.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

81

Shipman, 

2013100

N/A

N/A

Capacity: N/A

US medical 

school 

graduates

N = 1999-

2001:

48,862, 

2009-2011: 

56,276 

American 

Medical 

College 

Application 

Service for 2 

time periods

Comparison of US 

medical school 

graduates prior to 

expansion (1999-2001) 

to a period 10 years after 

expansion (2009-2011); 

in 2006, AAMC had 

called for a 30% 

increase in MD-granting 

medical school 

enrollment by 2015

Pre-post

RUCC to classify 

the county as 

nonrural (codes 

0-5) or rural 

(codes 6-9)

Schools with higher 

growth rates were more 

likely than schools with 

lower or no growth to 

produce graduates 

practicing in rural 

areas, range between 

5.6 and 8.6% (p<.001)

N/A Despite 

expansion, 

number of 

matriculant

s born in 

rural areas 

decreased

Despite expansion, 

the characteristics 

of matriculating 

medical students 

changed little, 

except at new 

schools. Further 

expansion may 

benefit from 

targeted 

consideration of 

workforce needs.

Talley, 

201187

US

Rural rotation 

site

Capacity: N/A

Emergency 

medicine 

(EM) 

residents

N = 197 

residents 

completed a 

rural rotation

111/126 

programs 

responded 

for 88% 

response 

rate

Comparison of different 

programs. Rural

rotations as 1) required, 

2) elective at 

predesignated site, 3) 

elective, but resident 

must create own rotation 

(no predesignated site), 

or 4) not available

Post-only

Rural: N/A

7% of residents who 

completed a rural 

rotation selected their 

initial job in a rural 

area. 

EM residency 

graduates were more 

likely to select a rural 

job when rural rotations 

were required (22%), 

compared to other 

options: predesignated 

(7%) or not 

predesignated (6%), 

elective or not available 

(7%; p < 0.001)

N/A N/A Elective rural

rotations at 

predesignated sites 

increase resident 

exposure to rural 

areas compared to 

programs without, 

but neither was 

associated with 

rural practice after 

graduation. EM 

residency programs 

that required a rural 

rotation had 

increased resident 

selection of rural 

jobs, but only 5% of 

programs had this 

requirement.

AAFP = American Academy of Family Physicians, ACGME = Accreditation Council for Graduate Medical Education, min = minutes, FQHC = Federally Qualified Health Center, 

HPSA = Health Professional Shortage Areas, PY = program year, RHC = Rural Health Centers, RUCC = Rural-Urban Continuum Codes, RUDT = Rural-Urban Density 

Typology, RR = relative risk, OR = odds ratio
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

82

Summary of Findings and Quality of Evidence for Key Question 5 

This section summarizes the effectiveness of interventions aimed at healthcare providers in 

training with the goal of recruiting and retaining them in rural practice after graduation or 

completion of training. The primary outcome is recruitment success for rural healthcare. 

The figure documents the percentage of students entering rural care across all studies reporting 

on 22 samples to provide a general overview. 

Figure 2: Effect of Student Interventions on Recruitment to Rural Healthcare

Percentage

Frequency

0 20 40 60 80 100

0 1 2

3

4

5

Note: The histogram shows the relative frequency of reported recruitment success expressed as the percentage of 

trainees practicing in rural healthcare. 

Across all identified approaches for healthcare providers in training, the percentage of choosing 

rural healthcare at least as the first site of practice varied widely. However, most estimates were 

in the range of 35 to 65%. The mean across all studies was 49%, the median 53%. 

Restricting to the studies reporting on programs of a single institution, the mean recruitment 

success was 53% (median 55%). This included programs for medicine students as well as 

programs specifically aimed at residents in general surgery, psychiatry, and family medicine. 

The 5 rural track programs exclusively aimed at medical residents88,93,96,102,110 reported a mean of 

providers successfully recruited into rural healthcare of 59% (median 60%). The largest success 

rate (86%) was reported for a small Texas rural training track program for family medicine 

residents, but the study included only 7 participants.93

Stratifying programs by intensity we found that training programs with greater than 6 months 

cumulative time spent in rural training had a median success rate of 51%, versus 53% for those 

comprising less than 6 months of rural experience. 
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

83

Some of the identified studies reported on data across multiple training institutions. A study 

evaluating whether the 2006 AAMC request for a 30% increase in US MD-granting medical 

school enrollment by 2015 was successful in producing more physicians practicing in rural care 

showed that in schools with high growth, 8.6% of graduates practice in rural healthcare, 

compared to 5.6, 7.4, and 7.3% for no-, low-, and moderate-growth schools.

100 A survey of 

program directors of emergency medicine residency programs reported that of all residents who 

had completed a rural rotation, 7% selected their initial job in a rural area.87 The percentage was 

22% when restricting to students in programs where the rural rotation was a required element.

An evaluation of 35 medical schools with 18 rural track programs that were able to identify

students’ practice location reported an average of 44% of their graduates practice in rural areas 

(range 20-73%).98 A study accessing data from 3 rural programs that include preferential 

admission of students likely to practice in rural areas as well as having a required rural 

curriculum (Physician Shortage Area Program, Jefferson Medical College; University of 

Minnesota Medical School Duluth campus; and Rural Medical Education Program, University of 

Illinois) reported that 63% of the graduates practice in a rural area.91

A study found that of residents having trained in a critical access hospital, 48% practiced in a 

critical access hospital or a rural health clinic and 36% who trained in a rural health clinic 

practiced in a critical access hospital or a rural health clinic.89 Family medicine residencies with 

rural training tracks supported by the Rural Training Track Technical Assistance Program 

reported that 72% of graduates began clinical practice in rural areas.99

Among the studies that reported on retention of trainees in rural areas, between 37% and 82% of 

trainees stayed and practiced in the rural areas in which they were trained105,110 but data on 

duration of retention was lacking.

We did not identify any data specific to VA providers. 
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

84

SUMMARY AND DISCUSSION

The review demonstrates rural healthcare workforce needs, determinants of providers’

geographic choices, the lack of intervention studies aimed at US providers, and a multitude of 

recent provider in training efforts. 

We identified a small number of studies quantifying current healthcare provider needs (8 

studies). Estimates were for specific regions and specific years, and studies operationalized 

provider need differently. Hence it is difficult to make concrete or generalizable evidence 

statements regarding the number of healthcare providers needed. However, all included studies 

reported current unmet healthcare provider needs that worsen with increasing rurality. There is in 

particular a healthcare provider shortage for primary care providers, mental health professionals, 

and general surgeons. 

We identified 3 published studies estimating future provider need (KQ1). Studies made 

predictions for primary care physicians, emergency physicians, and surgeons. We did not 

identify 2 or more studies reporting on the same provider group. We also did not find studies 

addressing other provider groups of interest for this review. All studies concluded that the supply 

is not likely to meet demand.

A large number of studies exploring the determinants of practicing in rural care (KQ2) has been 

published. Growing up in a rural community was the most consistent factor associated with 

practice location choice. Education efforts for physicians, such as rural tracks, also seem to 

increase the likelihood of practicing in a rural community. Positive associations were also shown 

for family, exposure to rural communities, a primary care and family medicine orientation, 

osteopathic degree, recreation activities, and rural lifestyle; however, the evidence base is very 

limited. Although a large number of studies explored potential factors, further studies are needed 

to determine the relative importance of the predictor variables.

For KQ3, we identified only 5 evaluations that were aimed at practicing providers, J-1 visa 

waiver, and loan forgiveness program evaluations. A J-1 visa program evaluation reported that 

53% of physicians did not complete their obligations but of the respondents who had completed 

commitments, 84% remained longer than required.83 A state-wide evaluation of various 

recruiting programs reported that 80% of placed providers have remained at their initial 

placement site upon completion of their obligation.86 The loan repayment programs reported on 

different outcomes. One reported 86% continued medical practice at their sponsoring facility 

after completing their obligation. One reported that of those recipients whose obligated practice 

location was rural, 84% were practicing in rural areas and 28% of program completers were 

practicing in rural areas. One evaluation highlighted that 74% of recipients were already working 

in or intended to work in an eligible community when they were made aware of the program but 

the program was important to retention in rural healthcare for 38%.

We did not identify any study specifically aimed at improving retention of fully trained

healthcare providers practicing in rural healthcare facilities (KQ4). 

We identified a large number of program evaluations focusing on providers in training.

However, all were programs aimed at medical students and residents (KQ5). Across individual 

approaches, studies reported a median success rate for recruitment into rural healthcare of 53%.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

85

DISCUSSION BY KEY QUESTION

The included studies quantify shortages for rural areas, for specific provider groups (primary 

care providers, mental health professionals, and general surgeons) and for a defined period of 

time. The specific estimates of unmet needs underpin the perceived shortage of healthcare 

providers, in particular for rural areas, as highlighted in numerous publications on the topic.7,9-

11,111-118 HRSA provides annually updated, online access to all designated Health Professional 

Shortage Areas (HPSAs) stratified by primary medical care, mental health, and dental health.119

The variability in the metrics used to define unmet need illustrate the difficulty of quantifying 

provider needs and comparing unmet needs across geographic regions and provider disciplines.

Although this review only targeted research describing practicing healthcare providers relevant 

to the current healthcare system, we identified a large number of studies contributing to key 

question 2. We identified growing up in a rural community as the most consistent factor 

associated with practice location choice. This factor has also been identified in earlier datasets120

and it appears that this aspect continues to play an important role in determining the choice of 

location. The recent literature also includes multivariate analyses that were able to confirm the 

effect without obvious confounders.

72,78 Despite this association, the evidence suggests that most 

rural physicians did not grow up in a small town. Among physicians moving to a rural setting, 

factors such as feeling established in practice with a close connection to patients and location 

were noted to be of importance. 65,80 A second key predictor that emerged in the literature is

education efforts such as rural tracks for physicians. These seem to increase the likelihood of 

practicing in a rural community, and although the choice of selecting a rural track may be in part 

determined by a personal affinity preceding the choice of school, multivariate analyses showed

that the effect cannot be entirely explained by the variable of growing up in a rural community.72

However, there is a lack of studies differentiating the relative importance of a personal affinity 

for rural communities, motivation through rural training, and effects of interventions attracting 

trained healthcare providers into rural care settings.61

For KQ3, we only identified a very small number of studies aimed at the recruitment of 

healthcare providers for rural communities. This is consistent with a recent Cochrane review on 

interventions to increase the proportion of health professionals practicing in rural and other 

underserved areas; the review includes only one study.121 The included study reported on an 

international setting (Taiwan). The Cochrane review concluded that there is currently limited 

reliable evidence regarding the effects of interventions aimed at addressing the inequitable 

distribution of health professionals. A 2010 review for the World Health Organization on 

interventions to increase attraction and retention of health workers in remote areas included 3 

studies directed at healthcare providers (rather than providers in training): studies addressed 

community service for doctors in South Africa, financial incentives in the Niger, and bundled 

interventions for rural areas in Mali.

27 Studies reported a 25% proportion of participants placed 

in rural areas, an increase of 44% in the number of doctors practicing outside the capital city, and 

the total number of physicians installed in rural areas over a 10-year period (100) but the review 

cautioned that there is a need for more thorough evaluations to support policy-makers in 

developing, implementing, and evaluating effective interventions to increase the availability of 

health workers in underserved areas.

One of the identified evaluations in our systematic review was a J-1 visa waiver program 

evaluation at state level and it highlighted that programs need to be evaluated carefully. The 
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

86

study reported that half of physicians did not complete their obligation. However, of the 

respondents who had completed their commitments, over 80% remained longer than required. 83

This is mirrored by earlier evaluations of the program; an evaluation for rural Wisconsin

concluded that to keep physicians practicing in communities, successful integration into the 

community is important.122 Research on loan forgiveness programs is sparse and the identified 

evaluations assessed different outcomes and results. Careful evaluation is needed given that one

included study highlighted that a large proportion of participants were working in or intended to 

work in an eligible community when they were made aware of the program. Hence, the program 

may influence their retention in rural healthcare but was not key to entering rural healthcare.

62 A 

critical review of interventions to redress the inequitable distribution of healthcare professionals 

to rural and remote areas summarized the reviewed literature published to 2008, as many servicelined scholarships, loans, and loan repayment programs have been described, but the effect of 

these on the rural or remote workforce are not clear.123

We did not identify any recent study specifically aimed at improving retention for healthcare 

providers in rural healthcare facilities (KQ4). This finding is consistent with a recent review on 

physician recruitment and retention in rural and underserved areas. The review highlighted that 

several authors have suggested recruitment and retention techniques; however, there is a need for 

a research agenda that includes valid, reliable, and rigorous analyses regarding formulating and 

implementing these strategies.124 Review articles have frequently pointed to models of telehealth 

and their expected influence on rural care,21,36 but there is a lack of studies providing empirical 

data of the effects of these innovations. There was a similar lack of evidence to evaluate whether 

there is a critical duration of rural practice after which providers are more likely to remain in that 

setting for an extended time or the duration of their career. Furthermore, we did not identify any 

study assessing the effect of education and continuous professional development on rural 

healthcare providers. Our review concentrated on specific healthcare provider groups; however, 

a review of reviews on interventions for supporting nurse retention in rural and remote areas also 

concluded that more knowledge is needed regarding the effectiveness of specific strategies to 

address the factors known to contribute to nurse retention,125 suggesting that the absence of 

systematic evaluations is not necessarily provider group-specific. While our review concentrated 

on the US setting, a 2010 systematic review on the international literature on retention incentives 

for health workers in rural and remote areas reported that little evidence demonstrated the 

effectiveness of any specific strategy, with the possible exception of health worker obligation.126

The review suggested a framework of 6 components (staffing, infrastructure, remuneration, 

workplace organization, professional environment, and social, family, and community support) 

and concluded that retention strategies should be rigorously evaluated.

In the last 10 years a large number of studies have been published that evaluate programs for 

healthcare providers in training (KQ5). However, the literature is dominated by studies focusing 

on medical students and residents. Across all approaches, studies reported a median success rate 

for recruiting healthcare providers to rural care of 53%, meaning that about half the students 

trained for rural healthcare enter rural settings. The success rate varied across datasets and 

programs but most estimates ranged between 35 and 65%. This included a study that combined 

data from 3 identified programs that give preference to students with a rural background and a 

rural track approach ensuring exposure to rural healthcare; the reported success rate was 63%.

91

Individual training programs varied widely in format and duration but we did not identify factors 

that systematically affected success rates. For example, training programs with greater than 6

months’ time spent in rural training had a median success rate of 51%, versus 53% for those 
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

87

comprising less than 6 months of rural experience. Our identified data are comparable to the 

estimate of a prior systematic review with data to 2006 that reported a weighted average 53 to 

64%.127

While the data contained in the included studies suggest that it is difficult to recruit, retain, and 

train rural providers, the studies provide little exploration of the specific challenges of rural 

practice. These challenges are illustrated well in the results of a 2011 survey performed by Chipp 

et al.

32 Rural providers were asked, simply, “What are the 3 things you wish someone would 

have told you about delivering healthcare in rural areas?” Responses relating to challenges of 

rural healthcare provision frequently touched on challenges in community relationship-building, 

personal and professional boundaries, and rural lifestyle challenges and self-care practices. One 

respondent, for example, replied, “Self care is one of the primary things, because of isolation; 

you have to be a person who knows how to nurture yourself, to replenish yourself.” Although an 

in-depth discussion of these challenges and difficulties lies outside the scope of this review, these 

results offer some understanding of the forces shaping the rural provider workforce and the 

results of the included studies.

LIMITATIONS

Workforce supply and demand is part of a complex and dynamic system. While our review 

targeted only recent research, that is within the last 10 years, the effects of newer developments 

such as the Affordable Care Act will add new components.128 Our systematic review captures 

research published to February 2015 but publications continue to address this complex 

topic.129,130

We restricted the review to a specific range of healthcare disciplines, chosen by consensus as 

those most frequently appearing in the literature pertinent to Community-Based Outpatient 

Clinics, Rural Health Clinics, and Critical Access Hospitals. Additional providers, whose role in 

providing rural care may be relevant, fell outside the scope of the study. HRSA provides 

information on health workforce projections for various other healthcare providers such as 

psychologists and pharmacists.131

Our review focused on specific empirical evidence, for example provider demand estimates, and 

program evaluations for healthcare practitioners had to report on recruitment or retention 

measures to be eligible. This left out studies assessing supply, rather than demand,132,133 or 

evaluations assessing provider satisfaction with programs.

134-136

Finally, as shown in the evidence tables throughout the report, “rural” was operationalized 

differently across identified studies which introduced heterogeneity across studies.

RESEARCH GAPS/FUTURE RESEARCH

Supply information is now widely available. However, few studies exist that have quantified 

provider needs for rural areas. HRSA provides a model for estimating provider demand that can 

be used for estimates. Predictive models are very complex and need to be continuously updated 

due to developments such as the Affordable Care Act. Although the shortage of healthcare 

providers is often cited, specific estimates are needed, in particular for healthcare providers other 

than physicians. Furthermore, there is a need to identify and assess the skills and competencies 

needed by current and future rural healthcare providers, in order to optimize the impact of these 
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

88

providers, to predict the supply and demand according to geographic region, to shape new 

models of care, and to effectively utilize technology to ameliorate issues physician shortage and 

patient access. The impact of federal and state policies on supply and demand warrants 

investigation. In particular, research considering the context above and focusing on VA-specific 

care in rural settings is needed.

Although we have identified many studies addressing providers’ geographic choices for practice, 

few were designed to determine the relative importance of contributing factors. For physicians in 

particular, studies should focus on the context of rural upbringing, academic preparation, and 

competitive medical school admissions processes to understand individual decisions of where 

providers choose to practice. In addition, there is a need for further, multivariate analyses, 

simultaneously exploring the effects of personal background, training, time spent in rural 

placements, and interventions aimed to recruit healthcare provider groups for rural care.

We identified only very few intervention evaluations that targeted providers, rather than students 

and residents. In particular, empirical evidence of strategies to improve provider retention in 

rural areas is missing. There is a lack of intervention studies evaluating organizational 

interventions (eg, estimating the effect of implemented continuous education, clinical support, 

and inter-professional collaboration). We also did not did not identify evaluations that reported 

on the current effect of a federal program, the National Health Service Corps. Earlier evaluations 

were positive137,138 but effect estimates should be intermittently updated to assess effects of the 

American Recovery and Reinvestment Act and the Patient Protection and Affordable Care Act.19

In recent years, additional approaches to recruiting and retaining US healthcare providers have 

been suggested that should be evaluated in future research studies.139-142 Data is currently lacking 

on the effects of improved access to Continuing Medical Education, availability of e-consults 

with specialists, expanded consult networks, and other interventions aimed at improving ease of 

practice in rural areas. 

Studies of training interventions for students and residents varied widely in duration, intensity, 

balance of acute care to rural or primary care exposure, geographic regions, program elements, 

and other factors making impacts difficult to compare across studies. Analyses are needed that 

parse out the specific aspects of the training experience that influenced success. To date it is, for 

example, unknown whether the rural-specific skills training is crucial, or if simply spending time 

in the rural setting is sufficient. There is a lack of studies evaluating the comparative 

effectiveness of different training programs. Moreover, effects of training programs for 

healthcare providers other than medical students and residents are needed, at least to demonstrate 

that effects are comparable to medical school training efforts. 

CONCLUSIONS

All included studies reported current unmet healthcare provider needs that worsen with 

increasing rurality. The small number of studies estimating future need also predicted unmet 

provider needs for rural healthcare. 

Growing up in a rural community is the most consistent factor associated with practice location.

Education efforts for physicians, such as rural tracks, also seem to increase the likelihood of 

practicing in a rural community. More research on the relative importance of factors is needed.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

89

More research is needed to evaluate existing healthcare provider recruitment interventions for 

rural healthcare. 

There is a lack of evidence regarding interventions to support healthcare provider retention in 

rural healthcare.

Current evaluations of rural training programs for medical students and residents suggest a 

median success rate of 53%.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

90

REFERENCES

1. Rosenblatt RA, Chen FM, Lishner DM, Doescher MP. The Future of Family Medicine 

and Implications for Rural Primary Care Physician Supply. Final Report #125, August 

2010. Seattle, WA: WWAMI Rural Health Research Center, University of Washington 

School of Medicine Department of Family Medicine.;2010.

2. Owen JA, Conaway MR, Bailey BA, Hayden GF. Predicting rural practice using different 

definitions to classify medical school applicants as having a rural upbringing. The 

Journal of rural health : official journal of the American Rural Health Association and 

the National Rural Health Care Association. Spring 2007;23(2):133-140.

3. Ortiz J, Bushy A, Zhou Y, Zhang H. Accountable care organizations: benefits and 

barriers as perceived by Rural Health Clinic management. Rural and remote health. AprJun 2013;13(2):2417.

4. Weeks WB, Wallace AE, West AN, Heady HR, Hawthorne K. Research on rural 

veterans: An analysis of the literature. Journal of Rural Health. Fal 2008;24(4):337-344.

5. MacDowell M, Glasser M, Fitts M, Fratzke M, Peters K. Perspectives on rural health 

workforce issues: Illinois-Arkansas comparison. The Journal of rural health : official 

journal of the American Rural Health Association and the National Rural Health Care 

Association. Spring 2009;25(2):135-140.

6. Doty B, Andres M, Zuckerman R, Borgstrom D. Use of Locum Tenens Surgeons to 

Provide Surgical Care in Small Rural Hospitals. World J.Surg. Feb 2009;33(2):228-232.

7. Rosenblatt RA, Andrilla CH, Curtin T, Hart LG. Shortages of medical personnel at 

community health centers: implications for planned expansion. Jama. Mar 1 

2006;295(9):1042-1049.

8. Doty B, Zuckerman R, Finlayson S, Jenkins P, Rieb N, Heneghan S. General surgery at 

rural hospitals: a national survey of rural hospital administrators. Surgery. May 

2008;143(5):599-606.

9. MacDowell M, Glasser M, Fitts M, Nielsen K, Hunsaker M. A national view of rural 

health workforce issues in the USA. Rural and remote health. Jul-Sep 2010;10(3):1531.

10. Thompson MJ, Lynge DC, Larson EH, Tachawachira P, Hart LG. Characterizing the 

general surgery workforce in rural America. Archives of surgery (Chicago, Ill. : 1960). 

Jan 2005;140(1):74-79.

11. Garrison-Jakel J. Patching the rural workforce pipeline--why don't we do more? The 

Journal of rural health : official journal of the American Rural Health Association and 

the National Rural Health Care Association. Spring 2011;27(2):239-240.

12. Mareck DG. Federal and state initiatives to recruit physicians to rural areas. The virtual 

mentor : VM. 2011;13(5):304-309.

13. Dill MJ, Salsberg ES. The Complexities of Physician Supply and Demand: Projections 

Through 2025. November 2008. Washington, DC: Association of American Medical 

Colleges, Center for Workforce Studies.;2008.

14. American Hospital Association. Physician Shortage &Distribution. 

http://www.aha.org/content/00-10/ruralphysshortage.pdf. Accessed 09/08/2015.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

91

15. Cofer JB, Burns RP. The developing crisis in the national general surgery workforce. 

Journal of the American College of Surgeons. May 2008;206(5):790-795; discussion 

795-797.

16. US Department of Health and Human Services, Health Resources and Services 

Administration, National Center for Health Workforce Analysis. Technical 

Documentation for Health Resources Service Administration’s Health Workforce 

Simulation Model. Rockville, MD: U.S. Department of Health and Human Services;2014.

17. Mullan F, Frehywot S, Jolley LJ. Aging, primary care, and self-sufficiency: health care 

workforce challenges ahead. The Journal of law, medicine & ethics : a journal of the 

American Society of Law, Medicine & Ethics. Winter 2008;36(4):703-708, 608.

18. Bridgham RG. Final report : HB 1615, (Chapter 367:3, Laws of 2008), establishing a 

Commission to Recommend Policies and Programs to Increase the Number of New 

Hampshire Individuals in Health Professions Servicing New Hampshire's Rural and 

Underserved Areas with a Focus on Primary Care. Concord, N.H.: N.H. General Court; 

2009.

19. Price J. The National Health Service Corps--a critical component of provider recruitment 

in North Carolina's rural and underserved communities. North Carolina medical journal. 

May-Jun 2010;71(3):251.

20. Bowman RC. Measuring primary care: the standard primary care year. Rural Remote 

Health. Jul-Sep 2008;8(3).

21. Hilty DM, Yellowlees PM, Cobb HC, Bourgeois JA, Neufeld JD, Nesbitt TS. Models of 

telepsychiatric consultation-liaison service to rural primary care. Psychosomatics. Apr 

2006;47(2):152-157.

22. Staton FS, Bhosle MJ, Camacho FT, Feldman SR, Balkrishnan R. How PAs improve 

access to care for the underserved. JAAPA : official journal of the American Academy of 

Physician Assistants. Jun 2007;20(6):32, 34, 36 passim.

23. Van Vleet A, Paradise J. Issue Brief: Tapping Nurse Practitioners to Meet Rising 

Demand for Primary Care. The Henry J. Kaiser Family Foundation; 2015.

24. Doescher MP, Andrilla CH, Skillman SM, Morgan P, Kaplan L. The contribution of 

physicians, physician assistants, and nurse practitioners toward rural primary care: 

findings from a 13-state survey. Medical care. Jun 2014;52(6):549-556.

25. Lindsay S. Gender differences in rural and urban practice location among mid-level 

health care providers. The Journal of rural health : official journal of the American Rural 

Health Association and the National Rural Health Care Association. Winter 

2007;23(1):72-76.

26. Ricketts TC. Workforce issues in rural areas: A focus on policy equity. Am. J. Public

Health. Jan 2005;95(1):42-48.

27. Dolea C, Stormont L, Braichet JM. Evaluated strategies to increase attraction and 

retention of health workers in remote and rural areas. Bull World Health Organ. May 

2010;88(5):379-385.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

92

28. Lehmann U, Dieleman M, Martineau T. Staffing remote rural areas in middle- and lowincome countries: a literature review of attraction and retention. BMC health services 

research. 2008;8:19.

29. Deveney K, Hunter J. Education for rural surgical practice: the Oregon Health & Science 

University model. The Surgical clinics of North America. Dec 2009;89(6):1303-1308, 

viii.

30. Henry LR, Hooker RS, Yates KL. The role of physician assistants in rural health care: a 

systematic review of the literature. The Journal of rural health : official journal of the 

American Rural Health Association and the National Rural Health Care Association. 

Spring 2011;27(2):220-229.

31. Baker E, Schmitz D, Epperly T, Nukui A, Miller CM. Rural Idaho Family Physicians' 

Scope of Practice. Journal of Rural Health. Win 2010;26(1):85-89.

32. Chipp C, Dewane S, Brems C, Johnson ME, Warner TD, Roberts LW. "If only someone 

had told me...": lessons from rural providers. The Journal of rural health : official journal 

of the American Rural Health Association and the National Rural Health Care 

Association. Winter 2011;27(1):122-130.

33. Filipova AA. Factors influencing the satisfaction of rural physician assistants: a crosssectional study. Journal of allied health. Spring 2014;43(1):22-31.

34. Huff C. Done recruiting? Start retaining. Trustee : the journal for hospital governing 

boards. Jan 2014;67(1):8-12, 11.

35. Meyer D. Technology, job satisfaction, and retention: rural mental health practitioners. 

The Journal of rural health : official journal of the American Rural Health Association 

and the National Rural Health Care Association. Spring 2006;22(2):158-163.

36. Gagnon MP, Pollender H, Trepanier A, Duplaa E, Ly BA. Supporting health 

professionals through information and communication technologies: a systematic review 

of the effects of information and communication technologies on recruitment and 

retention. Telemedicine journal and e-health : the official journal of the American 

Telemedicine Association. May 2011;17(4):269-274.

37. Palmer RT. Exploring online community among rural medical education students: A case 

study. Dissertation Abstracts International Section A: Humanities and Social Sciences. 

2014 2014;75(1-A(E)).

38. Everitt-Deering P. The adoption of information and communication technologies by rural 

general practitioners a socio technical analysis. [Internet Resource; Computer File; 

Archival Material]. 2008; http://eprints.vu.edu.au/1412.

39. Association of American Medical Colleges. AAMC Statement on the Physician 

Workforce, June 2006. 2006.

40. Myers MA, Reed KD. The virtual ICU (vICU): a new dimension for critical care nursing 

practice. Critical care nursing clinics of North America. Dec 2008;20(4):435-439.

41. Pathman DE. What outcomes should we expect from programs that pay 

physicians'training expenses in exchange for service? North Carolina medical journal. 

Jan-Feb 2006;67(1):77-82.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

93

42. Seligson RW, Highsmith PP. North Carolina Medical Society Foundation's Community 

Practitioner Program. North Carolina medical journal. Jan-Feb 2006;67(1):83-85.

43. Chen C, Xierali I, Piwnica-Worms K, Phillips R. The redistribution of graduate medical 

education positions in 2005 failed to boost primary care or rural training. Health affairs 

(Project Hope). Jan 2013;32(1):102-110.

44. Dorsey ER, Nicholson S, Frist WH. Commentary: improving the supply and distribution 

of primary care physicians. Academic medicine : journal of the Association of American 

Medical Colleges. May 2011;86(5):541-543.

45. Viswanathan M, Ansari MT, Berkman ND, et al. Assessing the Risk of Bias of Individual 

Studies in Systematic Reviews of Health Care Interventions. 2008.

46. Swan N. Better access to mental health care. Education Lifeline to the Bush ;; no. 711; 

Variation: Education Lifeline to the Bush ;; no. 711. [Visual Material; Internet 

Resource]. 2007; 1 DVD-video (PAL) (56 min.) : sd., color ; 54 53/54 in. + facilitator's 

guide (17 pages) + learning guide (30 pages). Available at: 

http://www.rhef.com.au/programs/711/711.html

47. Branch K. Alaska’s Health Workforce Vacancy Study: 2012 Findings Report. August 

2014. Anchorage, AK: Alaska Center for Rural Health, Alaska’s Area Health Education 

Center, University of Alaska Anchorage;2014.

48. Maizel SE. Maryland's surgical workforce-2007: an in-depth analysis and implications 

for the future. Journal of the American College of Surgeons. Mar 2009;208(3):454-461.

49. Stewart RM, Liao LF, West M, Sirinek KR. The general surgery workforce shortage is 

worse when assessed at county level. Am. J. Surg. Dec 2013;206(6):1016-1022.

50. Thomas KC, Ellis AR, Konrad TR, Holzer CE, Morrissey JP. County-level estimates of 

mental health professional shortage in the United States. Psychiatric services 

(Washington, D.C.). Oct 2009;60(10):1323-1328.

51. Ghosh D, Sterns AA, Drew BL, Hamera E. Geospatial study of psychiatric mental healthadvanced practice registered nurses (PMH-APRNs) in the United States. Psychiatric 

services (Washington, D.C.). Dec 2011;62(12):1506-1509.

52. Rayburn WF, Klagholz JC, Murray-Krezan C, Dowell LE, Strunk AL. Distribution of 

American Congress of Obstetricians and Gynecologists Fellows and Junior Fellows in 

Practice in the United States. Obstet. Gynecol. May 2012;119(5):1017-1022.

53. Williams TE, Jr., Satiani B, Ellison EC. A comparison of future recruitment needs in 

urban and rural hospitals: the rural imperative. Surgery. Oct 2011;150(4):617-625.

54. Hendryx M. Mental health professional shortage areas in rural Appalachia. Journal of 

Rural Health. Spr 2008;24(2):179-182.

55. Wilson EA, Whitler ET, Asher LM. Meeting the Need of a Rural State for Primary Care 

Physicians: A Health Care Reform Challenge. J. Public Health Manag. Pract. Mar-Apr 

2011;17(2):147-153.

56. Camargo CA, Jr., Ginde AA, Singer AH, et al. Assessment of emergency physician 

workforce needs in the United States, 2005. Academic emergency medicine : official 

journal of the Society for Academic Emergency Medicine. Dec 2008;15(12):1317-1320.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

94

57. Chen F, Fordyce M, Andes S, Hart LG. Which medical schools produce rural physicians? 

A 15-year update. Academic medicine : journal of the Association of American Medical 

Colleges. Apr 2010;85(4):594-598.

58. Fordyce MA, Doescher MP, Chen FM, Hart LG. Osteopathic physicians and 

international medical graduates in the rural primary care physician workforce. Family 

medicine. Jun 2012;44(6):396-403.

59. Kornelsen J, Grzybowski S. Is local maternity care an optional service in rural 

communities? Journal of obstetrics and gynaecology Canada : JOGC = Journal 

d'obstetrique et gynecologie du Canada : JOGC. Apr 2005;27(4):329-331.

60. Phillips RL, Dodoo MS, Petterson S, et al. Specialty and geographic distribution of the 

physician workforce: what influences medical student & resident choices? March 2, 

2009. Washington, DC: The Robert Graham Center. Available at http://www.grahamcenter.org/online/etc/medialib/graham/documents/publications/mongraphsbooks/2009/rgcmo-specialty-geographic.Par.0001.File.tmp/Specialty-geographycompressed.pdf;2009.

61. Rabinowitz HK, Diamond JJ, Markham FW, Santana AJ. The relationship between 

entering medical students' backgrounds and career plans and their rural practice outcomes 

three decades later. Academic medicine : journal of the Association of American Medical 

Colleges. Apr 2012;87(4):493-497.

62. Renner DM, Westfall JM, Wilroy LA, Ginde AA. The influence of loan repayment on 

rural healthcare provider recruitment and retention in Colorado. Rural and remote health. 

Oct-Dec 2010;10(4):1605.

63. Mason PB, Cossman JS. Does one medical school's admission policy help a rural state 

"grow their own" physicians? Journal of the Mississippi State Medical Association. Sep 

2012;53(9):284-286, 288-292.

64. Smith B, Muma RD, Burks L, Lavoie MM. Factors that influence physician assistant 

choice of practice location. JAAPA : official journal of the American Academy of 

Physician Assistants. Mar 2012;25(3):46-51.

65. Henry LR, Hooker RS. Retention of physician assistants in rural health clinics. The 

Journal of rural health : official journal of the American Rural Health Association and 

the National Rural Health Care Association. Summer 2007;23(3):207-214.

66. US Department of Health and Human Services. Physician supply and demand: 

Projections to 2020. 2006; 

http://bhpr.hrsa.gov/healthworkforce/supplydemand/medicine/physician2020projections.

pdf.

67. Duffrin C, Diaz S, Cashion M, Watson R, Cummings D, Jackson N. Factors associated 

with placement of rural primary care physicians in North Carolina. Southern medical 

journal. Nov 2014;107(11):728-733.

68. Glasser M, MacDowell M, Hunsaker M, et al. Factors and outcomes in primary care 

physician retention in rural areas. [Internet Resource; Archival Material]. 2010; 

http://www.ajol.info/index.php/safp/article/view/61437
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

95

69. Hancock C, Steinbach A, Nesbitt TS, Adler SR, Auerswald CL. Why doctors choose 

small towns: a developmental model of rural physician recruitment and retention. Social 

science & medicine (1982). Nov 2009;69(9):1368-1376.

70. Helland LC, Westfall JM, Camargo CA, Jr., Rogers J, Ginde AA. Motivations and 

barriers for recruitment of new emergency medicine residency graduates to rural 

emergency departments. Annals of emergency medicine. Dec 2010;56(6):668-673.

71. Heneghan SJ, Bordley Jt, Dietz PA, Gold MS, Jenkins PL, Zuckerman RJ. Comparison 

of urban and rural general surgeons: motivations for practice location, practice patterns, 

and education requirements. Journal of the American College of Surgeons. Nov 

2005;201(5):732-736.

72. Hughes S, Zweifler J, Schafer S, Smith MA, Athwal S, Blossom HJ. High school census 

tract information predicts practice in rural and minority communities. The Journal of 

rural health : official journal of the American Rural Health Association and the National 

Rural Health Care Association. Summer 2005;21(3):228-232.

73. Jarman BT, Cogbill TH, Mathiason MA, et al. Factors correlated with surgery resident 

choice to practice general surgery in a rural area. Journal of surgical education. Nov-Dec 

2009;66(6):319-324.

74. Kimball EB, Crouse BJ. Perspectives of female physicians practicing in rural Wisconsin. 

WMJ : official publication of the State Medical Society of Wisconsin. Aug 

2007;106(5):256-259.

75. Pepper CM, Sandefer RH, Gray MJ. Recruiting and retaining physicians in very rural 

areas. The Journal of rural health : official journal of the American Rural Health 

Association and the National Rural Health Care Association. Summer 2010;26(2):196-

200.

76. Rabinowitz HK, Diamond JJ, Markham FW, Santana AJ. The relationship between 

matriculating medical students' planned specialties and eventual rural practice outcomes. 

Academic medicine : journal of the Association of American Medical Colleges. Aug 

2012;87(8):1086-1090.

77. Schiff T, Felsing-Watkins J, Small C, Takayesu A, Withy K. Addressing the physician 

shortage in Hawai'i: recruiting medical students who meet the needs of Hawai'i's rural 

communities. Hawai'i journal of medicine & public health : a journal of Asia Pacific 

Medicine & Public Health. Apr 2012;71(4 Suppl 1):21-25.

78. Shannon CK, Jackson J. A study of predictive validity of physician assistant students' 

reported practice site intent. The journal of physician assistant education : the official 

journal of the Physician Assistant Education Association. 2011;22(3):29-32.

79. Snyder J, Nehrenz G, Danielsen R, Pedersen D. Educational debt: does it have an 

influence on initial job location and specialty choice? The journal of physician assistant 

education : the official journal of the Physician Assistant Education Association. 

2014;25(4):39-42.

80. Stenger J, Cashman SB, Savageau JA. The primary care physician workforce in 

Massachusetts: implications for the workforce in rural, small town America. The Journal 

of rural health : official journal of the American Rural Health Association and the 

National Rural Health Care Association. Fall 2008;24(4):375-383.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

96

81. Whitacre BE, Pace V, Hackler JB, Janey M, Landgraf CE, Pettit WJ. An evaluation of 

osteopathic school programs designed to promote rural location by graduates. 

International Journal of Osteopathic Medicine. 2011;14(1):17-23.

82. Zink T, Center B, Finstad D, et al. Efforts to graduate more primary care physicians and 

physicians who will practice in rural areas: examining outcomes from the university of 

Minnesota-duluth and the rural physician associate program. Academic medicine : 

journal of the Association of American Medical Colleges. Apr 2010;85(4):599-604.

83. Kahn TR, Hagopian A, Johnson K. Retention of J-1 visa waiver program physicians in 

Washington State's health professional shortage areas. Academic medicine : journal of 

the Association of American Medical Colleges. Apr 2010;85(4):614-621.

84. Wheeler DLLCE. Oklahoma State University Center for Rural Health report to the 

Senate interim study on comprehensive assessment of healthcare workforce issues. 

[Internet Resource; Computer File]. 2009; 1 online resource (20 p. : 21.41 MB) : col. ill., 

col. maps. Available at: http://digitalprairie.ok.gov/u?/stgovpub,19719

85. Bringing health care to the heartland : an evaluation of Minnesota's loan forgiveness 

programs for select health care occupations. [Minnesota]: Minnesota Dept. of Health, 

Office of Rural Health and Primary Care; 2007.

86. Wheeler M, Endres M, Pauley K, Mahone M, Melton N. Dr. Who? Providing stability to 

recruiting and retaining health care professionals in West Virginia. The West Virginia 

medical journal. Jul-Aug 2013;109(4):62-64.

87. Talley BE, Ann Moore S, Camargo CA, Jr., Rogers J, Ginde AA. Availability and 

potential effect of rural rotations in emergency medicine residency programs. Academic 

emergency medicine : official journal of the Society for Academic Emergency Medicine. 

Mar 2011;18(3):297-300.

88. Bonham C, Salvador M, Altschul D, Silverblatt H. Training Psychiatrists for Rural 

Practice: A 20-Year Follow-up. Academic Psychiatry. Oct 2014;38(5):623-626.

89. Phillips RL, Petterson S, Bazemore A. Do residents who train in safety net settings return 

for practice? Academic medicine : journal of the Association of American Medical 

Colleges. Dec 2013;88(12):1934-1940.

90. Rabinowitz HK, Diamond JJ, Markham FW, Santana AJ. Increasing the supply of rural 

family physicians: recent outcomes from Jefferson Medical College's Physician Shortage 

Area Program (PSAP). Academic medicine : journal of the Association of American 

Medical Colleges. Feb 2011;86(2):264-269.

91. Rabinowitz HK, Petterson S, Boulger JG, et al. Medical school rural programs: a 

comparison with international medical graduates in addressing state-level rural family 

physician and primary care supply. Academic medicine : journal of the Association of 

American Medical Colleges. Apr 2012;87(4):488-492.

92. Quinn KJ, Kane KY, Stevermer JJ, et al. Influencing residency choice and practice 

location through a longitudinal rural pipeline program. Academic medicine : journal of 

the Association of American Medical Colleges. Nov 2011;86(11):1397-1406.

93. Nash LR, Olson MM, Caskey JW, Thompson BL. Outcomes of a Texas family medicine 

residency rural training track: 2000 through 2007. Texas medicine. Sep 2008;104(9):59-

63.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

97

94. Halaas GW, Zink T, Finstad D, Bolin K, Center B. Recruitment and retention of rural 

physicians: outcomes from the rural physician associate program of Minnesota. The 

Journal of rural health : official journal of the American Rural Health Association and 

the National Rural Health Care Association. Fall 2008;24(4):345-352.

95. Halaas GW. The Rural Physician Associate Program: successful outcomes in primary 

care and rural practice. Rural and remote health. Apr-Jun 2005;5(2):453.

96. Antonenko DR. Rural surgery: the North Dakota experience. The Surgical clinics of 

North America. Dec 2009;89(6):1367-1372, x.

97. Deveney K, Deatherage M, Oehling D, Hunter J. Association between dedicated rural 

training year and the likelihood of becoming a general surgeon in a small town. JAMA 

surgery. Sep 2013;148(9):817-821.

98. Deutchman M. Medical School Rural Tracks in the US. Policy brief: September 2013. 

Washington, DC: National Rural Health Association. Available at: 

http://www.ruralhealthweb.org/index.cfm?objectid = 28B352C5-3048-651AFE2D53C27202BAF6;2013.

99. Patterson DG, Longenecker R, Schmitz D, Phillips RL, Skillman SM, Doescher MP. 

January 2013 Policy Brief: Rural Residency Training for Family Medicine Physicians: 

Graduate Early-Career Outcomes, 2008-2012. 2013; 

https://www.raconline.org/rtt/pdf/rural-family-medicine-training-early-career-outcomes2013.pdf. Accessed 8/20/2015.

100. Shipman SA, Jones KC, Erikson CE, Sandberg SF. Exploring the workforce implications 

of a decade of medical school expansion: variations in medical school growth and 

changes in student characteristics and career plans. Academic medicine : journal of the 

Association of American Medical Colleges. Dec 2013;88(12):1904-1912.

101. Baker HH, Pathman DE, Nemitz JW, Boisvert CS, Schwartz RJ, Ridpath LC. Which U.S. 

medical schools are providing the most physicians for the Appalachian region of the 

United States? Academic medicine : journal of the Association of American Medical 

Colleges. Apr 2012;87(4):498-505.

102. Crane S, Jones G. Innovation in rural family medicine training: the Mountain Area 

Health Education Center's rural-track residency program. North Carolina medical 

journal. Jan-Feb 2014;75(1):29-30.

103. Crump WJ, Fricker RS, Ziegler C, Wiegman DL, Rowland ML. Rural track training 

based at a small regional campus: equivalency of training, residency choice, and practice 

location of graduates. Academic medicine : journal of the Association of American 

Medical Colleges. Aug 2013;88(8):1122-1128.

104. Glasser M, Hunsaker M, Sweet K, MacDowell M, Meurer M. A comprehensive medical 

education program response to rural primary care needs. Academic medicine : journal of 

the Association of American Medical Colleges. Oct 2008;83(10):952-961.

105. MacDowell M, Glasser M, Hunsaker M. A decade of rural physician workforce outcomes 

for the Rockford Rural Medical Education (RMED) Program, University of Illinois. 

Academic medicine : journal of the Association of American Medical Colleges. Dec 

2013;88(12):1941-1947.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

98

106. Kallail KJ, McCurdy S. Scholars in Rural Health: Outcomes From an Assured 

Admissions Program. Fam. Med. Nov-Dec 2010;42(10):729-731.

107. Rabinowitz HK, Diamond JJ, Markham FW, Santana AJ. Increasing the supply of 

women physicians in rural areas: outcomes of a medical school rural program. Journal of 

the American Board of Family Medicine : JABFM. Nov-Dec 2011;24(6):740-744.

108. Rabinowitz HK, Diamond JJ, Markham FW, Santana AJ. Retention of rural family 

physicians after 20-25 years: outcomes of a comprehensive medical school rural program. 

Journal of the American Board of Family Medicine : JABFM. Jan-Feb 2013;26(1):24-27.

109. Rabinowitz HK, Diamond JJ, Markham FW, Rabinowitz C. Long-term retention of 

graduates from a program to increase the supply of rural family physicians. Academic 

medicine : journal of the Association of American Medical Colleges. Aug 

2005;80(8):728-732.

110. Ross R. Fifteen-year outcomes of a rural residency: aligning policy with national needs. 

Family medicine. Feb 2013;45(2):122-127.

111. Vogt HB. South Dakota's rural physician shortage: how might it be addressed? South 

Dakota medicine : the journal of the South Dakota State Medical Association. Apr 

2008;61(4):125, 127.

112. Shortage of general surgeons coming? OR Manager. 2008;24(6).

113. Whitcomb ME. The challenge of providing doctors for rural America. Academic 

medicine : journal of the Association of American Medical Colleges. Aug 

2005;80(8):715-716.

114. Philipp DL, Wright DL. Recruiting healthcare professionals to rural areas. Radiology 

management. Nov-Dec 2005;27(6):44-50.

115. Thomas CR, Holzer CE, 3rd. The continuing shortage of child and adolescent 

psychiatrists. Journal of the American Academy of Child and Adolescent Psychiatry. Sep 

2006;45(9):1023-1031.

116. Fraher EP, Knapton A, Sheldon GF, Meyer A, Ricketts TC. Projecting surgeon supply 

using a dynamic model. Annals of surgery. May 2013;257(5):867-872.

117. Gunn J. Minnesota's rural health workforce shortages. Minnesota medicine. Dec 

2013;96(12):41-45.

118. Weldon T. Physician shortages and the medically underserved. Trends in America; 

Variation: Trends in America.: Council of State Governments; 2008: 

http://www.csg.org/knowledgecenter/docs/TIA_PhysicianShortage_Final_screen.pdf

119. US Department of Health and Human Services, Health Resources and Services

Administration. Lists of Designated Primary Medical Care, Mental Health, and Dental 

Health Professional Shortage Areas Federal register. 2015;80(126):37637-37638.

120. Wade ME, Brokaw JJ, Zollinger TW, et al. Influence of hometown on family physicians' 

choice to practice in rural settings. Family medicine. Apr 2007;39(4):248-254.

121. Grobler L, Marais BJ, Mabunda SA, Marindi PN, Reuter H, Volmink J. Interventions for 

increasing the proportion of health professionals practising in rural and other underserved 

areas. The Cochrane database of systematic reviews. 2009(1):Cd005314.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

99

122. Crouse BJ, Munson RL. The effect of the physician J-1 visa waiver on rural Wisconsin. 

WMJ : official publication of the State Medical Society of Wisconsin. Oct 

2006;105(7):16-20.

123. Wilson NW, Couper ID, De Vries E, Reid S, Fish T, Marais BJ. A critical review of 

interventions to redress the inequitable distribution of healthcare professionals to rural 

and remote areas. Rural and remote health. Apr-Jun 2009;9(2):1060.

124. Lee DM, Nichols T. Physician recruitment and retention in rural and underserved areas. 

International journal of health care quality assurance. 2014;27(7):642-652.

125. Mbemba G, Gagnon MP, Pare G, Cote J. Interventions for supporting nurse retention in 

rural and remote areas: an umbrella review. Human resources for health. 2013;11:44.

126. Buykx P, Humphreys J, Wakerman J, Pashen D. Systematic review of effective retention 

incentives for health workers in rural and remote areas: towards evidence-based policy. 

The Australian journal of rural health. Jun 2010;18(3):102-109.

127. Rabinowitz HK, Diamond JJ, Markham FW, Wortman JR. Medical school programs to 

increase the rural physician supply: a systematic review and projected impact of 

widespread replication. Academic medicine : journal of the Association of American 

Medical Colleges. Mar 2008;83(3):235-243.

128. US Department of Health and Human Services. The Affordable Care Act- What it Means 

for Rural America. 2013; 

http://www.hhs.gov/healthcare/facts/factsheets/2013/09/rural09202013.html.

129. Patterson DG, Keppel G, Skillman SM, Berry C, Daniel C, Doescher MP. Recruitment of 

Non-U.S. Citizen Physicians to Rural and Underserved Areas through Conrad State 30 J1 Visa Waiver Programs. Final Report #148, April 2015. Seattle, WA: WWAMI Rural 

Health Research Center, University of Washington;2015.

130. Wendling AL, Phillips J, Short W, Fahey C, Mavis B. Thirty Years Training Rural 

Physicians: Outcomes From the Michigan State University College of Human Medicine 

Rural Physician Program. practice. 2015;17:18.

131. US Department of Health and Human Services, Health Resources and Services 

Administration. U.S. Health Workforce Projections. 

http://bhpr.hrsa.gov/healthworkforce/supplydemand/usworkforce/projections/index.html.

132. Peterson LE, Bazemore A, Bragg EJ, Xierali I, Warshaw GA. Rural-urban distribution of 

the U.S. Geriatrics physician workforce. Journal of the American Geriatrics Society. Apr 

2011;59(4):699-703.

133. Lynge DC, Larson EH, Thompson M, Rosenblatt RA, Hart LG. A longitudinal analysis 

of the general surgery workforce in the United States, 1981-2005. Arch. Surg. Apr 

2008;143(4):345-350.

134. Toner JA, Ferguson KD, Sokal RD. Continuing interprofessional education in geriatrics 

and gerontology in medically underserved areas. Journal of Continuing Education in the 

Health Professions. 2009;29(3):157-160.

135. Suter E, Deutschlander S, Mickelson G, et al. Can interprofessional collaboration provide 

health human resources solutions? A knowledge synthesis. Journal of interprofessional 

care. Jul 2012;26(4):261-268.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

100

136. Tumosa N, Horvath KJ, Huh T, et al. Health care workforce development in rural 

america: when geriatrics expertise is 100 miles away. Gerontology & geriatrics 

education. 2012;33(2):133-151.

137. Holmes GM. Increasing physician supply in medically underserved areas. Labour Econ. 

Oct 2005;12(5):697-725.

138. Pathman DE, Fryer GE, Jr., Phillips RL, Smucny J, Miyoshi T, Green LA. National 

Health Service Corps staffing and the growth of the local rural non-NHSC primary care 

physician workforce. The Journal of rural health : official journal of the American Rural 

Health Association and the National Rural Health Care Association. Fall 

2006;22(4):285-293.

139. Alfero C. FORWARD NM Pathways to Health Careers. 2014; 

https://www.raconline.org/success/project-examples/724. Accessed 09/02/2015.

140. Primm D. Tennessee Rural Partnership. 2014; https://www.raconline.org/success/projectexamples/777. Accessed 09/02/2015.

141. Stempniak M. The hiring headache. Rural hospitals band together to lure physicians. 

Hospitals & health networks / AHA. Dec 2012;86(12):18-19.

142. Schmitz DF, Baker E, Nukui A, Epperly T. Idaho rural family physician workforce study: 

the Community Apgar Questionnaire. Rural and remote health. 2011;11(3):1769.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

101

APPENDIX 1. SEARCH STRATEGY 

KQ1 – RURAL HEALTH PROVIDER NEEDS

DATABASE SEARCHED & TIME PERIOD COVERED:

 PubMed – 1/1/2005-2/11/2015 

LANGUAGE:

 English

SEARCH STRATEGY:

Rural*[tiab] OR agricultur*[tiab] OR wilderness* OR frontier* OR (native AND reservation*) OR 

farmer OR farmers OR farming OR farm OR farms OR nonurban* OR "non-urban" OR remote*[tiab] 

OR outback* OR isolated[tiab] OR "small town" OR "small towns" OR village*[tiab] OR settlement* OR 

"Rural Population"[Mesh] OR "Rural Nursing"[Mesh] OR "Rural Health Services"[Mesh] OR "Rural 

Health"[Mesh] OR "Hospitals, Rural"[Mesh]

AND

"Health Personnel"[Mesh] OR physician*[tiab] OR nurses[tiab] OR nursing[tiab] OR hospitalist*[tiab] 

OR hospital staff*[tiab] OR healthcare professional*[tiab] OR health care professional*[tiab] OR 

doctor[tiab] OR doctors[tiab] OR health manpower[mh] OR manpower[tiab] OR workforce OR medical 

staff, hospital

AND

need[tiab] OR needs[tiab] OR needed[tiab] OR needing[tiab] OR supply[tiab] OR demand[tiab] OR 

"supply and distribution" [Subheading] 

AND

predict* OR projected OR future OR trend* 

DATABASE SEARCHED & TIME PERIOD COVERED:

 CIN/AHL – 1/1/2005-2/11/2015 

LANGUAGE:

 English

SEARCH STRATEGY:

Rural*[tiab] OR agricultur*[tiab] OR wilderness* OR frontier* OR (native AND reservation*) OR farmer 

OR farmers OR farming OR farm OR farms OR nonurban* OR "non-urban" OR remote*[tiab] OR 

outback* OR isolated[tiab] OR "small town" OR "small towns" OR village*[tiab] OR settlement* OR 

"Rural Population"[Mesh] OR "Rural Nursing"[Mesh] OR "Rural Health Services"[Mesh] OR "Rural 

Health"[Mesh] OR "Hospitals, Rural"[Mesh] 

AND

"Health Personnel" OR physician* OR nurses OR nursing OR hospitalist* OR hospital staff* OR 

healthcare professional* OR health care professional* OR doctor OR doctors OR manpower 

AND

need OR needs OR needed OR needing OR supply OR demand 

Narrow by SubjectGeographic: - usa 

KQ2 – DECISION FACTORS

DATABASE SEARCHED & TIME PERIOD COVERED:
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

102

 PubMed – 1/1/2005-2/13/2015 

LANGUAGE:

 English

SEARCH STRATEGY:

Rural*[tiab] OR agricultur*[tiab] OR wilderness* OR frontier* OR (native AND reservation*) OR 

farmer OR farmers OR farming OR farm OR farms OR nonurban* OR "non-urban" OR remote*[tiab] 

OR outback* OR isolated[tiab] OR "small town" OR "small towns" OR village*[tiab] OR settlement* OR 

"Rural Population"[Mesh] OR "Rural Nursing"[Mesh] OR "Rural Health Services"[Mesh] OR "Rural 

Health"[Mesh] OR "Hospitals, Rural"[Mesh]

AND

"Health Personnel"[Mesh] OR physician*[tiab] OR nurses[tiab] OR nursing[tiab] OR hospitalist*[tiab] 

OR hospital staff*[tiab] OR healthcare professional*[tiab] OR health care professional*[tiab] OR 

doctor[tiab] OR doctors[tiab] OR health manpower[mh] OR manpower[tiab] OR workforce OR medical 

staff, hospital

AND

choice* OR choos* OR decision* OR decid* 

AND

incentive* OR attract* OR pecuniary OR non-pecuniary OR income OR monetary OR economic* OR 

financial OR opportunit*) OR influen* 

DATABASE SEARCHED & TIME PERIOD COVERED:

 CIN/AHL – 1/1/2005-2/13/2015 

LANGUAGE:

 English

SEARCH STRATEGY #1: 

Rural*[tiab] OR agricultur*[tiab] OR wilderness* OR frontier* OR (native AND reservation*) OR farmer 

OR farmers OR farming OR farm OR farms OR nonurban* OR "non-urban" OR remote*[tiab] OR 

outback* OR isolated[tiab] OR "small town" OR "small towns" OR village*[tiab] OR settlement* OR 

"Rural Population"[Mesh] OR "Rural Nursing"[Mesh] OR "Rural Health Services"[Mesh] OR "Rural 

Health"[Mesh] OR "Hospitals, Rural"[Mesh] 

AND

"Health Personnel" OR physician* OR nurses OR nursing OR hospitalist* OR hospital staff* OR 

healthcare professional* OR health care professional* OR doctor OR doctors OR manpower 

AND

choice* OR choos* OR decision* OR decid* 

SEARCH STRATEGY #2:

Rural*[tiab] OR agricultur*[tiab] OR wilderness* OR frontier* OR (native AND reservation*) OR farmer 

OR farmers OR farming OR farm OR farms OR nonurban* OR "non-urban" OR remote*[tiab] OR 

outback* OR isolated[tiab] OR "small town" OR "small towns" OR village*[tiab] OR settlement* OR 

"Rural Population"[Mesh] OR "Rural Nursing"[Mesh] OR "Rural Health Services"[Mesh] OR "Rural 

Health"[Mesh] OR "Hospitals, Rural"[Mesh] 

AND

"Health Personnel" OR physician* OR nurses OR nursing OR hospitalist* OR hospital staff* OR 

healthcare professional* OR health care professional* OR doctor OR doctors OR manpower 

AND
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

103

incentive* OR attract* OR pecuniary OR non-pecuniary OR income OR monetary OR economic* OR 

financial OR opportunit* OR incentive* 

KQ3 & 4 – RECRUITMENT & RETENTION

DATABASE SEARCHED & TIME PERIOD COVERED:

 PubMed – 1/1/2005-2/23/2015 

LANGUAGE:

 English

SEARCH STRATEGY:

Rural*[tiab] OR agricultur*[tiab] OR wilderness* OR frontier* OR (native AND reservation*) OR 

farmer OR farmers OR farming OR farm OR farms OR nonurban* OR "non-urban" OR remote*[tiab] 

OR outback* OR isolated[tiab] OR "small town" OR "small towns" OR village*[tiab] OR settlement* OR 

"Rural Population"[Mesh] OR "Rural Nursing"[Mesh] OR "Rural Health Services"[Mesh] OR "Rural 

Health"[Mesh] OR "Hospitals, Rural"[Mesh]

AND

"Health Personnel"[Mesh] OR physician*[tiab] OR nurses[tiab] OR nursing[tiab] OR hospitalist*[tiab] 

OR hospital staff*[tiab] OR healthcare professional*[tiab] OR health care professional*[tiab] OR 

doctor[tiab] OR doctors[tiab] OR health manpower[mh] OR manpower[tiab] OR workforce OR medical 

staff, hospital

AND

"Personnel Selection"[Mesh] OR recruit* OR retention OR turnover OR turn over* OR burnout OR 

burn* out 

AND

interven* OR increas* OR program OR programs[tiab] OR programme*[tiab] OR project[tiab] OR 

projects[tiab] OR telehealth OR telemedicine OR ehealth

DATABASE SEARCHED & TIME PERIOD COVERED:

 CIN/AHL – 1/1/2005-2/23/2015 

LANGUAGE:

 English

SEARCH STRATEGY:

Rural*[tiab] OR agricultur*[tiab] OR wilderness* OR frontier* OR (native AND reservation*) OR farmer 

OR farmers OR farming OR farm OR farms OR nonurban* OR "non-urban" OR remote*[tiab] OR 

outback* OR isolated[tiab] OR "small town" OR "small towns" OR village*[tiab] OR settlement* OR 

"Rural Population"[Mesh] OR "Rural Nursing"[Mesh] OR "Rural Health Services"[Mesh] OR "Rural 

Health"[Mesh] OR "Hospitals, Rural"[Mesh] 

AND

"Health Personnel" OR physician* OR nurses OR nursing OR hospitalist* OR hospital staff* OR 

healthcare professional* OR health care professional* OR doctor OR doctors OR manpower 

AND

recruit* OR retention OR retain* OR personnel selection OR turnover OR turn over* OR burnout OR 

burn* out 
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

104

KQ5 – EDUCATION

DATABASE SEARCHED & TIME PERIOD COVERED:

PUBMED - 1/1/2005-2/24/2015 

LANGUAGE:

 English

SEARCH STRATEGY:

Rural*[tiab] OR agricultur*[tiab] OR wilderness* OR frontier* OR (native AND reservation*) OR farmer 

OR farmers OR farming OR farm OR farms OR nonurban* OR "non-urban" OR remote*[tiab] OR 

outback* OR isolated[tiab] OR "small town" OR "small towns" OR village*[tiab] OR settlement* OR 

"Rural Population"[Mesh] OR "Rural Nursing"[Mesh] OR "Rural Health Services"[Mesh] OR "Rural 

Health"[Mesh] OR "Hospitals, Rural"[Mesh]

AND

"Schools, Health Occupations"[Mesh]) OR "Students, Health Occupations"[Mesh] OR internship and 

residency[mh] OR graduate[tiab] OR post-graduate[tiab] OR postgraduate[tiab] OR college[tiab] 

AND

training OR train[ti] OR educat*[tiab] OR medical education OR education, professional

AND

program OR programs[tiab] OR programme*[tiab] OR project[tiab] OR projects[tiab] OR effort[tiab] OR 

efforts[tiab] OR evaluat*[tiab] OR improv*[tiab] OR success*[tiab] OR efficacy OR survey* OR 

questionnaire* 

DATABASE SEARCHED & TIME PERIOD COVERED:

 CIN/AHL – 1/1/2005-2/24/2015 

LANGUAGE:

 English

SEARCH STRATEGY:

Rural*[tiab] OR agricultur*[tiab] OR wilderness* OR frontier* OR (native AND reservation*) OR farmer 

OR farmers OR farming OR farm OR farms OR nonurban* OR "non-urban" OR remote*[tiab] OR 

outback* OR isolated[tiab] OR "small town" OR "small towns" OR village*[tiab] OR settlement* OR 

"Rural Population"[Mesh] OR "Rural Nursing"[Mesh] OR "Rural Health Services"[Mesh] OR "Rural 

Health"[Mesh] OR "Hospitals, Rural"[Mesh]) 

AND

"Health Personnel" OR physician* OR nurses OR nursing OR hospitalist* OR hospital staff* OR 

healthcare professional* OR health care professional* OR doctor OR doctors OR manpower 

AND

 training OR train OR trained OR educat* OR graduat* OR post-graduate OR postgraduate OR college

DATABASE SEARCHED & TIME PERIOD COVERED:

GREY LITERATURE REPORT – 1/1/2010-1/16/2015 

NUMBER OF RESULTS: 76

SEARCH STRATEGY:

Rural
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

105

APPENDIX 2. LIST OF EXCLUDED STUDIES

This appendix lists the publications assessed as full text and not meeting inclusion criteria.

BACKGROUND

A large number of publications did not meet inclusion criteria for the review but were retained as 

background information. Publications either reported more information on an included study 

(multiple publication), potentially contained sources of studies potentially meeting inclusion 

criteria, or were used in the introduction and discussion. 

[no author] Challenges, solutions & opportunities : affordable housing, workforce training, recruitment & retention 

of health care professionals. Northern Arizona University, the W.A. Franke College of Business; 2007: 

http://www.franke.nau.edu/RPI/projects/ARPF2007FinalReport.pdf

[no author] Shortage of general surgeons coming? OR Manager. 2008;24(6).

[no author] Young physicians not keen on rural areas. Managed care (Langhorne, Pa.). 2012;21(10):16.

Aseltine RH, Jr., Katz MC, Geragosian AH. Connecticut 2009 Primary Care Survey: physician satisfaction, 

physician supply and patient access to medical care. Connecticut medicine. 2010;74(5):281-291.

Avery DM, Jr., Wheat JR, Leeper JD, McKnight JT, Ballard BG, Chen J. Admission factors predicting family 

medicine specialty choice: a literature review and exploratory study among students in the Rural Medical Scholars 

Program. The Journal of rural health : official journal of the American Rural Health Association and the National 

Rural Health Care Association. 2012;28(2):128-136.

Baker E, Schmitz D, Epperly T, Nukui A, Miller CM. Rural Idaho Family Physicians' Scope of Practice. Journal of 

Rural Health. 2010;26(1):85-89.

Ballance D, Kornegay D, Evans P. Factors that influence physicians to practice in rural locations: a review and 

commentary. The Journal of rural health : official journal of the American Rural Health Association and the 

National Rural Health Care Association. 2009;25(3):276-281.

Balshem H, Helfand M, Schunemann HJ, et al. GRADE guidelines: 3. Rating the quality of evidence. Journal of 

clinical epidemiology. 2011;64(4):401-406.

Barnighausen T, Bloom DE. Financial incentives for return of service in underserved areas: a systematic review. 

BMC health services research. 2009;9:86.

Barrett FA, Lipsky MS, Lutfiyya MN. The impact of rural training experiences on medical students: a critical 

review. Academic medicine : journal of the Association of American Medical Colleges. 2011;86(2):259-263.

Bowman RC. Measuring primary care: the standard primary care year. Rural Remote Health. 2008;8(3).

Bridgham RG. Final report : HB 1615, (Chapter 367:3, Laws of 2008), establishing a Commission to Recommend 

Policies and Programs to Increase the Number of New Hampshire Individuals in Health Professions Servicing New 

Hampshire's Rural and Underserved Areas with a Focus on Primary Care. Concord, N.H.: N.H. General Court; 

2009.

Broughan TA. SAGES 2007 rural surgery panel. Surgical endoscopy. 2008;22(7):1579-1581.

Carlton EL, Simmons LA. Health decision-making among rural women: physician access and prescription 

adherence. Rural Remote Health. 2011;11(1).

Chen C, Xierali I, Piwnica-Worms K, Phillips R. The redistribution of graduate medical education positions in 2005 

failed to boost primary care or rural training. Health affairs (Project Hope). 2013;32(1):102-110.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

106

Chipp C, Dewane S, Brems C, Johnson ME, Warner TD, Roberts LW. "If only someone had told me...": lessons 

from rural providers. The Journal of rural health : official journal of the American Rural Health Association and the 

National Rural Health Care Association. 2011;27(1):122-130.

Cogbill TH, Cofer JB, Jarman BT. Contemporary issues in rural surgery. Current problems in surgery. 

2012;49(5):263-318.

Collier D. Rural Michigan physician retention study reveals motivators. Michigan medicine. 2010;109(5):21.

Cook AF, Hoas H. Hide and seek: The elusive rural psychiatrist. Academic Psychiatry. 2007;31(6):419-422.

Corbett CD. Recruitment and retention of physicians in rural North Dakota : testing for congruency between current 

policies and physician motivation. 2012:ii, 37 leaves ; 29 cm. Dissertation: Paper (M.S.)--Minnesota State 

University Moorhead, 2012.

Crouse BJ, Munson RL. The effect of the physician J-1 visa waiver on rural Wisconsin. WMJ : official publication 

of the State Medical Society of Wisconsin. 2006;105(7):16-20.

Curran V, Rourke L, Snow P. A framework for enhancing continuing medical education for rural physicians: A 

summary of the literature. Medical teacher. 2010;32(11):e501-508.

Deveney K, Deatherage M, Oehling D, Hunter J. Association between dedicated rural training year and the 

likelihood of becoming a general surgeon in a small town. JAMA surgery. 2013;148(9):817-821.

Dill MJ, Salsberg ES. The Complexities of Physician Supply and Demand: Projections Through 2025. November 

2008. Washington, DC: Association of American Medical Colleges, Center for Workforce Studies.;2008.

Doescher MP, Andrilla CH, Skillman SM, Morgan P, Kaplan L. The contribution of physicians, physician 

assistants, and nurse practitioners toward rural primary care: findings from a 13-state survey. Medical care. 

2014;52(6):549-556.

Dolea C, Stormont L, Braichet JM. Evaluated strategies to increase attraction and retention of health workers in 

remote and rural areas. Bull World Health Organ. 2010;88(5):379-385.

Dornan T, Littlewood S, Margolis SA, Scherpbier A, Spencer J, Ypinazar V. How can experience in clinical and 

community settings contribute to early medical education? A BEME systematic review. Med. Teach. 2006;28(1):3-

18.

Dorsey ER, Nicholson S, Frist WH. Commentary: improving the supply and distribution of primary care physicians. 

Academic Medicine: Journal of the Association of American Medical Colleges. 2011;86(5):541-543.

Doty B, Andres M, Zuckerman R, Borgstrom D. Use of Locum Tenens Surgeons to Provide Surgical Care in Small 

Rural Hospitals. World J. Surg. 2009;33(2):228-232.

Doty B, Zuckerman R, Finlayson S, Jenkins P, Rieb N, Heneghan S. General surgery at rural hospitals: a national 

survey of rural hospital administrators. Surgery. 2008;143(5):599-606.

Edwards JB, Wilson JL, Behringer BA, et al. Practice locations of graduates of family physician residency and nurse 

practitioner programs: considerations within the context of institutional culture and curricular innovation through 

Titles VII and VIII. The Journal of rural health: official journal of the American Rural Health Association and the 

National Rural Health Care Association. 2006;22(1):69-77.

Escarce JJ, Kapur K. Do patients bypass rural hospitals? Determinants of inpatient hospital choice in rural 

California. J. Health Care Poor Underserved. 2009;20(3):625-644.

Everitt-Deering P. The adoption of information and communication technologies by rural general practitioners a 

socio technical analysis. [Internet Resource; Computer File; Archival Material]. 2008; http://eprints.vu.edu.au/1412.

Filipova AA. Factors influencing the satisfaction of rural physician assistants: a cross-sectional study. Journal of 

allied health. 2014;43(1):22-31.

Fordyce MA, Chen FM, Doescher MP, Hart LG. 2005 physician supply and distribution in rural areas of the United 

States [Internet]. Seattle, WA: Rural Health Research and Policy Centers;2007.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

107

Fournier GM, Henderson C. Incentives and physician specialty choice: a case study of Florida's Program in Medical 

Sciences. Inquiry : a journal of medical care organization, provision and financing. 2005;42(2):160-170.

Fraher EP, Knapton A, Sheldon GF, Meyer A, Ricketts TC. Projecting surgeon supply using a dynamic model. 

Annals of surgery. 2013;257(5):867-872.

Gagnon MP, Pollender H, Trepanier A, Duplaa E, Ly BA. Supporting health professionals through information and 

communication technologies: a systematic review of the effects of information and communication technologies on 

recruitment and retention. Telemedicine journal and e-health : the official journal of the American Telemedicine 

Association. 2011;17(4):269-274.

Garrison-Jakel J. Patching the rural workforce pipeline--why don't we do more? The Journal of rural health : official 

journal of the American Rural Health Association and the National Rural Health Care Association. 2011;27(2):239-

240.

Getson DS. Rural practice realities. The West Virginia medical journal. 2013;109(4):34-37.

Grobler L, Marais BJ, Mabunda SA, Marindi PN, Reuter H, Volmink J. Interventions for increasing the proportion 

of health professionals practising in rural and other underserved areas. The Cochrane database of systematic 

reviews. 2009(1):Cd005314.

Haggerty TS, Fields SA, Selby-Nelson EM, Foley KP, Shrader CD. Physician Wellness in Rural America: A 

Review. International journal of psychiatry in medicine. 2013;46(3):303-313.

Halaas GW. The Rural Physician Associate Program: successful outcomes in primary care and rural practice. Rural 

and remote health. 2005;5(2):453.

Henry LR, Hooker RS, Yates KL. The role of physician assistants in rural health care: a systematic review of the 

literature. The Journal of rural health : official journal of the American Rural Health Association and the National 

Rural Health Care Association. 2011;27(2):220-229.

Hilty DM, Yellowlees PM, Cobb HC, Bourgeois JA, Neufeld JD, Nesbitt TS. Models of telepsychiatric 

consultation-liaison service to rural primary care. Psychosomatics. 2006;47(2):152-157.

Hirsh D, Walters L, Poncelet AN. Better learning, better doctors, better delivery system: possibilities from a case 

study of longitudinal integrated clerkships. Medical teacher. 2012;34(7):548-554.

Holmes GM. Increasing physician supply in medically underserved areas. Labour Econ. 2005;12(5):697-725.

Huff C. Where are the specialists? Hospitals & health networks / AHA. 2011;85(12):26-28, 31, 21.

Huff C. Please doc, stay. Give docs a reason to set up shop. Hospitals & health networks / AHA. 2012;86(2):20.

Huff C. Done recruiting? Start retaining. Trustee : the journal for hospital governing boards. 2014;67(1):8-12, 11.

Kochar MS. The J-1 visa waiver program for rural Wisconsin. WMJ : official publication of the State Medical 

Society of Wisconsin. 2006;105(7):13.

Larson EH, Hart LG. Growth and change in the physician assistant workforce in the United States, 1967-2000. 

Journal of allied health. 2007;36(3):121-130.

Lee DM, Nichols T. Physician recruitment and retention in rural and underserved areas. International journal of 

health care quality assurance. 2014;27(7):642-652.

Lindsay S. Gender differences in rural and urban practice location among mid-level health care providers. The 

Journal of rural health : official journal of the American Rural Health Association and the National Rural Health 

Care Association. 2007;23(1):72-76.

Lynch S. Hospice and palliative care access issues in rural areas. The American journal of hospice & palliative care. 

2013;30(2):172-177.

Lynge DC. Rural general surgeons: manpower and demographics. Surgical endoscopy. 2008;22(7):1593-1594.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

108

Lynge DC, Larson EH, Thompson M, Rosenblatt RA, Hart LG. A longitudinal analysis of the general surgery 

workforce in the United States, 1981-2005. Arch. Surg. 2008;143(4):345-350.

MacDowell M, Glasser M, Fitts M, Fratzke M, Peters K. Perspectives on rural health workforce issues: IllinoisArkansas comparison. The Journal of rural health : official journal of the American Rural Health Association and 

the National Rural Health Care Association. 2009;25(2):135-140.

MacDowell M, Glasser M, Fitts M, Nielsen K, Hunsaker M. A national view of rural health workforce issues in the 

USA. Rural and remote health. 2010;10(3):1531.

MacDowell M, Glasser M, Hunsaker M. A decade of rural physician workforce outcomes for the Rockford Rural 

Medical Education (RMED) Program, University of Illinois. Academic medicine : journal of the Association of 

American Medical Colleges. 2013;88(12):1941-1947.

Maley M, Worley P, Dent J. Using rural and remote settings in the undergraduate medical curriculum: AMEE Guide 

No. 47. Med. Teach. 2009;31(11):969-983.

Mareck DG. Federal and state initiatives to recruit physicians to rural areas. The virtual mentor : VM. 

2011;13(5):304-309.

Mbemba G, Gagnon MP, Pare G, Cote J. Interventions for supporting nurse retention in rural and remote areas: an 

umbrella review. Human resources for health. 2013;11:44.

Meyer D. Technology, job satisfaction, and retention: rural mental health practitioners. The Journal of rural health : 

official journal of the American Rural Health Association and the National Rural Health Care Association. 

2006;22(2):158-163.

Mullan F, Frehywot S, Jolley LJ. Aging, primary care, and self-sufficiency: health care workforce challenges ahead. 

The Journal of law, medicine & ethics : a journal of the American Society of Law, Medicine & Ethics. 

2008;36(4):703-708, 608.

Murphy KL. Alaska health care workforce shortages : impact of state legislation. 2011:ix, 74 leaves ; 28 cm. 

Dissertation: Thesis (M.Public Health)--University of Alaska Anchorage, 2011.

Nakayama DK, Hughes TG. Issues That Face Rural Surgery in the United States. Journal of the American College 

of Surgeons. 2014;219(4):814-818.

Nance ML, Carr BG, Branas CC. Access to pediatric trauma care in the United States. Archives of pediatrics & 

adolescent medicine. 2009;163(6):512-518.

Ortiz J, Bushy A, Zhou Y, Zhang H. Accountable care organizations: benefits and barriers as perceived by Rural 

Health Clinic management. Rural and remote health. 2013;13(2):2417.

Palmer RT. Exploring online community among rural medical education students: A case study. Dissertation 

Abstracts International Section A: Humanities and Social Sciences. 2014;75(1-A(E)).

Pathman DE. What outcomes should we expect from programs that pay physicians'training expenses in exchange for 

service? North Carolina medical journal. 2006;67(1):77-82.

Pathman DE, Fryer GE, Jr., Phillips RL, Smucny J, Miyoshi T, Green LA. National Health Service Corps staffing 

and the growth of the local rural non-NHSC primary care physician workforce. The Journal of rural health : official 

journal of the American Rural Health Association and the National Rural Health Care Association. 2006;22(4):285-

293.

Patterson DG, Keppel G, Skillman SM, Berry C, Daniel C, Doescher MP. Recruitment of Non-U.S. Citizen 

Physicians to Rural and Underserved Areas through Conrad State 30 J-1 Visa Waiver Programs. Final Report #148, 

April 2015. Seattle, WA: WWAMI Rural Health Research Center, University of Washington;2015.

Patterson DG, Longenecker R, Schmitz D, Skillman SM, Doescher MP. January 2011 Policy Brief: Training 

Physicians for Rural Practice: Capitalizing on Local Expertise to Strengthen Rural Primary Care. 2011; 

https://www.raconline.org/rtt/pdf/policybrief_jan11.pdf. Accessed 8/20/2015.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

109

Patterson DG, Longenecker R, Schmitz D, et al. January 2012 Policy Brief: Rural Residency Training for Family 

Medicine Physicians: Graduate Early-Career Outcomes. 2012; https://www.raconline.org/rtt/pdf/rural-familymedicine-training-early-career-outcomes-2012.pdf. Accessed 8/20/2015.

Peterson LE, Bazemore A, Bragg EJ, Xierali I, Warshaw GA. Rural-urban distribution of the U.S. Geriatrics 

physician workforce. Journal of the American Geriatrics Society. 2011;59(4):699-703.

Philipp DL, Wright DL. Recruiting healthcare professionals to rural areas. Radiology management. 2005;27(6):44-

50.

Price J. The National Health Service Corps--a critical component of provider recruitment in North Carolina's rural 

and underserved communities. North Carolina medical journal. 2010;71(3):251.

Quarry WA. A research study outlining the key issues and strategies needed to improve recruitment and retention 

among primary care physicians in rural communities. 2012.

Rabinowitz HK. AM last page. Truths about the rural physician supply. Academic medicine : journal of the 

Association of American Medical Colleges. 2011;86(2):272.

Rabinowitz HK, Diamond JJ, Markham FW, Rabinowitz C. Long-term retention of graduates from a program to 

increase the supply of rural family physicians. Academic medicine : journal of the Association of American Medical 

Colleges. 2005;80(8):728-732.

Rabinowitz HK, Diamond JJ, Markham FW, Santana AJ. Increasing the supply of women physicians in rural areas: 

outcomes of a medical school rural program. Journal of the American Board of Family Medicine : JABFM. 

2011;24(6):740-744.

Rabinowitz HK, Diamond JJ, Markham FW, Santana AJ. The relationship between matriculating medical students' 

planned specialties and eventual rural practice outcomes. Academic medicine : journal of the Association of 

American Medical Colleges. 2012;87(8):1086-1090.

Rabinowitz HK, Diamond JJ, Markham FW, Wortman JR. Medical school programs to increase the rural physician 

supply: a systematic review and projected impact of widespread replication. Academic medicine : journal of the 

Association of American Medical Colleges. 2008;83(3):235-243.

Ricketts TC. Workforce issues in rural areas: A focus on policy equity. Am. J. Public Health. 2005;95(1):42-48.

Roh CY, Moon MJ. Nearby, but not wanted? The bypassing of rural hospitals and policy implications for rural 

health care systems. Policy Stud. J. 2005;33(3):377-394.

Rosenblatt RA, Andrilla CH, Curtin T, Hart LG. Shortages of medical personnel at community health centers:

implications for planned expansion. Jama. 2006;295(9):1042-1049.

Schmitz DF, Baker E, Nukui A, Epperly T. Idaho rural family physician workforce study: the Community Apgar 

Questionnaire. Rural and remote health. 2011;11(3):1769.

Seligson RW, Highsmith PP. North Carolina Medical Society Foundation's Community Practitioner Program. North 

Carolina medical journal. 2006;67(1):83-85.

Staton FS, Bhosle MJ, Camacho FT, Feldman SR, Balkrishnan R. How PAs improve access to care for the 

underserved. JAAPA : official journal of the American Academy of Physician Assistants. 2007;20(6):32, 34, 36 

passim.

Stempniak M. The hiring headache. Rural hospitals band together to lure physicians. Hospitals & health networks / 

AHA. 2012;86(12):18-19.

Thompson MJ, Hagopian A, Fordyce M, Hart LG. Do international medical graduates (IMGs) "fill the gap" in rural 

primary care in the United States? A national study. The Journal of rural health : official journal of the American 

Rural Health Association and the National Rural Health Care Association. 2009;25(2):124-134.

Thompson MJ, Lynge DC, Larson EH, Tachawachira P, Hart LG. Characterizing the general surgery workforce in 

rural America. Archives of surgery (Chicago, Ill. : 1960). 2005;140(1):74-79.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

110

Toner JA, Ferguson KD, Sokal RD. Continuing interprofessional education in geriatrics and gerontology in 

medically underserved areas. Journal of Continuing Education in the Health Professions. 2009;29(3):157-160.

Traverso G, McMahon GT. Residency training and international medical graduates: Coming to America no more. 

JAMA: Journal of the American Medical Association. 2012;308(21):2193-2194.

Tumosa N, Horvath KJ, Huh T, et al. Health care workforce development in rural america: when geriatrics expertise 

is 100 miles away. Gerontology & geriatrics education. 2012;33(2):133-151.

Van Vleet A, Paradise J. Issue Brief: Tapping Nurse Practitioners to Meet Rising Demand for Primary Care. The 

Henry J. Kaiser Family Foundation; 2015.

Vogt HB. South Dakota's rural physician shortage: how might it be addressed? South Dakota medicine : the journal 

of the South Dakota State Medical Association. 2008;61(4):125, 127.

Wade ME, Brokaw JJ, Zollinger TW, et al. Influence of hometown on family physicians' choice to practice in rural 

settings. Family medicine. 2007;39(4):248-254.

Walters L, Greenhill J, Richards J, et al. Outcomes of longitudinal integrated clinical placements for students, 

clinicians and society. Medical Education. 2012;46(11):1028-1041.

Weeks WB, Wallace AE, West AN, Heady HR, Hawthorne K. Research on rural veterans: An analysis of the 

literature. Journal of Rural Health. 2008;24(4):337-344.

Weldon T. Physician shortages and the medically underserved. Trends in America; Variation: Trends in America.: 

Council of State Governments; 2008: 

http://www.csg.org/knowledgecenter/docs/TIA_PhysicianShortage_Final_screen.pdf

Wheat JR, Higginbotham JC, Yu J, Leeper JD. Physicians for rural America: the role of institutional commitment 

within academic medical centers. The Journal of rural health : official journal of the American Rural Health 

Association and the National Rural Health Care Association. 2005;21(3):221-227.

Whitcomb ME. The challenge of providing doctors for rural America. Academic medicine : journal of the 

Association of American Medical Colleges. 2005;80(8):715-716.

Wilson NW, Couper ID, De Vries E, Reid S, Fish T, Marais BJ. A critical review of interventions to redress the 

inequitable distribution of healthcare professionals to rural and remote areas. Rural and remote health. 

2009;9(2):1060.

Xierali IM, Sweeney SA, Phillips RL, Jr., Bazemore AW, Petterson SM. Increasing graduate medical education 

(GME) in critical access hospitals (CAH) could enhance physician recruitment and retention in rural America. 

Journal of the American Board of Family Medicine : JABFM. 2012;25(1):7-8.

Zuckerman R, Doty B, Gold M, et al. General surgery programs in small rural New York state hospitals: A pilot 

survey of hospital administrators. Journal of Rural Health. 2006;22(4):339-342.

Zurn P, Vujicic M, Lemiere C, et al. A technical framework for costing health workforce retention schemes in 

remote and rural areas. Human resources for health. 2011;9:8.

DESIGN

Publications excluded based on the study design, (eg, the publication did not report on empirical 

data). 

[no author] Shortages of rural generalist physicians may be due to poor recruitment rather than retention problems. 

AHRQ Research Activities. 2005;293:19.

Barr P. Looking for an oasis: large sections of rural America continue to suffer from a drought of general surgeons. 

Mod Healthc. 2012;42(13):28-31.

Basco WT, Rimsza ME. Pediatrician workforce policy statement. Pediatrics. 2013;132(2):390-397.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

111

Behringer B, Friedell GH. Appalachia: where place matters in health. Preventing chronic disease. 2006;3(4):A113.

Bellinger N. Predictors of primary care physicians practicing in medically underserved and rural areas of Indiana. 

Variation: IUPUI theses and dissertations.2009: http://hdl.handle.net/1805/1954

Berens D. Recruitment or retention: J-1 visa lessons. WMJ : official publication of the State Medical Society of 

Wisconsin. 2006;105(7):11.

Brooks KD, Cieslak JE, Radcliffe PM, Sjogren K. Primary care in Minnesota: an academic health center's 

perspective. Minnesota medicine. 2008;91(5):40-44.

Calman NS, Hauser D, Forte G, Continelli T. New York State physicians: characteristics and distribution in health 

professional shortage areas. Journal of urban health : bulletin of the New York Academy of Medicine. 

2007;84(2):307-309.

Casaletto JJ, Wadman MC, Ankel FK, Bourne CL, Ghaemmaghami CA. Emergency Medicine Rural Rotations: A 

Program Director's Guide. Annals of emergency medicine. 2013;61(5):578-583.

Casey BR, Owens J, Gross DA, Dixon LM. Rural Kentucky's physician shortage: strategies for producing, 

recruiting, and retaining primary care providers within a medically underserved region. The Journal of the Kentucky 

Medical Association. 2005;103(10):505-513.

Chen F, Hart LG. Another look at rural health. Health affairs (Project Hope). 2006;25(2):570-571; author reply 571.

Chewning LH, Spade J. Rural hospitals and rural physicians: understanding the physician workforce challenges that 

affect rural communities and providers. North Carolina medical journal. 2007;68(3):191-193.

Click IA. Practice characteristics of graduates of east tennessee state university quillen college of medicine: Factors 

related to career choices in primary care. Dissertation Abstracts International Section A: Humanities and Social 

Sciences. 2014;74(9-A(E)).

Cofer JB, Burns RP. The developing crisis in the national general surgery workforce. Journal of the American 

College of Surgeons. 2008;206(5):790-795; discussion 795-797.

Cofer JB, Petros TJ, Burkholder HC, Clarke PC. General Surgery at Rural Tennessee Hospitals: A Survey of Rural 

Tennessee Hospital Administrators. Am. Surg. 2011;77(7):820-825.

Colgan R, Lewin L, Southall N, Williams M, Baquet C. University of Maryland School of Medicine increases 

medical student education in primary care. Maryland medicine : MM : a publication of MEDCHI, the Maryland 

State Medical Society. 2011;12(4):26-27, 30.

Collins S. Primary care shortages: strengthening this sector is urgently needed, now and in preparation for healthcare 

reform. American health & drug benefits. 2012;5(1):40-47.

Danto LA. National rural health service. Bulletin of the American College of Surgeons. 2010;95(8):43.

Edgerly D. Rural Colorado care. West Routt Fire Protection District provides BLS & ALS. JEMS : a journal of 

emergency medical services. 2013;38(6):23.

Emery JG. Comment on: Medical students on long-term regional and rural placements: what is the financial cost to 

supervisors? Rural and remote health. 2012;12:2247; discussion 2287.

Farmer D. Rural surgery is global surgery: seeking solutions to the growing surgical workforce crisis. JAMA 

surgery. 2013;148(9):821-822.

Farrell PM. Plan to address physician shortage requires proper support. WMJ : official publication of the State 

Medical Society of Wisconsin. 2005;104(6):73-74.

Ferguson WJ, Cashman SB, Savageau JA, Lasser DH. Family Medicine Residency Characteristics Associated With 

Practice in a Health Professions Shortage Area. Fam. Med. 2009;41(6):405-410.

Galloro V. Hard recruitment for smaller markets. Report shows importance of physicians to rural hospitals' financial 

health. Mod Healthc. 2008;38(22):30-31.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

112

Garpestad B. Funding for frontier home care: an anatomy of Montana's rural health care. Caring. 2009;28(4):38-41.

Glasser M, Peters K, Macdowell M. Rural Illinois hospital chief executive officers' perceptions of provider 

shortages and issues in rural recruitment and retention. The Journal of rural health : official journal of the American 

Rural Health Association and the National Rural Health Care Association. 2006;22(1):59-62.

Greenhill J, Poncelet AN. Transformative learning through longitudinal integrated clerkships. Medical education. 

2013;47(4):336-339.

Gunn J. Minnesota's rural health workforce shortages. Minnesota medicine. 2013;96(12):41-45.

Halaas GW. The Rural Physician Associate Program: new directions in education for competency. Rural and remote 

health. 2005;5(4):359.

Handel DA, Hedges JR. Improving rural access to emergency physicians. Academic emergency medicine : official 

journal of the Society for Academic Emergency Medicine. 2007;14(6):562-565.

Heady HR, Phillips D, Briggs E. Rural healthcare: how to recruit and retain health professionals. The Journal of 

medical practice management : MPM. 2006;21(4):219-224.

Hooker RS, Klocko DJ, Larkin GL. Physician Assistants in Emergency Medicine: The Impact of Their Role. 

Academic Emergency Medicine. 2011;18(1):72-77.

Huff C. Give docs a reason to set up shop. H&HN: Hospitals & Health Networks. 2012;86(2):20.

Huff C. You got the docs, now keep them! 5 strategies for rural hospitals to help retain their hard-won physicians. 

Hospitals & health networks / AHA. 2014;88(4):32-35, 32.

Huffstutter PJ. Rural surgeons--we must grow our own. Bulletin of the American College of Surgeons. 

2010;95(4):16-18; discussion 19.

Johna S. The rural surgeon: an endangered species. World J Surg. 2006;30(3):267-268.

Johnson CA, Johnson BE. Family medicine practice in rural any-country. Family medicine. 2008;40(1):55-56.

Kane KY, Quinn KJ, Stevermer JJ, et al. Summer in the country: changes in medical students' perceptions following 

an innovative rural community experience. Academic medicine : journal of the Association of American Medical 

Colleges. 2013;88(8):1157-1163.

Kirschling JM, Horvey-McPherson L, Curley D. Maine's nursing workforce legislation: Lessons from a rural state. 

Nurs. Outlook. 2008;56(2):63-69.

Kutscher B. The rural route. Hospitals in underserved areas taking different roads to recruit, retain physicians. Mod 

Healthc. 2013;43(18):30-31.

Lasher WF, Silverman SB. Relationship between residency training and practice location in primary care residency 

programs in Texas. [Internet Resource; Archival Material]. 2008; http://hdl.handle.net/2152/3687

Lauver LS, Swan BA, West MM, et al. Kids Into Health Careers: A Rural Initiative. Journal of Rural Health. 

2011;27(1):114-121.

Longo WE, Sumpio B, Duffy A, Seashore J, Udelsman R. Early specialization in surgery: the new frontier. The 

Yale journal of biology and medicine. 2008;81(4):187-191.

Lynge DC, Larson EH. Workforce issues in rural surgery. The Surgical clinics of North America. 2009;89(6):1285-

1291, vii.

Mantone J. Protective moves. Caps may lure docs to rural communities: study. Mod Healthc. 2005;35(23):8-9.

Margolis SA. Retaining rural medical practitioners: time for a new paradigm? Rural and remote health. 

2005;5(1):393.

Margolis SA. Letter to the editor Retaining rural medical practitioners: time for a new paradigm? Rural & Remote 

Health. 2005;5(1).
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

113

McHugh G, Stewart G. Regional and rural specialists. Anaesthesia and intensive care. 2010;38(4):778-779.

Mobley J, Zuniga MA. Training primary care physicians for local health authority duties in Texas. American journal 

of public health. 2012;102(7):e21-26.

Morley CP. Supporting Physicians Who Work in Challenging Contexts: A Role for the Academic Health Center. J. 

Am. Board Fam. Med. 2012;25(6):756-758.

Nelson BV, Talboy GE, Jr. Acute care surgery: redefining the general surgeon. Missouri medicine. 

2010;107(5):313-315.

Nocella IC. Recruitment of family physicians into rural California : predictors and possibilities. 2005:x, 144 leaves.

Nusbaum N. Commentary: Physician retirement and physician shortages. Journal of Community Health. 

2009;34(5):353-356.

O'Brien ML. Perception of rural caregiving. 2008:vii, 70 leaves ; 28 cm Dissertation: Thesis (M.S.)--University of 

Vermont, 2008.

Oklahoma. Physician Manpower Training C. Oklahoma Medical Loan Repayment Program. [Internet Resource; 

Archival Material]. 2012; Medium: Fact Sheet;. Available at: 

http://digitalprairie.ok.gov/cdm/ref/collection/stgovpub/id/229002

Page S, Birden H. Twelve tips on rural medical placements: what has worked to make them successful. Medical 

teacher. 2008;30(6):592-596.

Penn CL. Now hiring! Arkansas, like the nation, faces imminent health care worker shortages. The Journal of the 

Arkansas Medical Society. 2006;103(4):84-87.

Potter AJ, Mueller KJ, Mackinney AC, Ward MM. Effect of tele-emergency services on recruitment and retention of 

US rural physicians. Rural and remote health. 2014;14(3):2787.

Prengaman MP, Bigbee JL, Baker E, Schmitz DF. Development of the Nursing Community Apgar Questionnaire 

(NCAQ): a rural nurse recruitment and retention tool. Rural Remote Health. 2014;14(1).

Quinn KJ. Experiences influencing physician rural practice and retention: A phenomenological study. Dissertation 

Abstracts International Section A: Humanities and Social Sciences. 2012;72(8-A):2697.

Rabinowitz HK. Addressing physician specialty maldistribution. Jama. 2009;302(12):1270; author reply 1270.

Ricketts TC, 3rd, Walker E. Health reform and workforce: the North Carolina connection. North Carolina medical 

journal. 2010;71(3):250-253.

Robertson JE, Jr., Boyd J, Hedges JR, Keenan EJ. Strategies for increasing the physician workforce: the Oregon 

model for expansion. Academic medicine : journal of the Association of American Medical Colleges. 

2007;82(12):1158-1162.

Rosenblatt RA. Commentary: do medical schools have a responsibility to train physicians to meet the needs of the 

public? The case of persistent rural physician shortages. Academic medicine : journal of the Association of 

American Medical Colleges. 2010;85(4):572-574.

Rourke J. How can medical schools contribute to the education, recruitment and retention of rural physicians in their 

region? Bull World Health Organ. 2010;88(5):395-396.

Schein M. Editorial comment: Surgery in rural America. World J Surg. 2006;30(3):271-272.

Scutchfield FD. Increasing the medical school application pool: a key to training more rural physicians. The Journal 

of the Kentucky Medical Association. 2009;107(11):464.

Sharp DB. Factors related to the recruitment and retention of nurse practitioners in rural areas. University of Texas 

at El Paso. 2010;108.

Sheldon GF. Surgical practice in rural areas. Introduction. The Surgical clinics of North America. 2009;89(6):xviixix.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

114

Smith RJ. Frontier residents' perception of health care access. [Internet Resource]. 2008; vii, 75 leaves : ill. 

Dissertation: Thesis (M Nursing)--Montana State University--Bozeman, 2008. Available at: 

http://etd.lib.montana.edu/etd/2008/smith/SmithR0508.pdf.

Stempniak M. Rural hospitals band together to lure physicians. H&HN: Hospitals & Health Networks. 

2012;86(12):18-19.

Steptoe AP, Sullivan AF, Rogers J, Ginde AA, Camargo CA. Factors In Recruiting and Retaining Emergency 

Physicians to Rural Emergency Departments. Annals of emergency medicine. 2010;56(3):S72-S72.

Stevens CB. The dilemma of the rural psychiatrist. Journal of Psychiatric Practice. 2012;18(5):369-372.

Sumaya CV, Opara CM, Espino DV. The Geriatrician and Geriatric Psychiatrist Workforce in Texas: 

Characteristics, Challenges, and Policy Implications. J. Aging Health. 2013;25(6):1050-1064.

Szabo J. Rural hospitals look near and far for allied health professionals. Hospitals & health networks / AHA. 

2011;85(5):30-32, 32.

Thrall TH. Physician recruitment. Encouraging med students to return to rural areas. Hospitals & health networks / 

AHA. 2007;81(6):20.

Thrall TH. Physician recruitment. Garrison Keillor: doctor magnet. Hospitals & health networks / AHA. 

2008;82(2):24.

Tong STC, Makaroff LA, Xierali IM, et al. Proportion of Family Physicians Providing Maternity Care Continues to 

Decline. J. Am. Board Fam. Med. 2012;25(3):270-271.

Turlington AR. Barriers to rural mental health care: Perspectives from those who live there. Dissertation Abstracts 

International: Section B: The Sciences and Engineering. 2008;69(1-B):703.

Wakefield M, Amundson M, Moulton P, Gibbens B. Policy brief: North Dakota Health Care Workforce: planning 

together to meet future health care needs. The Prairie rose. 2007;76(2):9-13.

Welch JP. Whither goest general surgery? Archives of surgery (Chicago, Ill. : 1960). 2008;143(5):444-450.

Westergaard RP, Stockman LJ, Hyland HA, Guilfoyle SM, Fangman JJ, Vergeront JM. Provider Workforce 

Assessment in a Rural Hepatitis C Epidemic: Implications for Scale-up of Antiviral Therapy. Journal of primary 

care & community health. 2014.

Zastrow AW. Dilemma of general surgery recruitment in rural America. World J Surg. 2006;30(3):269-270.

Zigmond J. Help wanted. Benefits include an idyllic rural setting, a friendly community and some assistance paying 

off those hefty medical school loans. Mod Healthc. 2006;36(31):30-31.

Zuckerman RS. Rural surgery and surgical education. Surgical endoscopy. 2008;22(7):1592.

Zukowsky K, Swan BA, Powell M, et al. Implementing an MSN nursing program at a distance through an urbanrural partnership. Advances in neonatal care : official journal of the National Association of Neonatal Nurses. 

2011;11(2):114-118.

DUPLICATE 

Publications excluded because they are identical to another citation in the database.

Quinn KJHJL. Experiences influencing physician rural practice and retention a phenomenological study. [Internet 

Resource; Computer File]. 2009; 1 online resource (ix, 157 p.) Dissertation: Ph. D.; University of Missouri--

Columbia; 2009. Available at: http://hdl.handle.net/10355/9673

INTERVENTION

Publications excluded due to the intervention (the publication did not report on an intervention or 

did not report on an intervention relevant to provider recruitment or retention).
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

115

Anderson DM, Whitler ET, Johnson AO, Elam CL, Wilson EA, Asher LM. Increasing the medical school applicant 

pool: a key to training more rural physicians. The Journal of the Kentucky Medical Association. 2009;107(9):355-

360.

Casey MM, Hung PY, Moscovice I, Prasad S. The Use of Hospitalists by Small Rural Hospitals: Results of a 

National Survey. Med. Care Res. Rev. 2014;71(4):356-366.

Rodney WM, Martinez C, Collins M, Laurence G, Pean C, Stallings J. OB Fellowship Outcomes 1992-2010: Where 

Do They Go, Who Stops Delivering, and Why? Fam. Med. 2010;42(10):712-716.

Singh R, Mathiassen L, Stachura ME, Astapova EV. Sustainable Rural Telehealth Innovation: A Public Health Case 

Study. Health Serv. Res. 2010;45(4):985-1004.

OUTCOME

A large number of studies were excluded because they did not report on healthcare provider 

recruitment or retention measures.

Allen JEt, Davis AF, Hu W, Owusu-Amankwah E. Residents' Willingness-to-Pay for Attributes of Rural Health 

Care Facilities. The Journal of rural health : official journal of the American Rural Health Association and the 

National Rural Health Care Association. 2015;31(1):7-18.

Allen SM, Ballweg RA, Cosgrove EM, et al. Challenges and opportunities in building a sustainable rural primary 

care workforce in alignment with the Affordable Care Act: the WWAMI program as a case study. Academic 

medicine : journal of the Association of American Medical Colleges. 2013;88(12):1862-1869.

Arora S, Kalishman S, Dion D, et al. Partnering Urban Academic Medical Centers And Rural Primary Care 

Clinicians To Provide Complex Chronic Disease Care. Health Affairs. 2011;30(6):1176-1184.

Beverly EA, Reynolds S, Balbo JT, Adkins S, Longenecker R. Changing first-year medical students' attitudes 

toward primary care. Family medicine. 2014;46(9):707-712.

Bing-You RG, Bates PW, Epstein SK, Kuhlik AB, Norris TE. Using decentralized medical education to address the 

workforce needs of a rural state: a partnership between Maine Medical Center and Tufts University school of 

medicine. Rural and remote health. 2010;10(2):1494.

Borgstrom DC, Heneghan SJ. Bassett healthcare rural surgery experience. The Surgical clinics of North America. 

2009;89(6):1321-1323, viii-ix.

Breedlove G, Lamping B, Smith JA. The Kansas Health Education Training Center: caring for the underserved. 

Kansas Nurse. 2006;81(3):1-3.

Breon TA. Rural surgical practice: an Iowa group model. The Surgical clinics of North America. 2009;89(6):1359-

1366, x.

Brooks KD, Eley DS, Zink T. Profiles of rural longitudinal integrated clerkship students: a descriptive study of six 

consecutive student cohorts *. Medical teacher. 2014;36(2):148-154.

Bushardt RL, Whitt FK, Gregory T. Training physician assistants for rural Appalachia: an academic partnership for 

interprofessional collaboration. North Carolina medical journal. 2014;75(1):53-55.

Canon S, Shera A, Patel A, et al. A pilot study of telemedicine for post-operative urological care in children. Journal 

of telemedicine and telecare. 2014;20(8):427-430.

Cheng D, Fernandez J. Emergency medicine residency underrepresentation in rural states. The American journal of 

emergency medicine. 2005;23(1):92-93.

Cogbill TH, Jarman BT. Rural general surgery training: the Gundersen Lutheran approach. The Surgical clinics of 

North America. 2009;89(6):1309-1312, viii.

Colegrove DJ, Whitacre BE. Interest in rural medicine among osteopathic residents and medical students. Rural and 

remote health. 2009;9(3):1192.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

116

Colon-Gonzalez MC, McCall-Hosenfeld JS, Weisman CS, Hillemeier MM, Perry AN, Chuang CH. "Someone's got 

to do it" - Primary care providers (PCPs) describe caring for rural women with mental health problems. Mental 

health in family medicine. 2013;10(4):191-202.

Colwill JM, Cultice JM, Kruse RL. Will generalist physician supply meet demands of an increasing and aging 

population? Health affairs (Project Hope). 2008;27(3):w232-241.

Conger MM, Plager KA. Advanced nursing practice in rural areas: connectedness versus disconnectedness. Online 

Journal of Rural Nursing & Health Care. 2008;8(1):24-38.

Cook AF, Hoas H. Ethics and rural healthcare: what really happens? What might help? American Journal of 

Bioethics. 2008;8(4):52-56.

Cook MR, Yoon M, Hunter J, Kwong KR, Kiraly L. A nonmetropolitan surgery clerkship increases interest in a 

surgical career. Am. J. Surg. 2015;209(1):21-25.

Cosgrove EM, Harrison GL, Kalishman S, et al. Addressing physician shortages in New Mexico through a 

combined BA/MD program. Academic medicine : journal of the Association of American Medical Colleges. 

2007;82(12):1152-1157.

Crump WJ, Fricker RS, Crump AM. Just what are rural premedical students thinking? A report of the first 6 years of 

a pathways program. The Journal of rural health : official journal of the American Rural Health Association and the 

National Rural Health Care Association. 2010;26(1):97-99.

Decker MR, Bronson NW, Greenberg CC, Dolan JP, Kent KC, Hunter JG. The General Surgery Job Market: 

Analysis of Current Demand for General Surgeons and Their Specialized Skills. Journal of the American College of 

Surgeons. 2013;217(6):1133-1139.

Dehn RW. Workforce issues impacting PAs. JAAPA: Journal of the American Academy of Physician Assistants. 

2008;21(1):56-57.

DeKastle R. Telesurgery: providing remote surgical observations for students. AORN Journal. 2009;90(1):93-101.

Dennison DA, Yin Z, Kibbe D, Burns S, Trowbridge F. Training health care professionals to manage overweight 

adolescents: experience in rural Georgia communities. The Journal of rural health : official journal of the American 

Rural Health Association and the National Rural Health Care Association. 2008;24(1):55-59.

Dent MM, Mathis MW, Outland M, Thomas M, Industrious D. Chronic disease management: teaching medical 

students to incorporate community. Family medicine. 2010;42(10):736-740.

Deutchman ME, Nearing K, Baumgarten B, Westfall JM. Interdisciplinary rural immersion week. Rural and remote 

health. 2012;12:2045.

Dick JF, 3rd, Wilper AP, Smith S, Wipf J. The effect of rural training experiences during residency on the selection 

of primary care careers: a retrospective cohort study from a single large internal medicine residency program. 

Teaching and learning in medicine. 2011;23(1):53-57.

Donnon T, Woloschuk W, Myhre D. Issues related to medical students' engagement in integrated rural placements: 

an exploratory factor analysis. Canadian journal of rural medicine : the official journal of the Society of Rural 

Physicians of Canada = Journal canadien de la medecine rurale : le journal officiel de la Societe de medecine rurale 

du Canada. 2009;14(3):105-110.

Downey LH, Wheat JR, Leeper JD, Florence JA, Boulger JG, Hunsaker ML. Undergraduate rural medical education 

program development: focus group consultation with the NRHA Rural Medical Educators Group. The Journal of 

rural health : official journal of the American Rural Health Association and the National Rural Health Care 

Association. 2011;27(2):230-238.

Fannin JM, Barnes JN. Recruitment of physicians to rural America: a view through the lens of Transaction Cost 

Theory. The Journal of rural health : official journal of the American Rural Health Association and the National 

Rural Health Care Association. 2007;23(2):141-149.

Felix HC, Wootten EB, Stewart MK. The Arkansas Southern Rural Access Program: strategies for improving health 

care in rural areas of the state. The Journal of the Arkansas Medical Society. 2005;101(12):366-368.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

117

Foster PP, Williams JH, Estrada CA, et al. Recruitment of rural physicians in a diabetes internet intervention study: 

overcoming challenges and barriers. Journal of the National Medical Association. 2010;102(2):101-107.

Freed GL, Dunham KM, Loveland-Cherry C, Martyn KK. Family Nurse Practitioners: Roles and Scope of Practice 

in the Care of Pediatric Patients. Pediatrics. 2010;126(5):861-864.

Frellick M. The nurse practitioner will see you now. Advanced practice providers fill the physician gap. Hospitals & 

health networks / AHA. 2011;85(7):44-46, 48-49.

Frintner MP, Cull WL. Pediatric training and career intentions, 2003–2009. Pediatrics. 2012;129(3):522-528.

Frintner MP, Mulvey HJ, Pletcher BA, Olson LM. Pediatric resident debt and career intentions. Pediatrics. 

2013;131(2):312-318.

Gazewood JD, Rollins LK, Galazka SS. Beyond the horizon: the role of academic health centers in improving the 

health of rural communities. Academic medicine : journal of the Association of American Medical Colleges. 

2006;81(9):793-797.

Geske JA, Hartman T, Goodman B, Paulman P. Influence of a rural family medicine rotation on residency selection: 

MS3 versus MS4. Family medicine. 2011;43(8):556-559.

Giles WH, Arnold JD, Layman TS, et al. Education of the rural surgeon: experience from Tennessee. The Surgical 

clinics of North America. 2009;89(6):1313-1319, viii.

Gillman LM, Vergis A. General surgery graduates may be ill prepared to enter rural or community surgical practice. 

American journal of surgery. 2013;205(6):752-757.

Groth H, House H, Overton R, Deroo E. Board-certified emergency physicians comprise a minority of the 

emergency department workforce in iowa. The western journal of emergency medicine. 2013;14(2):186-190.

Halaas GW, Zink T, Brooks KD, Miller J. Clinical skills day: preparing third year medical students for their rural 

rotation. Rural and remote health. 2007;7(4):788.

Heidelbaugh J, Cooke J, Wimsatt L. Opportunities for medical student engagement with family medicine. Family 

medicine. 2013;45(7):484-491.

Hilty DM, Marks S, Wegelin J, Callahan EJ, Nesbitt TS. A randomized, controlled trial of disease management 

modules, including telepsychiatric care, for depression in rural primary care. Psychiatry (Edgmont (Pa. : 

Township)). 2007;4(2):58-65.

Hughes S, Zweifler JA, Garza A, Stanich MA. Trends in rural and urban deliveries and vaginal births: California 

1998-2002. The Journal of rural health : official journal of the American Rural Health Association and the National 

Rural Health Care Association. 2008;24(4):416-422.

Kaplan L, Brown M, Andrilla CH, Hart LG. Rural-urban practice patterns of nurse practitioners in Washington 

State. Journal for Nurse Practitioners. 2009;5(3):169-175.

Keenum AJ, Rawlings LM, Odoi A, et al. Tennessee emergency medicine workforce, 2009. Tennessee medicine : 

journal of the Tennessee Medical Association. 2013;106(3):41-43.

Kemper L, Barker AR, McBride TD, Mueller K. 2012 rural Medicare Advantage quality ratings and bonus 

payments. Rural policy brief. 2014(2014 1):1-4.

Kippenbrock T, Buron B, Odell E, Narcisse MR. Minimal Changes and Missed Opportunities: A Decade Look at 

Nurse Practitioners in the Lower Mississippi River Delta States. Journal of Professional Nursing. 2014;30(3):266-

272.

Kosteniuk J, Morgan D, Innes A, et al. Who steers the ship? Rural family physicians' views on collaborative care 

models for patients with dementia. Primary health care research & development. 2014;15(1):104-110.

Laditka JN, Laditka SB, Probst JC. More may be better: evidence of a negative relationship between physician 

supply and hospitalization for ambulatory care sensitive conditions. Health services research. 2005;40(4):1148-

1166.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

118

Li P, Ward MM, Schneider JE. Factors associated with Iowa rural hospitals' decision to convert to critical access 

hospital status. The Journal of rural health : official journal of the American Rural Health Association and the 

National Rural Health Care Association. 2009;25(1):70-76.

Lindeke L, Jukkala A, Tanner M. Perceived barriers to nurse practitioner practice in rural settings. The Journal of 

rural health : official journal of the American Rural Health Association and the National Rural Health Care 

Association. 2005;21(2):178-181.

Liu JJ, Bellamy G, Barnet B, Weng S. Bypass of local primary care in rural counties: effect of patient and 

community characteristics. Ann. Fam. Med. 2008;6(2):124-130.

Longo DR, Gorman RJ, Ge B. Rural medical school applicants: do their academic credentials and admission 

decisions differ from those of nonrural applicants? The Journal of rural health : official journal of the American 

Rural Health Association and the National Rural Health Care Association. 2005;21(4):346-350.

Macy ML, Butchart AT, Singer DC, Gebremariam A, Clark SJ, Davis MM. Looking Back on Rear-Facing Car 

Seats: Surveying US Parents in 2011 and 2013. Academic pediatrics. 2014.

Makaroff LA, Xierali IM, Petterson SM, Shipman SA, Puffer JC, Bazemore AW. Factors Influencing Family 

Physicians' Contribution to the Child Health Care Workforce. Ann. Fam. Med. 2014;12(5):427-431.

Marth NJ. Advanced practice registered nurse (APRN) supply in a rural state : trends to inform policy. 2010:vii, 74 

leaves.

McCollister HM, Severson PA, LeMieur TP, Roberts SA, Gujer MW. Building and maintaining a successful surgery 

program in rural Minnesota. The Surgical clinics of North America. 2009;89(6):1349-1357, ix.

Miller BF, Petterson S, Brown Levey SM, Payne-Murphy JC, Moore M, Bazemore A. Primary care, behavioral 

health, provider colocation, and rurality. Journal of the American Board of Family Medicine : JABFM. 

2014;27(3):367-374.

Mistretta MJ. Differential effects of economic factors on specialist and family physician distribution in Illinois: a 

county-level analysis. The Journal of rural health : official journal of the American Rural Health Association and the 

National Rural Health Care Association. 2007;23(3):215-221.

Moesinger R, Hill B. Establishing a rural surgery training program: a large community hospital, expert subspecialty 

faculty, specific goals and objectives in each subspecialty, and an academic environment lay a foundation. Journal of 

surgical education. 2009;66(2):106-112.

Mohr NM, Collier J, Hassebroek E, Groth H. Characterizing critical care physician staffing in rural America: a 

description of Iowa intensive care unit staffing. J Crit Care. 2014;29(2):194-198.

Mueller KJ, Potter AJ, MacKinney AC, Ward MM. Lessons From Tele-Emergency: Improving Care Quality And 

Health Outcomes By Expanding Support For Rural Care Systems. Health Affairs. 2014;33(2):228-234.

Murphy KL, Kobayashi D, Golden SL, Nageswaran S. Rural and nonrural differences in providing care for children 

with complex chronic conditions. Clin Pediatr (Phila). 2012;51(5):498-503.

Nation CL, Gerstenberger A, Bullard D. Preparing for change: the plan, the promise, and the parachute. Academic 

medicine : journal of the Association of American Medical Colleges. 2007;82(12):1139-1144.

Nelson RE, Battistone MJ, Ashworth WD, et al. Cost effectiveness of training rural providers to perform joint 

injections. Arthritis care & research. 2014;66(4):559-566.

Overman BA, Petri L, Knoki-Wilson U. A view from inside Arizona and New Mexico Indian country: pursuing a 

health career path. Rural and remote health. 2007;7(2):682.

Owen JA, Conaway MR, Bailey BA, Hayden GF. Predicting rural practice using different definitions to classify 

medical school applicants as having a rural upbringing. The Journal of rural health : official journal of the American 

Rural Health Association and the National Rural Health Care Association. 2007;23(2):133-140.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

119

Pathman DE, Morgan JC, Konrad TR, Goldberg L. States' Experiences With Loan Repayment Programs for Health 

Care Professionals in a Time of State Budget Cuts and NHSC Expansion. Journal of Rural Health. 2012;28(4):408-

415.

Pathman DE, Ricketts TC, 3rd. Interdependence of general surgeons and primary care physicians in rural 

communities. The Surgical clinics of North America. 2009;89(6):1293-1302, vii-viii.

Petelin JB, Nelson ME, Goodman J. Deployment and early experience with remote-presence patient care in a 

community hospital. Surgical endoscopy. 2007;21(1):53-56.

Phillips RL, Jr., Bazemore AW, Dodoo MS, Shipman SA, Green LA. Family physicians in the child health care 

workforce: opportunities for collaboration in improving the health of children. Pediatrics. 2006;118(3):1200-1206.

Pretorius RW, Milling DA, McGuigan D. Influence of a rural background on a medical student's decision to 

specialize in family medicine. Rural and remote health. 2008;8(3):928.

Ray TE. Perceptions of Nurse Practitioners on Rural and Remote Health Care Practice. Walden University. 

2014;183.

Rayman KM, Edwards J. Rural primary care providers' perceptions of their role in the breast cancer care continuum. 

The Journal of rural health : official journal of the American Rural Health Association and the National Rural Health 

Care Association. 2010;26(2):189-195.

Reed AJ, Schmitz D, Baker E, Nukui A, Epperly T. Association of "Grit" and Satisfaction in Rural and Nonrural 

Doctors. J. Am. Board Fam. Med. 2012;25(6):832-839.

Reschovsky JD, Staiti AB. Access and quality: does rural America lag behind? Health affairs (Project Hope). 

2005;24(4):1128-1139.

Rhoads Kinder SJ, Cherepski A, Freeman K. E-Learning in Obstetrics and HIV: Utilizing Online Interactive 

Scenarios to Replicate Critical Decision Making Moments. JOGNN: Journal of Obstetric, Gynecologic & Neonatal 

Nursing. 2014;43(Supp 1):S5-6.

Richardson I, Slifkin R, Randolph R, Holmes GM. A rural-urban comparison of allied health professionals' average 

hourly wage. Journal of allied health. 2010;39(3):e91-96.

Ricketts TC. The migration of physicians and the local supply of practitioners: a five-year comparison. Academic 

medicine : journal of the Association of American Medical Colleges. 2013;88(12):1913-1918.

Ricketts TC, Randolph R. Urban-rural flows of physicians. Journal of Rural Health. 2007;23(4):277-285.

Ricketts TC, Randolph R. The diffusion of physicians. Health Affairs. 2008;27(5):1409-1415.

Rittenhouse DR, Fryer GE, Jr., Phillips RL, Jr., et al. Impact of Title VII training programs on community health 

center staffing and national health service corps participation. Annals of family medicine. 2008;6(5):397-405.

Roh CY, Lee KH, Fottler MD. Determinants of hospital choice of rural hospital patients: the impact of networks, 

service scopes, and market competition. Journal of medical systems. 2008;32(4):343-353.

Roseamelia C, Greenwald JL, Bush T, Pratte M, Wilcox J, Morley CP. A Qualitative Study of Medical Students in a 

Rural Track: Views on Eventual Rural Practice. Fam. Med. 2014;46(4):259-266.

Rosenblatt RA, Hagopian A, Andrilla CH, Hart G, Jr. Will rural family medicine residency training survive? Family 

medicine. 2006;38(10):706-711.

Rossi A, Rossi D, Rossi M, Rossi P. Continuity of care in a rural critical access hospital: Surgeons as primary care 

providers. Am. J. Surg. 2011;201(3):359-362.

Royston PJ, Mathieson K, Leafman J, Ojan-Sheehan O. Medical student characteristics predictive of intent for rural 

practice. Rural Remote Health. 2012;12(3).

Sawyer BT, Ginde AA. Scope of practice and autonomy of physician assistants in rural versus urban emergency 

departments. Academic emergency medicine : official journal of the Society for Academic Emergency Medicine. 

2014;21(5):520-525.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

120

Schwartz MR. The physician pipeline to rural and underserved areas in Pennsylvania. The Journal of rural health : 

official journal of the American Rural Health Association and the National Rural Health Care Association. 

2008;24(4):384-389.

Seidel RW, Kilgus MD. Agreement between telepsychiatry assessment and face-to-face assessment for Emergency 

Department psychiatry patients. J. Telemed. Telecare. 2014;20(2):59-62.

Shannon CK. A gender-based study of attitudes and practice characteristics of rural physicians in West Virginia. 

The West Virginia medical journal. 2006;102(6):22-25.

Shannon CK, Baker H, Jackson J, Roy A, Heady H, Gunel E. Evaluation of a required statewide interdisciplinary 

Rural Health Education Program: student attitudes, career intents and perceived quality. Rural and remote health. 

2005;5(4):405.

Shapiro J, Longenecker R. Country doctors in literature: helping medical students understand what rural practice is 

all about. Academic medicine : journal of the Association of American Medical Colleges. 2005;80(8):724-727.

Sirinek KR, Willis R, Stewart RM. Geographic maldistribution of general surgery PGYI residents: another US 

surgical desert. Am. J. Surg. 2014;208(6):1023-1027.

Slagle DR, Byington RL, Verhovsek EL. Rural Versus Urban: Tennessee Health Administrators' Strategies on 

Recruitment and Retention for Allied Health Professionals. Journal of Health Care Finance. 2012;38(4):91-104.

Smoot RL, Farley DR. Minnesota general surgeons. Where do they come from? Minnesota medicine. 

2006;89(12):46-48.

Snyder CL. Temporal geomapping of pediatric surgeons in the United States. Journal of pediatric surgery. 

2008;43(3):424-429.

Stingley S, Schultz H. Helmsley Trust Support For Telehealth Improves Access To Care In Rural And Frontier 

Areas. Health Affairs. 2014;33(2):336-341.

Tatkon-coker AL. The relationship of medicare and medicaid reimbursement on job dissatisfaction and spousal 

influence on the retention of rural health practitioners in North Central Iowa and South Central Minnesota. 2007:xiv, 

279 leaves ; 229 cm. Dissertation: Thesis (DBA) -- Nova Southeastern University, 2007.

Toner JA, Ferguson KD, Sokal RD. Continuing interprofessional education in geriatrics and gerontology in 

medically underserved areas. Journal of Continuing Education in the Health Professions. 2009;29(3):157-160.

Tong ST, Makaroff LA, Xierali IM, Puffer JC, Newton WP, Bazemore AW. Family physicians in the maternity care 

workforce: factors influencing declining trends. Maternal and child health journal. 2013;17(9):1576-1581.

Vanleit B, Cubra J. Student-developed problem-based learning cases: preparing for rural healthcare practice. Rural 

and remote health. 2005;5(4):399.

Wadman MC, Hoffman LH, Erickson T, Tran TP, Muelleman RL. The impact of a rural emergency department 

rotation on applicant ranking of a US emergency medicine residency program. Rural & Remote Health. 

2007;7(4):686.

Wayne SJ, Kalishman S, Jerabek RN, Timm C, Cosgrove E. Early predictors of physicians' practice in medically 

underserved communities: a 12-year follow-up study of University of New Mexico School of Medicine graduates. 

Academic medicine : journal of the Association of American Medical Colleges. 2010;85(10 Suppl):S13-16.

Weeks WB, Wallace AE. Rural-urban differences in primary care physicians' practice patterns, characteristics, and 

incomes. Journal of Rural Health. 2008;24(2):161-170.

Wheat JR, Leeper JD, Brandon JE, Guin SM, Jackson JR. The Rural Medical Scholars Program Study: Data to 

Inform Rural Health Policy. J. Am. Board Fam. Med. 2011;24(1):93-101.

Wimmer P. Professionalism among medical practitioners a case study of rural physicians. [Internet Resource; 

Computer File]. 2007; http://scholar.lib.vt.edu/theses/available/etd-02232007-083630
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

121

Wolf L, Delao AM. Identifying the Educational Needs of Emergency Nurses in Rural and Critical Access Hospitals. 

J. Contin. Educ. Nursing. 2013;44(9):424-428.

Wright DB. Care in the country: a historical case study of long-term sustainability in 4 rural health centers. Am. J. 

Public Health. 2009;99(9):1612-1618.

Xu X, Siefert KA, Jacobson PD, Lori JR, Gueorguieva I, Ransom SB. Malpractice burden, rural location, and 

discontinuation of obstetric care: a study of obstetric providers in Michigan. The Journal of rural health : official 

journal of the American Rural Health Association and the National Rural Health Care Association. 2009;25(1):33-

42.

Zayas LE, McGuigan D. Experiences promoting healthcare career interest among high-school students from 

underserved communities. J. Natl. Med. Assoc. 2006;98(9):1523-1531.

Zink T, Halaas GW, Brooks KD. Learning professionalism during the third year of medical school in a 9-monthclinical rotation in rural Minnesota. Medical teacher. 2009;31(11):1001-1006.

Zink T, Halaas GW, Finstad D, Brooks KD. The rural physician associate program: the value of immersion learning 

for third-year medical students. The Journal of rural health : official journal of the American Rural Health 

Association and the National Rural Health Care Association. 2008;24(4):353-359.

Zink T, Power DV, Finstad D, Brooks KD. Is there equivalency between students in a longitudinal, rural clerkship 

and a traditional urban-based program? Family medicine. 2010;42(10):702-706.

PARTICIPANTS

Publications excluded because they did not report on healthcare providers or reported on

providers outside the scope of the review.

Amerson RMGLS, thesis a. Understanding the occupational choices of rural white southern males. [Book; Computer 

File]. 2012; 1 online resource (208 p.) Dissertation: Ed. D.; Georgia Southern University; 2012. Available at: Host: 

http://digitalcommons.georgiasouthern.edu/etd/570/.

Bradbury GB. Retention of staff nurses in a rural hospital setting in the eastern part of North Carolina: The impact of 

leadership style. Capella University. 2013;152.

Bratt MM. Retaining the next generation of nurses: the Wisconsin nurse residency program provides a continuum of 

support. J Contin Educ Nurs. 2009;40(9):416-425.

Cramer M, Nienaber J, Helget P, Agrawal S. Comparative analysis of urban and rural nursing workforce shortages 

in Nebraska hospitals. Policy, politics & nursing practice. 2006;7(4):248-260.

Crump WJ, Fricker RS, Flick KF, Gerwe-Wickham K, Greenwell K, Willen KL. A Rural Pathways Program for 

High School Students: Reinforcing a Sense of Place. Fam. Med. 2014;46(9):713-717.

DeOnna J. Rural healthcare manager academy: a best practice in Pennsylvania workforce development. The 

Pennsylvania nurse. 2008;63(4):14-16.

Freeman VA, Slifkin RT, Patterson PD. Recruitment and retention in rural and urban EMS: results from a national 

survey of local EMS directors. Journal of public health management and practice : JPHMP. 2009;15(3):246-252.

Fu MC, Buerba RA, Gruskay J, Grauer JN. Longitudinal urban-rural discrepancies in the US orthopaedic surgeon 

workforce. Clinical orthopaedics and related research. 2013;471(10):3074-3081.

Gifford VM. Factors that contribute to rural provider retention, service utilization, and engagement in mentorship by 

cultural experts. 2012:xiii, 208 leaves ; 228 cm. Dissertation: Thesis (Ph.D.)--University of Alaska Anchorage, 

2012.; Thesis (Ph.D.)--University of Alaska Fairbanks, 2012.

Gossler HE. Reducing the rural physician shortage in the Pacific Northwest incentivizing rural physicians through 

financial reimbursement. [Internet Resource; Computer File]. 2011; 1 online resource (vi, 37 p.) Dissertation: Thesis 

(M.P.A.)--Washington State University, December 2011.; M.P.A.; Washington State University; 2011. Available at: 

http://www.dissertations.wsu.edu/Thesis/Fall2011/h_gossler_112211.pdf
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

122

Grauer JN, Fu MC. Editor's spotlight/take 5: longitudinal urban-rural discrepancies in the US orthopaedic surgeon 

workforce. Interview by Seth S. Leopold. Clinical orthopaedics and related research. 2013;471(10):3071-3073.

Greenwood JE. The impact of geography, training, and experience on scope of practice among Certified Registered 

Nurse Anesthetists. Virginia Commonwealth University. 2014;176.

Hauenstein EJ, Glick DF, Kane C, Kulbok P, Barbero E, Cox K. A Model to Develop Nurse Leaders for Rural 

Practice. Journal of Professional Nursing. 2014;30(6):463-473.

Herrmann JA, Hershow RC. One Medicine, one university: The DVM/MPH program at the University of Illinois. J. 

Vet. Med. Educ. 2008;35(2):194-198.

Hughes J, Cates JR, Liddon N, Smith JS, Gottlieb SL, Brewer NT. Disparities in how parents are learning about the 

human papillomavirus vaccine. Cancer epidemiology, biomarkers & prevention : a publication of the American 

Association for Cancer Research, cosponsored by the American Society of Preventive Oncology. 2009;18(2):363-

372.

Kruger GJ. Perceived factors affecting nurse recruitment in northwest Minnesota. University of North Dakota. 

2009;102.

Meyer D, Hamel-Lambert J, Tice C, Safran S, Bolon D, Rose-Grippa K. Recruiting and retaining mental health 

professionals to rural communities: an interdisciplinary course in Appalachia. The Journal of rural health : official 

journal of the American Rural Health Association and the National Rural Health Care Association. 2005;21(1):86-

91.

Molanari DL, Jaiswal A, Hollinger-Forrest T. Rural Nurses: Lifestyle Preferences and Education Perceptions. 

Online Journal of Rural Nursing & Health Care. 2011;11(2):16-26.

Murray MF, Havener JM, Davis PS, Jastremski C, Twichell ML. The rural pipeline: building a strong nursing 

workforce through academic and service partnerships. The Nursing clinics of North America. 2011;46(1):107-121.

Nayar P, Nguyen AT, Apenteng B, Yu F. Preventable hospitalizations: does rurality or non-physician clinician 

supply matter? Journal of community health. 2012;37(2):487-494.

Nelson W, Rosenberg MC, Weiss J, Goodrich M. New Hampshire critical access hospitals: CEOs' report on ethical 

challenges. Journal of healthcare management / American College of Healthcare Executives. 2009;54(4):273-283; 

discussion 283-274.

Pruthi RS, Neuwahl S, Nielsen ME, Fraher E. Recent trends in the urology workforce in the United States. Urology. 

2013;82(5):987-993.

Rhyne RL, Daniels ZM, Skipper BJ, Sanders ML, VanLeit BJ. Interdisciplinary health education and career choice 

in rural and underserved areas. Medical Education. 2006;40(6):504-513.

Skillman SM. Changes in the rural registered nurse workforce from 1980 to 2004. Seattle: WWAMI Center for 

Health Workforce Studies, University of Washington, School of Medicine, Dept. of Family Medicine; 2007.

Slagle DR. Rural Versus Urban: Tennessee Health Administratorsâ Strategies on Recruitment and Retention for 

Allied Health Professionals. [Internet Resource]. 2010; http://etd-submit.etsu.edu/etd/theses/available/etd-0702110-

121920/

Sullivan D, Rohlfsen L. Professional Politics and the Challenge of Anesthesia Availability in Rural Hospitals. 

Inequalities and Disparities in Health Care and Health: Concerns of Patients, Providers and Insurers. Vol 25. 

Bingley: Emerald Group Publishing Limited; 2007:149-167.

Thaker SI, Pathman DE, Mark BA, Ricketts TC, III. Service-linked scholarships, loans, and loan repayment 

programs for nurses in the Southeast. Journal of Professional Nursing. 2008;24(2):122-130.

Trepanier A, Gagnon MP, Mbemba GI, et al. Factors associated with intended and effective settlement of nursing 

students and newly graduated nurses in a rural setting after graduation: a mixed-methods review. International 

journal of nursing studies. 2013;50(3):314-325.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

123

Wachtel RE, Dexter F. Training rotations at hospitals as a recruitment tool for Certified Registered Nurse 

Anesthetists. AANA journal. 2012;80(4 Suppl):S45-48.

Weichel D. Orthopedic surgery in rural American hospitals: a survey of rural hospital administrators. The Journal of 

rural health : official journal of the American Rural Health Association and the National Rural Health Care 

Association. 2012;28(2):137-141.

SETTING

Publications excluded because they did not report on a healthcare setting or an US healthcare 

setting.

[no author] Rural Retention Program (RRP) policy framework for health authorities. [Internet Resource; Computer 

File]. 2012; 1 online resource (17 p.). Available at: 

http://www.llbc.leg.bc.ca/public/pubdocs/bcdocs2013/526533/retentionprogram.pdf.

Bardella IJ. Solutions for recruitment and retention of rural and remote health workforce. Family medicine. 

2010;42(4):288-289.

Cayley W, Jr. Patients as allies in student education and rural recruitment. Family medicine. 2010;42(4):288.

Chen LC. Striking the right balance: health workforce retention in remote and rural areas. Bull World Health Organ. 

2010;88(5):323, a.

Couper ID, Hugo JFM, Conradie H, Mfenyana K. Influences on the choice of health professionals to practise in 

rural areas. SAMJ S. Afr. Med. J. 2007;97(11):1082-1086.

Deirdre J, Florence M, Olive Y. Putting the (R) Ural in Preceptorship. [Internet Resource; Archival Material]. 2012; 

http://dx.doi.org/10.1155/2012/528580

Eley D, Young L, Przybeck TR. Exploring temperament and character traits in medical students; a new approach to 

increase the rural workforce. Medical teacher. 2009;31(3):e79-84.

Eley D, Young L, Przybeck TR. Exploring the Temperament and Character Traits of Rural and Urban Doctors. 

Journal of Rural Health. 2009;25(1):43-49.

Haji M, Durairaj V, Zurn P, Stormont L, Mapunda M. Emerging opportunities for recruiting and retaining a rural 

health workforce through decentralized health financing systems. Bull World Health Organ. 2010;88(5):397-399.

Igumbor EU, Kwizera EN. The positive impact of rural medical schools on rural intern choices. Rural and remote 

health. 2005;5(2):417.

Kuhn MKM, Ochsen C. Demographic and geographic determinants of regional physician supply. [Internet 

Resource; Archival Material]. 2009; http://hdl.handle.net/10419/39775

Peach HG. Rural placement programs. Rural and remote health. 2011;11(3):1844.

Peake SL, Judd N. Supporting rural community-based critical care. Curr. Opin. Crit. Care. 2007;13(6):720-724.

Roberts C, Daly M, Kumar K, Perkins D, Richards D, Garne D. A longitudinal integrated placement and medical 

students' intentions to practise rurally. Medical education. 2012;46(2):179-191.

Schneider HB. Attracting medical students to rural areas. CMAJ : Canadian Medical Association journal = journal 

de l'Association medicale canadienne. 2008;179(8):801.

Sen Gupta TK, Hays RB, Kelly G, Buettner PG. Are medical student results affected by allocation to different sites 

in a dispersed rural medical school? Rural and remote health. 2011;11(1):1511.

Singer J. Urban and rural health care environments: factors affecting new graduates' choice for employment. Journal 

of Medical Imaging & Radiation Sciences. 2008;39(3):173-174.

Solowiej K, Upton P, Upton D, et al. A scheme to support the recruitment and retention of allied health 

professionals to hard to fill posts in rural areas including commentaries by Kevin O'Toole; Matthew J Leach; Leonie 
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

124

Segal; Ana Manzano-Santaella and Crispin Coombs. International Journal of Therapy & Rehabilitation. 

2010;17(10):545-555.

Stone N. The Rural Interprofessional Education Project (RIPE). Journal of interprofessional care. 2006;20(1):79-81.

Strasser R, Neusy AJ. Context counts: training health workers in and for rural and remote areas. Bull World Health 

Organ. 2010;88(10):777-782.

Tolhurst HM, Adams J, Stewart SM. An exploration of when urban background medical students become interested 

in rural practice. Rural and remote health. 2006;6(1):452.

Vogel L. Rural premedicine program aims to tackle doctor shortages. CMAJ : Canadian Medical Association journal 

= journal de l'Association medicale canadienne. 2014;186(3):177.

Whalley DSB. Why do practitioners work in deprived areas? : identifying affinity factors for urban deprived general 

practice. [Internet Resource; Computer File]. 2012; 1 online resource. Dissertation: Thesis (Ph.D.)--University of 

Manchester, 2012. Available at: http://www.manchester.ac.uk/escholar/uk-ac-man-scw:160829.

Whitelaw AS, Hsu R, Corfield AR, Hearns S. Establishing a rural Emergency Medical Retrieval Service. 

Emergency medicine journal : EMJ. 2006;23(1):76-78.

Woloschuk W, Myhre D, Jackson W, McLaughlin K, Wright B. Comparing the performance in family medicine 

residencies of graduates from longitudinal integrated clerkships and rotation-based clerkships. Academic medicine : 

journal of the Association of American Medical Colleges. 2014;89(2):296-300.

Yonge O, Myrick F, Ferguson L. The Challenge of Evaluation in Rural Preceptorship. Online Journal of Rural 

Nursing & Health Care. 2011;11(2):3-15.

Ypinazar VA, Margolis SA. Clinical simulators: applications and implications for rural medical education. Rural 

and remote health. 2006;6(2):527.

TIMING

Publications excluded due to timing because they exclusively reported on practicing in rural care 

before 2005.

Backer EL, McLlvain HE, Paulman PM, Ramaekers RC. The characteristics of successful family physicians in rural 

Nebraska: A qualitative study of physician interviews. Journal of Rural Health. 2006;22(2):189-191.

Baldwin L, Patanian MM, Larson EH, et al. Modeling the mental health workforce in Washington State: using state 

licensing data to examine provider supply in rural and urban areas. Journal of Rural Health. 2003;22(1):50-58.

Brown SR, Birnbaum B. Student and resident education and rural practice in the Southwest Indian Health Service: a 

physician survey. Family medicine. 2005;37(10):701-705.

Daniels ZM, Vanleit BJ, Skipper BJ, Sanders ML, Rhyne RL. Factors in recruiting and retaining health 

professionals for rural practice. The Journal of rural health : official journal of the American Rural Health 

Association and the National Rural Health Care Association. 2007;23(1):62-71.

Doty B, Heneghan S, Gold M, et al. Is a broadly based surgical residency program more likely to place graduates in 

rural practice? World J.Surg. 2006;30(12):2089-2093.

Evans TC, Wick KH, Brock DM, Schaad DC, Ballweg R. Academic degrees and clinical practice characteristics: the 

University of Washington physician assistant program: 1969-2000. The Journal of rural health : official journal of 

the American Rural Health Association and the National Rural Health Care Association. 2006;22(3):212-219.

Florence JA, Goodrow B, Wachs J, Grover S, Olive KE. Rural health professions education at East Tennessee State 

University: survey of graduates from the first decade of the community partnership program. The Journal of rural 

health : official journal of the American Rural Health Association and the National Rural Health Care Association. 

2007;23(1):77-83.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

125

Krist AH, Johnson RE, Callahan D, Woolf SH, Marsland D. Title VII funding and physician practice in rural or lowincome areas. The Journal of rural health : official journal of the American Rural Health Association and the 

National Rural Health Care Association. 2005;21(1):3-11.

Lang F, Ferguson KP, Bennard B, Zahorik P, Sliger C. The Appalachian Preceptorship: over two decades of an 

integrated clinical-classroom experience of rural medicine and Appalachian culture. Academic medicine : journal of 

the Association of American Medical Colleges. 2005;80(8):717-723.

Miller T, Hooker RS, Mains DA. Characteristics of osteopathic physicians choosing to practice rural primary care. 

The Journal of the American Osteopathic Association. 2006;106(5):274-279.

Pacheco M, Weiss D, Vaillant K, et al. The impact on rural New Mexico of a family medicine residency. Academic 

medicine : journal of the Association of American Medical Colleges. 2005;80(8):739-744.

Peterson LE, Dodoo M, Bennett KJ, Bazemore A, Phillips RL, Jr. Nonemergency medicine-trained physician 

coverage in rural emergency departments. The Journal of rural health : official journal of the American Rural Health 

Association and the National Rural Health Care Association. 2008;24(2):183-188.

Smucny J, Beatty P, Grant W, Dennison T, Wolff LT. An evaluation of the Rural Medical Education Program of the 

State University Of New York Upstate Medical University, 1990-2003. Academic medicine : journal of the 

Association of American Medical Colleges. 2005;80(8):733-738.

Thomas CR, Holzer CE, 3rd. The continuing shortage of child and adolescent psychiatrists. Journal of the American 

Academy of Child and Adolescent Psychiatry. 2006;45(9):1023-1031.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

126

APPENDIX 3. RISK OF BIAS ASSESSMENT

KQ1 STUDIES

ID Data source 

reporting

External validity

Branch, 201447 Low risk Low risk

Camargo, 200856 Low risk Low risk

Ghosh, 201151 Low risk Low risk

Hendryx, 200854 Low risk Low risk

Maizel, 200948 Low risk Unclear

Rayburn, 201252 Low risk Unclear

Rosenblatt, 20101 Low risk Low risk

Stewart, 201349 Low risk Low risk

Thomas, 200950 Low risk Unclear

Williams, 201153 Low risk Low risk

Wilson, 201155 Low risk Low risk

KQ2 STUDIES

ID Response rate Confounding 

variables 

Other limitations

Chen, 201057 Unclear High risk

DHHS, 200666 Unclear High risk

Duffrin, 201467 High risk High risk

Fordyce, 201258 Unclear High risk

Glasser, 201068 Low risk High risk

Hancock, 200969 High risk High risk Recall bias and small sample size with 

likely selection bias

Helland, 201070 Low risk High risk

Heneghan, 200571 High risk High risk Selection bias, response bias, did not 

address non-responders

Henry, 200765 High risk High risk Qualitative results only

Hughes, 200572 Unclear High risk

Jarman, 200973 High risk High risk

Kimball, 200774 Low risk High risk Qualitative study with selection bias

MacDowell, 2013105 Unclear High risk

Mason, 201263 Unclear Unclear

Pepper, 201075 Low risk High risk 

Phillips,200960 Unclear Low risk
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

127

Phillips, 201389 Low risk High risk

Rabinowitz, 201261 Low risk High risk 

Renner, 201062 Low risk Unclear

Schiff, 201277 Low risk High risk

Shannon, 201178 Unclear Low risk Use of self-reported data, limited sample 

size, and limited external validity

Smith, 201264 High risk High risk

Snyder, 201479 High risk High risk

Stenger, 200880 Low risk Unclear Self-reported data, selection bias

Whitacre, 201081 Unclear Low risk

Zink, 201082 Unclear Unclear

KQ3 AND KQ4 STUDIES

ID Selection 

bias

Performance 

bias

Attrition 

bias

Detection 

bias

Other bias

Kahn, 201083 High risk Low risk Low risk Low risk N/A

No Author, 

200785

High risk Low risk Low risk Low risk N/A

Renner, 201062 High risk Unclear Low risk Low risk N/A

Wheeler, 200984 High risk Low risk Low risk Low risk N/A

Wheeler, 201386 High risk Unclear Unclear Unclear Recruitment data no 

denominator, retention data 

not stratified by program

KQ5 STUDIES

ID Selection bias

Antonenko, 200996 High risk

Baker, 2012101 Unclear

Bonham, 201488 High risk

Crane, 2014102 High risk

Crump, 2013103 Unclear

Deutchman, 201398 High risk

Deveney, 200929,97 High risk

Glasser, 200868,104 High risk

Kallail, 2010106 High risk

Mason, 201263 High risk

Nash, 200893 High risk

Patterson, 201399 Unclear
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

128

Phillips,200960 Low risk

Phillips, 201389 Unclear

Quinn, 201192 Low risk

Rabinowitz, 201190 Low risk

Rabinowitz, 201291 Low risk

Rabinowitz, 2013108 Low risk

Ross, 2013110 High risk

Shipman, 2013100 Low risk

Talley, 201187 Low risk

Whitacre, 201081 High risk

Zink, 201082 Unclear
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

129

APPENDIX 4. PEER REVIEW COMMENTS/AUTHOR RESPONSES

Comments Response

Appears comprehensive and very useful. Thank you

Very thorough and complete. Thank you

Missing studies: Phillips RL; Dodoo MS; Petterson S; et al. Specialty and Geographic Distribution of the Physician 

Workforce: What Influences Medical Student and Resident Choices? Robert Graham Center: Policy Studies in 

Family Medicine and Primary Care, Washington, DC, 2009.

The study is included in the final report.

It appears VA studies on Workforce were not including in this synthesis. Was this by design? We have limited this systematic review to 

studies in the public domain and applied this 

inclusion criterion consistently.

Aren't there unpublished data on VA health care workforce supply and demand? Please see above

Specifically retention studies for National Health Service Corps program completers and reports from Community 

Health Centers regarding retention patterns would be helpful. That CHCs and RHCs were not included as part of 

the study should be further explained and clarified for readers since these sites are supported by federal rural 

workforce investments.

We have added more detail regarding 

completers as suggested. Inclusion criteria 

specified provider specialties, rather than 

types of healthcare delivery centers but we 

have added rural health clinics as one of the 

examples why the provider groups were 

selected.

In my opinion this report is outstanding! The authors did an excellent job of identifying the problem and evaluating 

the scientific rigour of the studies. Just a few suggestions:

- page 7, line 43 "Study Selection" - I would recommend that the authors provide more detail on how disagreements 

were resolved (e.g., consensus, third reviewer, etc.). 

Added as suggested

- Page 13, line 28 "Technical Expert Panel". Please change Bureau of Health Professions to Bureau of Health 

Workforce. We had a merger last year and with it came the name change. 

Revised as suggested

- Page 70, line 51 - "Limitations" The HRSA workforce projections covered many more specialties than 

psychologists and pharmacists. Would suggest changing to various health providers or health professions.

We have revised the sentence slightly to 

address this point.

- Page 71, line 10-11 "Research Gaps/Future Research" While the supply and demand microsimulation models 

used by HRSA are complex, they have been updated and do include the effect of the ACA in the calculations. The 

effect of the ACA in the HRSA model and other models indicates about a 2% effect in demand due to the ACA. 

HRSA has also put out supply and demand numbers for many health professionals other than physicians (e.g., 

allied health professions, dentists, nurses, etc). While we have identified numbers of providers needed these 

estimates are based on current delivery models and this is an area that needs future research. Identifying a metric 

for upcoming delivery models will improve the estimates of providers needed (e.g., team-based care, PCMH, ACOs, 

etc.). Further, another area of research that is needed is not only identifying outputs of providers needed, but what 

skills are needed by the providers. I think it might be helpful to mention in this section that there is a need to identify 

the skills and competencies needed in the existing and new workforce. How do we go about training the existing 

workforce to work in the new delivery models such as team-based care, PCMH, ACO's etc. 

Thank you for this suggestion, we have 

added this point.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

130

- Page 71, line 26/27 - "Research Gaps/Future Research" The HRSA demand model does account for the effect of 

the ACA. To date, the HRSA model and other existing models, have only found the ACA to have approximately a 

2% effect on the demand of health professionals. 

We have revised the paragraph to avoid 

misinterpretation of the sentence.

- Page 71, line 32 - "Research Gaps/Future Research" One of the difficulties with comparing training programs 

across health professions is that there are so many differences in the time of training, locations used for clinical 

training and providing training in rural settings. Most medical schools are still training using the acute care setting 

and rarely have a rural track for students. I think that is where research needs to occur in the shift of training from 

acute care to rural/primary care settings.

Added as suggested

This report is well written, carefully done, and nicely demonstrates the complexity and nuance of rural healthcare 

need, the determinants of provider’s geographic choices, and the many issues around provider recruitment and 

retention. With the exception of my comments regarding the lens of “place,” the discussion, limitations, and 

research gaps are well-written, thorough and justified by the review. My comments regarding the conclusions of the 

review are listed at the end of this review document.

Methods:

I only have one question regarding the selection criteria. How was it determined that a particular study was relevant 

to CBOC’s and CAH’s? Were there explicit criteria other than specialty?

We have addressed this point in the limitation 

section and added more detail on the 

decision. 

Comments regarding the key study questions: KQ1: What are the current versus projected healthcare provider 

needs by numbers and disciplines in the next 20 years in rural areas?

I find it interesting that the studies of general surgery use a somewhat arbitrary ratio of general surgeons/100,000 

population, when the general surgeon in a rural community has a very different scope of practice, potentially 

underestimating the need. A general surgeon in an urban area has other specialty surgeons who can take up the 

slack. The same can be said about family physicians practicing in rural communities, where specialists are 

underrepresented relative to urban places and where generalists practice a wider scope. All of this complicates the 

question of provider need.

Thank you for this thoughtful comment. To at 

least partially address this point we have 

elaborated more on the definition of need in 

the discussion. 

KQ2: What factors influence healthcare providers’ geographic choices for practice?

Macy Foundation Graham Center Report 2009, not peer-reviewed in the usual sense, but a well- done research 

study and report, vetted by experts:

Phillips et all, 2009.

Please see above

This report nicely demonstrates the relative importance of the predictor variables, and identifies location of medical 

school as the strongest independent predictor, i.e. the geography of education and training, an even stronger factor 

than growing up in a rural place – the factor which this review concludes is the most consistent factor. This is one of 

the few studies to use multivariate analysis across a large number of variables. The Pepper study from Wyoming is 

an exception and found no association with place of training. Unfortunately, residency training programs in that 

region do not vary much by the usual measures of rurality. The Rabinowitz study in Philadelphia has the opposite 

problem, occurring primarily in an urban place with a region characterized by an intricate patchwork of rural and 

urban.

We agree with this observation and have 

expanded on the discussion.

There is growing evidence that the place of education and training (the context) is as important if not more important

than either individual characteristics or the program (often described in this review as “educational effort” or 

content). The difficulty is that the geography question has not been asked in most studies nor have research studies 

generally been addressed through that lens. Generally success under question KQ5 is attributed to the “training,” 

not the duration and “place” of training experiences. This represents a significant cognitive framing bias among 

researchers in many studies.

Very interesting point; we have added it to 

the future research section.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

131

The fact that the Majority of rural providers did not grow up in small town 61,76, stems from the reality that, although 

growing up in a rural area is associated with a propensity for rural practice, there are too few such students even 

entering medical school, and of those who do, the majority still end up in urban practice. A modest percentage of a 

small number is an even smaller number. Therefore the selective admissions of individuals is unlikely by itself to 

address workforce needs, further supporting the importance of both developing effective educational interventions 

and identifying effective contexts. 

In addition, although frequently articulated as conventional wisdom repeated in this report that “the choice to select 

a school with a rural track is likely to be influenced by an affinity to rural healthcare,” to my knowledge this 

statement has not been proven to be true. Given the adverse odds of medical school admission for many 

applicants, choice of medical school is often limited by where an individual gets admitted. Many applicants may 

have chosen to go elsewhere if they had been able to freely choose, but many do not have that luxury. The most 

frequently cited example of this, whether true or not, is among osteopathic students, some of whom admit they 

would have preferred to go to an allopathic institution and yet many go into rural practice.

Very insightful comment. We have added 

more detail to the type of study that should 

be conducted in the future research needs 

section to at least partially address this point. 

This review also lends support for scaling up efforts such as those reported by Patterson et al, that seek to follow 

unit record data over a career and use geography (GIS; geocoding of education and training experiences, as well 

as “lived experience” in a rural place – building a web of relationships, i.e. experiential place integration) as an 

examined factor in career decisions and retention in practice over time. The qualitative study by Hancock et al 

(Evidence Table 3) is the only study I found in your references that addresses this in any depth. Unfortunately, that 

study does not address specific interventions, such as deliberate rural placement in education and training.

If it’s not too late, here is an important addition to the literature that is relevant to KQ2:

Wendling AL; Phillips J; Short W; Fahey C; Mavis B. Thirty Years Training Rural Physicians: Outcomes From the 

Michigan State University College of Human Medicine Rural Physician Program, Academic Medicine, just published 

ahead of print September 2015.

The article by Wendling was published after 

the search date for our systematic review; 

however, for the interested reader we have 

added a reference to the paper to the 

discussion section.

KQ3: What interventions have been shown to increase rural healthcare provider recruitment?

KQ4: What interventions have been shown to increase rural healthcare provider retention?

Unless geography (“place”-ment) is considered an intervention (see discussion above), I agree that the evidence 

base for any one intervention in recruitment and retention is painfully limited. One question that needs to be 

answered is, “For those loan repayment individuals who remained in a rural place, is there evidence for the effect of 

duration in a rural place independent of other predictive factors?”

We have added this interesting point to the 

future research section.

I was surprised that Community Apgar did not make it into the review. The Community Apgar project is specifically 

designed to answer KQ3 and KQ4, and I am curious as to why it was excluded. It does identify modifiable factors 

important to recruitment and retention. Although published in the international J Rural & Remote Health, it clearly 

reports on work in Idaho. This instrument has now been used successfully in multiple States to identify factors that 

increase retention. Unfortunately, although presented in multiple forums over the past 4 years, I am unaware that 

these more recent results have yet been published in the peer-reviewed literature.

Schmitz D; Baker E; Nukui: Epperly T. Idaho rural family physician workforce study: the Community Apgar 

Questionnaire. Rural Remote Health 2011;11(3):1769. Epub 2011 Jul 25.

We restricted the review to evaluations of 

specific interventions and the reference is an 

analytic study assessing individual variables. 

We have added the study to the future 

research section for suggestions for 

potentially promising interventions that 

should be tested.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

132

KQ5: What is the efficacy of current rural-specific resident and healthcare professions’ student training and 

education efforts? 

I’m not sure why this article was excluded from KQ5, because it does compare RTT programs with the other family 

medicine residencies in NM:

M. Pacheco, D. Weiss, K. Vaillant, S. Bachofer, B. Garrett, W. H. Dodson, 3rd, C. Urbina, B. Umland, D. Derksen, 

W. Heffron and A. Kaufman. The impact on rural New Mexico of a family medicine residency. Acad Med. 2005; 

80:739-44

The study was outside the scope of the 

intervention because it reported on data 

before 2005.

There is a lack of comparative studies across a variety of settings or even all settings, and across medical students 

or residents in any one region or in any one specialty. It is very important that comparative effectiveness be 

demonstrated, and unfortunately, studies using case-control methods and/or multivariate analyses are difficult to 

find. The database from which to run such queries is still quite limited in capacity and attention to geography. 

Encouraging developments in this regard are (1) the “RTT Masterfile” referenced in Patterson et al and (1) the 

NRHA Rural Medical Educator group’s developing project with the Data Commons, both of which seek to create a 

database of unit record data, including place of education that should be able to address questions of comparative 

effectiveness. The accrediting bodies of both medical school and residency have not generally kept geographically 

relevant data, choosing with regard to their database structure, whether by intention or simply omission, to be 

agnostic of place.

To address this comment we have 

highlighted this in the future research section. 

CONCLUSIONS (From the end of the report)

•All included studies reported current unmet provider needs that worsen with increasing rurality. The small number 

of studies estimating future need also predicted unmet provider needs that worsen with increasing rurality.

Justified.

•Growing up in a rural community is the most consistent factor associated with practice location choice. More 

research into the relative importance of factors is needed.

Except for context of education and training – that seems to be the most consistent factor across KQ2 through KQ5, 

and is explicitly identified in multiple studies (e.g. Hancock, Patterson)

We have expanded the conclusion section 

and provided more detail to address this 

comment. 

•More research is needed to evaluate healthcare provider recruitment interventions for rural healthcare.

Justified.

•There is a lack of evidence regarding interventions to support healthcare provider retention in rural healthcare.

Justified.

•Current evaluations of rural training programs for medical students and residents suggest a median success rate of 

53%.

Hidden in the median statistic is the variability associated with duration of education and training in a rural place 

(the lens of geography and place, as opposed to educational program or effort). As well as the intensity of training 

(multiple rural locations over years of training compared to a single 4 week experience,) and its broad impact. 

Although the metanalysis has not been done, and the data is incomplete (geocoding is not a strength of medical 

school or residency data), there is a consistent theme among studies across KQ2 through KQ5 that warrants further 

exploration.

To address this point we have added an 

analysis stratified by intensity operationalized 

as more than 6 months cumulative time 

spent in rural locations. 

Except for the 3 references noted above (one of which has only appeared in the past month), I commend the team 

for identifying and appropriately vetting a comprehensive list of relevant literature.

Thank you.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

133

This is comprehensive and needed report on the state of rural healthcare with respect to provider demand, provider 

geographic choices, strategies to increase provider recruitment and retention, and the success of approaches to 

increase students choosing to practice in rural areas. Unfortunately, the synthesis did not reveal a large body of 

evidence in these areas, pointing to the need for targeted research examining successful approaches and specific 

workforce projections. 

I find it very interesting that the strongest association with rural practice is growing up in a rural community, 

measured by various proxies. One wonders if interventions to attract and retain other providers are futile given the 

fact that rural practice may be in part of one's psyche. Targeting providers from all disciplines who have spent early 

years in rural settings seems a logical approach (e.g., pipelines from colleges and universities in rural communities 

to professional schools with rural emphases, pipeline programs for high school students in rural schools). As the 

report notes in its Research Gaps/Future Research section on page 71, multi-variate analyses which 

simultaneously study the effects of personal background (e.g., rural upbringing, gender, SES), training needs and 

interventions are needed to determine the relative importance of different factors. 

Another glaring gap noted by the report is the need for studies examining factors associated with non-physicians 

practicing and staying in rural practice. There is a small body of literature pointing to the need for professional 

communities of practice, continuing professional education, and mentoring programs which are not addressed by 

distance education. Rural providers have reported that they leave rural practice because of isolation and lack of 

professional colleagues. 

Thank you, very insightful comments.

The report also reveals the need for VA-specific studies as currently none exist. However, I would draw your 

attention to the Tumosa et al. paper noted above, Health care workforce development in rural america: when 

geriatrics expertise is 100 miles away. The VA Office of Rural Health supports the Geriatric Scholars Program, a 

multi-modal education intervention aimed at bringing geriatrics knowledge and skills to rural VA providers. The 

program has found to have impacted geriatric competencies pre and post-education. I think the review would be 

remiss not to refer to this program.

This is an interesting paper but does not 

report on our outcomes of interest; we have 

added the reference to the discussion on 

provider satisfaction with programs.

I guess it is the nature of an evidence-based synthesis, but the paper, while a strong technical piece, lacks some 

texture in that it does not draw on published work on rural practice and the challenges of rural practice. For 

instance, there is an excellent manuscript by Chipp et al (#31 in the reference list of the report) which reflects on 

challenges of rural practice through the voices of practitioners. If possible, some of this qualitative data in the 

summary or introduction would give this generally excellent review a bit more "life" and underscore the need and 

challenges of rural practice. 

We have added some more detail to the 

introduction and the discussion to highlight 

the challenges faced by rural practitioners. 

Specific comments: 

P. 2, line 42: the tense needs to be changed from future to past tense

Corrected

P. 3, line 6: perhaps there should be a bit of background on the GRADE approach for readers unfamiliar with this 

method

We have expanded the description and 

added a reference

P. 6, line 40: please define "contemporary context" Added as suggested

P. 7, Topic Development: Who determined the synthesis' key questions? Added as suggested

Title of report should be ' Rural Healthcare Workforce: A Systematic Review'. Thank you so this suggestion, we agree and 

have added the term

Page 1, line 13 change to " increase rural provider"; Changed

Page 4 line 3 can you elaborate on type of sponsoring facility? We have added more information but the 

information in the original article was very 

limited. 
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

134

page 11, line 7 "what qualifies as a training site?"; We have added a definition to the methods 

section 

page 23 page 8 why did you not included unpublished VA data on workforce? Please see above

page 68 line 6 "what constitutes small?" provide number of studies; Added

page 68 line 59 - spell out HRSA; Revised

page 6 lines 6-8 can this point be elaborated in discussion? Added as suggested

Page 5 lines 51 -53 can the international literature be elaborated on further? Those approaches could work in VA. The existing studies are described in detail in 

the discussion (page 69-70)

Page 6 lines 26-28 can the new research recently published be further elaborated? These are the studies included in our report. 

They are described under key question 5, 

evidence table 6.

Page 71 - Future research - gaps - Can we specifically say we need further research on the following: 

1). Factors impacting rural workforce in VA. 

2). Simpler models to predict supply and demand for a range of health professions in a given geographic area, 

3) How US and state policies affect supply and demand of health professionals, 

4) How and where technology can best ameliorate shortages of providers, 

5) how and where new models of care can best ameliorate shortage of providers; 

Thank you for these suggestions, we have 

added points 1, and 3-5 to the section; 

regarding 3) we are not sure that simpler 

models are possible; as shown, there are a 

number of factors contributing to predictions

Page 71 line 32, can this be elaborated? how easy access to CME opportunities, e- consults with specialists, mini 

residencies, provider education and consult networks impact retention of primary care providers in rural

Without empirical evidence it is difficult to 

speculate; to address this point w have 

added the topics to the future research 

section.

Since many if not most of the readers of this report will only read the executive summary, it’s important to explicitly 

state what has been found by this very well done, thorough review of the literature.

The Discussion section should be expanded to state explicitly what has been found by the review. It’s probably true 

that more research would be helpful, but as a policy maker, that does not help determine where to put resources in 

2016. The stated findings and conclusions should point to what the current literature--the known body of evidence--

points us to, especially where possible enhancement of policies appear to be helpful (e.g. rural training tracks vs

loan repayment programs) to make a difference in this decade.

We have expanded the discussion as 

suggested

International interventions are mentioned but without reference, and in fact in Discussion by Key Question Section, 

for KQ3, international references make up most of the discussion for KQ3. Should explain use/non-use of 

international references/models.

The discussion section places the identified 

research in context of the international 

literature.

(not all publications have been received at the time of the draft report)

Presumably these publications will have minor impact, but would be useful to know which ones were not included in 

the analysis.

The studies have been added to the final 

report.

ES, KQ1: It seems reasonable that this is the main finding, yet the lead sentence defines a limitation Duly noted but given the scope of KQ1 this 

seems justified 

Again, it seems that the important finding of the literature review is that ALL reports state that supply is not met, and 

that with increasing rurality the demand increases. It would also seem reasonable to choose a metric to compare 

current or anticipated need, e.g. physicians/population or providers/county, etc.

We have added information on the variability 

of the reported metrics to address this 

comment.
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

135

ES, KQ2: I didn’t see any cited studies that programs did NOT improve likelihood. Seems reasonable to make a 

stronger declarative statement that is supported by a large # of studies, rather than couching as “seems to increase 

likelihood.”

We have added a discussion of the study 

limitations to the discussion section to 

address this point

It would be helpful to quantify the number of physicians. Less than half completed and of those are large 

percentage stayed longer: How many and how much longer? This is important because the J-1 visa program is 

well known and policy makers need to know more about its impact.

Added

Can you define the difference between recipients and program completers? Added as suggested

May point to an important policy issue that needs further discussion, which is that there has not been high quality 

research conducted about provider retention. Retention is obviously a priority the need to understand retention 

factors is also a priority. How about NHSC retention rates as a model?

Thank you for this interesting comment.

It would be helpful for the authors to expand this discussion for uninformed readers as to why rural healthcare 

needs are more complex than metropolitan or urban needs. What are the factors that determine complexity? Am 

not clear on why the discussion focuses mostly on what is not included in this very thorough review rather than on 

what the literature demonstrates.

We have revised the wording and expanded 

the introduction to address this point

Conclusion KQ1: I think you mean population needs We have reworded the sentence to make it 

clearer that studies reported unmet 

healthcare provider needs

Conclusion KQ3: With the average U.S. medical school graduation rate of entering rural practice at about 5%, (even 

the AAMC (Shipman) study of new schools it peaks at only 8+%), there appears to me that there is good evidence 

as cited in bullet #5 that rural training programs are successful. Though more research may be needed, there are 

findings that could be highlighted, e.g. the rather poor success of J-1 visa programs, and +/- success of loan 

programs that have been somewhat central in recruitment/retention efforts over last several decades. 

We acknowledge the point but to address it 

we have expanded the discussion regarding 

the lack of comparative effectiveness studies 

to provide more information regarding the 

interpretation of results.

Would be helpful to compare/contrast what the literature shows for these 3 interventions, e.g. in a table format. To address this point we have highlighted 

that Table 5 provides an overview of studies 

and evaluated programs

Introduction normative, coercive, utilitarian: An interesting way to sort the strategies. Not evident, based on 

discussion and conclusions, that this approach was used in the analysis or conclusions.

The small number of provider interventions 

unfortunately did not allow meaningful 

stratification 

Intro, Evidence syntheses are sparse and care environment has changed: Not sure what this means Reworded for clarification 

Intro, AAMC call for 30% increase:-Is there evidence that increased production of medical graduates has focused 

on increasing rural education?

Only one included study assessed this 

question and it concluded that despite 

expansion, the characteristics of 

matriculating medical students changed little, 

except at new schools. 

The Methods section is very nicely done and self-explanatory. Clearly written. The quality assessment exercises 

and rating the body of evidence are particularly helpful.

Thank you.

KQ1, we did not identify studies reporting on the same provider group: Please clarify. Unclear what this means. Revised to clarify

Evidence tables: These tables are very helpful to understand the content of the literature as well as the review 

process. 

Table 4: This table is very useful in understanding and synthesizing the strength of the literature.

Thank you
Rural Healthcare Workforce: A Systematic Review Evidence-based Synthesis Program

136

KQ2, most rural healthcare providers did not grow up in a rural community: It would be helpful to at least postulate, 

not necessarily in the results, but possibly in the discussion section, what factors may be at play that influenced 

those from non-rural backgrounds to choose a rural practice career.

Added as suggested

It seemed as though a number of the recruitment publications also include retention information, e.g. Rabinowitz, 

2013 who assessed “PSAP” graduates from 1978 – 1986 to assess continuation of rural practice. Unclear why the 

inclusion criteria for this KQ is limited to studies that only include retention data, or why retention data cannot be 

gleaned from studies that incorporate both recruitment and retention intervention outcomes. That the only finding 

for this KQ relates to the J-1 program is surprising and limits the effectiveness of the report.

To address this point we have clarified 

throughout that we didn’t find interventions

that focused on retention in fully trained 

providers practicing in rural care (rather than 

trainees). We have also added information 

on retention reported in studies addressing 

students.

The referenced figure (not shown in this Word version due to formatting incompatibility), needs clarification. Axes 

are not well labeled...Is the Y axis # of studies or interventions? Is the X axis the number of participants or school 

graduates? A helpful table would be to report %/# entering rural practice of intervention group c/w %/# without 

intervention.

We have added a note section to the figure 

to address this point

Would be helpful to cite comparison percentage from slow/low growth schools. Added as suggested 

This statement seems to contradict prior statement. “Hence it is difficult to make specific evidence statements for 

the number of healthcare providers needed across the studies.” Doesn’t the statement that there is a shortage imply 

a known or estimated quantity/metric that would alleviate the shortage?

We have clarified the wording to address this 

point

The review is thorough and the data extraction, along with quality assessments are very nicely done. My concern is 

that the report itself has not synthesized the information so that the reader comes away with new knowledge about 

what has been described and reported in the literature. Concluding that the problem is “complex” and more 

research is needed is in all likelihood accurate, however after reviewing almost 450 publications and thoroughly 

extracting 56, I would expect more declarative statements that include what the extracted information tells us.

More declarative statements are hampered 

by the number of studies contributing to 

some of the KQs and limitations in study 

designs; we feel we have made our 

statement sa strong as the evidence will 

support.

p. 4, line 30: Is this report only targeting MDs? what about nursing, pharmacists, psychologists, etc.? It appears 

other professions are not really mentioned throughout the report.

Nurse practitioners and physician assistants 

were also included but not pharmacist or 

psychologists; we have added this point to 

the limitation section

p. 10, line 18: Acronym should be spelled out when first mentioned as opposed to later in the document. Revised

p.24, line 55: Is there not more data or information to include here to support this statement? No, abstracted as reported