Increasing access to health workers in remote and rural areas through improved retention: global policy recommendations. 1.Health personnel. 2.Delivery of health care. 3.Rural health. 4.Rural health services. I.World Health Organization.#
The mention of specific companies or of certain manufacturers' products does not imply that they are endorsed or recommended by the World Health Organization in preference to others of a similar nature that are not mentioned. Errors and omissions excepted, the names of proprietary products are distinguished by initial capital letters.
All reasonable precautions have been taken by the World Health Organization to verify the information contained in this publication. However, the published material is being distributed without warranty of any kind, either expressed or implied. The responsibility for the interpretation and use of the material lies with the reader. In no event shall the World Health Organization be liable for damages arising from its use.

# Printed in France
Graphic design : Atelier Rasmussen / CH 2010 i
Half the world's people currently live in rural and remote areas. The problem is that most health workers live and work in cities. This imbalance is common to almost all countries and poses a major challenge to the nationwide provision of health services. Its impact, however, is most severe in low income countries. There are two reasons for this. One is that many of these countries already suffer from acute shortages of health workers -in all areas. The other is that the proportion of the population living in rural regions tends to be greater in poorer countries than in rich ones.
The World Health Organization (WHO) has therefore drawn up a comprehensive set of strategies to help countries encourage health workers to live and work in remote and rural areas. These include refining the ways students are selected and educated, as well as creating better working and living conditions.
The first step has been to establish what works, through a year-long process that has involved a wide range of experts from all regions of the world. The second is to share the results with those who need them, via the guidelines contained in this document. The third will be to implement them, and to monitor and evaluate progress, and -critically -to act on the findings of that monitoring and evaluation.
The guidelines are a practical tool that all countries can use. As such, they complement the WHO Global Code of Practice on the International Recruitment of Health Personnel, adopted by the Sixty-third World Health Assembly in May 2010.
The Code offers a framework to manage international migration over the medium to longer term. The guidelines are a tool that can be used straight away to address one of the first triggers to internal and international migration -dissatisfaction with living and working conditions in rural areas.
Together, the code of practice and these new guidelines provide countries with instruments to improve workforce distribution and enhance health services. Doing so will address a long-standing problem, contribute to more equitable access to health care, and boost prospects for improving maternal and child health and combating diseases such as AIDS, tuberculosis and malaria.

# Executive Summary
Why these recommendations? Policy-makers in all countries, regardless of their level of economic development, struggle to achieve health equity and to meet the health needs of their populations, especially vulnerable and disadvantaged groups. One of their most complex challenges is ensuring people living in rural and remote locations have access to trained health workers. Skilled and motivated health workers in sufficient numbers at the right place and at the right time are critical to deliver effective health services and improve health outcomes. A shortage of qualified health workers in remote and rural areas impedes access to health-care services for a significant percentage of the population, slows progress towards attaining the Millennium Development Goals and challenges the aspirations of achieving health for all. The World Health Organization (WHO) has produced these recommendations in response to requests from global leaders, civil society and Member States.

# What is the scope?
The evidence-based recommendations relate to the movements of health workers within the boundaries of a country and focus solely on strategies to increase the availability of health workers in remote and rural areas through improved attraction, recruitment and retention. As such they complement the current work of WHO on the Global Code of Practice on the International Recruitment of Health Personnel (see Annex 3). The recommendations apply to all types of health workers in the formal, regulated health sector, including health managers and support staff, as well as to students aspiring to or currently attending education programmes in health-related disciplines.

# What are the specific recommendations?
It is important to stress that there is much more helpful detail in the body of the report and that the best results will be achieved by choosing and implementing a bundle of contextually relevant recommendations.
A. EDUCATION RECOMMENDATIONS 1. Use targeted admission policies to enrol students with a rural background in education programmes for various health disciplines, in order to increase the likelihood of graduates choosing to practise in rural areas.
- Locate health professional schools, campuses and family medicine residency programmes outside of capitals and other major cities as graduates of these schools and programmes are more likely to work in rural areas.
- Expose undergraduate students of various health disciplines to rural community experiences and clinical rotations as these can have a positive influence on attracting and recruiting health workers to rural areas. 4. Revise undergraduate and postgraduate curricula to include rural health topics so as to enhance the competencies of health professionals working in rural areas, and thereby increase their job satisfaction and retention.
- Design continuing education and professional development programmes that meet the needs of rural health workers and that are accessible from where they live and work, so as to support their retention.
B. REGULATORY RECOMMENDATIONS 1. Introduce and regulate enhanced scopes of practice in rural and remote areas to increase the potential for job satisfaction, thereby assisting recruitment and retention.
- Introduce different types of health workers with appropriate training and regulation for rural practice in order to increase the number of health workers practising in rural and remote areas.
- Ensure compulsory service requirements in rural and remote areas are accompanied with appropriate support and incentives so as to increase recruitment and subsequent retention of health professionals in these areas. 4. Provide scholarships, bursaries or other education subsidies with enforceable agreements of return of service in rural or remote areas to increase recruitment of health workers in these areas.
C. FINANCIAL INCENTIVES RECOMMENDATION 1. Use a combination of fiscally sustainable financial incentives, such as hardship allowances, grants for housing, free transportation, paid vacations, etc., sufficient enough to outweigh the opportunity costs associated with working in rural areas, as perceived by health workers, to improve rural retention.
D. PERSONAL AND PROFESSIONAL SUPPORT RECOMMENDATIONS 1. Improve living conditions for health workers and their families and invest in infrastructure and services (sanitation, electricity, telecommunications, schools, etc.), as these factors have a significant influence on a health worker's decision to locate to and remain in rural areas.
- Provide a good and safe working environment, including appropriate equipment and supplies, supportive supervision and mentoring, in order to make these posts professionally attractive and thereby increase the recruitment and retention of health workers in remote and rural areas.
- Identify and implement appropriate outreach activities to facilitate cooperation between health workers from better served areas and those in underserved areas, and, where feasible, use telehealth to provide additional support to health workers in remote and rural areas.
- Develop and support career development programmes and provide senior posts in rural areas so that health workers can move up the career path as a result of experience, education and training, without necessarily leaving rural areas.
- Support the development of professional networks, rural health professional associations, rural health journals, etc., in order to improve the morale and status of rural providers and reduce feelings of professional isolation.
- Adopt public recognition measures such as rural health days, awards and titles at local, national and international levels to lift the profile of working in rural areas as these create the conditions to improve intrinsic motivation and thereby contribute to the retention of rural health workers.

# What principles should guide the formulation of national rural retention strategies?
A number of interconnected principles should underpin all efforts to improve the recruitment and retention of health workers in remote and rural areas. Adhering to the principle of health equity will help in allocating available resources in a way that contributes to the reduction of avoidable inequalities in health. And grounding rural retention policies in the national health plan will provide a framework for holding all partners accountable for producing tangible and measurable results.
The choice of interventions should be informed by an in-depth understanding of the health workforce. This requires, at a minimum, a comprehensive situation analysis, a labour market analysis, and an analysis of the factors that influence the decisions of health workers to relocate to, stay in or leave rural and remote areas. Giving due consideration to the broader social, economic and political factors at national, subnational and community levels that influence retention will help to ensure the choice of policy interventions are anchored in and tailored to the specific context of each country.
Assessing options and championing interventions to improve rural retention of health workers will require human resource management expertise at the central and local levels, while implementation of the chosen policies will require individuals with strong management and leadership skills, especially at the facility level. Engagement of stakeholders across several sectors is a critical element for the success of rural retention policies, as it is for any type of health system or health workforce policy. Rural and remote communities, professional associations and other relevant decision-makers must be included from the beginning to obtain and maintain the support of all involved.
A commitment to monitoring and evaluation and to operational research is essential in order to evaluate effectiveness, revise polices as necessary once implementation is underway, capture valuable lessons learnt, build the evidence base, and improve understanding about how interventions work and why they work in some contexts but fail in others.

# How to select and evaluate the interventions?
As in many areas of health systems policies, sound evaluations of rural retention interventions are lacking. In order to support the needed paradigm shift towards more and better evaluations, this report proposes a framework and five questions to guide policy-makers in the selection, design, implementation and monitoring and evaluation of appropriate rural retention interventions. The framework specifies the dimensions on which effects of retention strategies can be measured: attractiveness, recruitment, retention and health workforce and health systems performance. One of the challenges in evaluation is that each recommendation has more than one outcome (or effect), and no outcome can be achieved through only one intervention. This complexity adds to the task of measuring the results and attributing the achieved effects to specific interventions.

# How were the recommendations formulated?
The WHO Secretariat convened a gender-balanced group of experts comprised of researchers, policy-makers, funders, representatives of professional associations and programme implementers, drawn from each of the WHO regions. The expert group was asked to examine existing knowledge and evidence and to provide up-to-date, practical guidance to policy-makers on how to design, implement and evaluate strategies to attract and retain health workers in rural and remote areas.
The recommendations were developed following a comprehensive review of all relevant and available evidence related to health workforce attractiveness, recruitment and retention in remote and rural areas. They have also been informed by country experiences and judgements of the experts, who met six times between February 2009 and February 2010. The expert group considered that in this field it is equally important to understand whether an intervention works or not (effectiveness) and also why it works and how. Context may be responsible for different outcomes or results from the same intervention and thus needs to be better captured in the research on these interventions.
All efforts were made to comply with standards for reporting, processing and using evidence in the production of WHO guidelines as required by the Organization's Guidelines Review Committee (GRC). This includes using a system for assessing evidence for interventions known as GRADE (Grading of Recommendations Assessment, Development and Evaluation) and presenting the quality of the evidence in the GRADE format. But, because of the richness of the information in this field, particularly with regard to the mechanisms that make interventions work, the expert group decided to supplement the GRADE approach with additional evidence.
Various supporting materials are being published by the WHO Secretariat alongside this document.
- Annexes on CD-ROM include details of all the evidence that was used in developing the recommendations (GRADE evidence profiles, descriptive evidence tables).
- Several papers that informed the development of this report were published in May 2010 in a special them issue of the Bulletin of the World Health Organization.
- Three commissioned reviews have been published by WHO: a review of the impact of compulsory service on the recruitment and retention of health workers in rural areas; a "realistic" evaluation that sought to understand not only whether certain interventions worked or not, but also why and how; and a review of the role of outreach support on the recruitment of health workers in remote and rural areas.
- A series of comprehensive country case studies are being published including reports from Australia, China, Ethiopia, Lao People's Democratic Republic, Mali, Norway, Samoa, Senegal, Vanuatu and Zambia.

# What's next?
The document is available in print, on the WHO website and on CD-ROM, and it will be circulated through WHO channels for adaptation and implementation at country level. It will also be translated and subsequently disseminated. Some countries, including the Lao People's Democratic Republic and Mali, are already considering these recommendations to inform the design of their retention strategies, with the WHO Secretariat providing technical assistance, as required. In addition, several members of the expert group are leading a research effort to fill some of the evidence gaps that have emerged through the development of this document.
The recommendations are expected to remain valid until 2013. The Health Workforce Migration and Retention Unit in the Department of Human Resources for Health at WHO headquarters in Geneva will be responsible for initiating a revision of these global recommendations by that time, based on new evidence and research and feedback from countries that have been using the recommendations. The possibility to expand the scope of the recommendations, for example, to include recruitment and retention strategies for all underserved areas, shall also be considered.

# World Population
Nurses worldwide  Physicians worldwide  1. Introduction
Policy-makers in all countries, regardless of their level of economic development, struggle to achieve health equity and to meet the health needs of their populations, especially vulnerable and disadvantaged groups. One of their most complex challenges is ensuring people living in rural and remote locations have access to trained health workers. Skilled and motivated health workers in sufficient numbers at the right place and at the right time are critical to deliver effective health services and improve health outcomes. Insufficient numbers and types of qualified health workers in remote and rural areas impedes access to health-care services for a significant percentage of the population, slows progress towards attaining the Millennium Development Goals and challenges the aspirations of achieving health for all.
This is a global problem that affects almost all countries. Approximately one half of the global population lives in rural areas, but these areas are served by only 38% of the total nursing workforce and by less than a quarter of the total physician workforce (see Figure 1.1). The situation is especially dire in 57 countries where a critical shortage of trained health workers means an estimated one billion people have no access to essential health-care services (1). In Bangladesh, for example, 30% of nurses are located in four metropolitan districts where only 15% of the population lives (2). In South Africa, 46% of the population lives in rural areas, but only 12% of doctors and 19% of nurses are working there (3). To compound the problem, in some francophone sub-Saharan African countries, like Cote d'Ivoire, the Democratic Republic of the Congo and Mali, the overproduction of health workers relative to the capacity for absorption has led to medical unemployment in urban areas and shortages in rural areas (4).
Even high-income countries have shortages of health workers in remote and rural areas. In the United States of America (USA), 9% of registered physicians practise in rural areas where 20% of the population lives (5). France has large inequalities in the density of general practitioners, with higher densities in the south and the capital compared with the centre and north of the country (6). And in Canada, only 9.3% of physicians work in remote and rural areas where 24% of the population lives (7). Every government influences the health labour market through regulation, financing and information. An entirely free labour market will never lead to a well-distributed health workforce because many people are drawn to urban centres or in some cases to other countries.
An abundance of evidence and experience shows that political commitment and policy interventions are central to more equitable health workforce distribution. In countries as diverse as China, Cuba and Thailand, a variety of long-standing commitments towards the education, training and specific support of rural health workers have led to improvements in the population's access to committed health workers in these areas (8-10).
However, no country has completely solved these challenges; hence several recent international events have highlighted the importance of improving health worker retention and called for more effective policy interventions (see Box 1).

# Box 1. International calls to action
- The World Health Assembly resolutions on migration in 2004 and rapid scaling up of health workers in 2006 both requested Member States put in place mechanisms to address the retention of health workers.
- In March 2008, the Kampala Declaration from the First Global Forum of Human Resources for Health requested governments to "assure adequate incentives and an enabling and safe working environment for effective retention and equitable distribution of the health workforce".
- The G8 Communique of July 2008 restated the need to assure the effective retention of health workers.
- The November 2008 report from the Commission on Social Determinants of Health urged action by governments and international partners to specifically address the imbalances in the geographical distribution of health workers in rural areas as a structural determinant of poor health outcomes.
- In June 2009 the high-level Taskforce on Innovative International Financing for Health urged all governments to ensure that all people, including rural and remote populations, have access to safe, high-quality and essential health-care services.
The World Health Organization (WHO) responded to calls to action from global leaders, civil society and Member States by convening a group of experts to examine existing knowledge and evidence and to provide up-to-date, practical guidance to policy-makers on how to design, implement and evaluate strategies to attract and retain health workers in rural and remote areas.
In doing so, these recommendations support countries in their efforts to improve health outcomes by strengthening the capacity of health systems to provide quality health care that is accessible, responsive, effective, efficient and equitable.

# Target audience
This report emphasizes that sustained political, institutional and financial commitments are needed, as is the involvement of many different stakeholders. As such, this report is aimed at government leaders and national policy-makers across several sectors including health, finance, education, labour and public service. Stakeholders include health system managers, human resource managers, heads of education and training institutions, employers of health workers, professional associations representing different cadres of health workers, civil society, nongovernmental organizations and remote and rural communities.
This is the first time that global recommendations have been published on this important issue. This report builds on work that has already been done in human resources for health, including the Joint Learning Initiative Report (11) The recommendations focus solely on strategies to increase the availability of motivated and skilled health workers in remote and rural areas through improved attraction, recruitment and retention of health workers in these areas. They become relevant once a country has assessed the health needs of its population, has planned and projected the future needs for health workers, and is at the point of considering strategies for their production, distribution and retention.
At the same time, a variety of other factors might impede people's access to health services in rural or remote areas, including socioeconomic deprivation, geographical barriers and distance, transport, telecommunications, the cost of accessing services and the acceptability of services. Efforts to address these factors may also influence the availability of health workers in rural and remote areas.

# Types of health workers targeted
The recommendations apply to all types of health workers in the formal, regulated health sector (public and non-state), as well as to students aspiring to or currently attending education programmes in health-related disciplines. This includes health-care providers (doctors, nurses, midwives, mid-level health workers, pharmacists, dentists, lab technicians, community health workers, etc.) as well as managers and support workers (human resource managers, health managers, public health workers, epidemiologists, clinical engineers, teachers, trainers, etc.).

# Geographical areas covered
These recommendations are specifically aimed at remote and rural areas as opposed to all underserved areas. This is in part because their geographical situation requires specific interventions and because addressing rural and remote areas will also address the needs of underserved populations more broadly, but not vice versa.
Underserved areas are geographical areas where populations have limited access to qualified health-care providers and quality health-care services. They include remote and rural areas, small or remote islands, urban slums, conflict and post-conflict zones, refugee camps, minority and indigenous communities, and any place that has been severely affected by a major natural or man-made disaster. When the recommendations are revised the geographical scope could be expanded to include other underserved areas, if deemed necessary by the expert group and countries.
There are no precise universal definitions for "urban areas" and "rural areas". According to the United Nations, the distinction between urban and rural population is not amenable to a single definition that would be applicable to all countries because of national differences in the characteristics that distinguish urban from rural areas (16).
Each country's own definition for these terms generally takes into account two main elements: the settlement profile (population density, availability of economic structures) and the accessibility from an urban area (distance in kilometres or hour's drive).
For the purpose of these recommendations, "rural areas" are areas that are not urban in nature (17).
An urban area usually incorporates the population in a city or town plus that in the suburban areas lying outside of -but being adjacent to -the city boundaries (18).

# Categories of interventions covered
Although there are other ways of increasing the access of populations living in remote and rural areas to adequate health services, for example through different models of service delivery, or through internationally recruited health workers, these recommendations focus only on interventions that are within the remit of human resources planning and management. The four main categories of interventions are:
a) education b) regulation c) financial incentives d)
personal and professional support.
Detailed descriptions of the recommended interventions are provided in Chapter 3.

# Process for formulating the global recommendations
An initial literature search was conducted by WHO in 2008, and a background paper was prepared for the first meeting of the expert group in February 2009. In selecting the members, careful consideration was given to achieving a gender balanced group, with representation from all WHO regions and relevant constituencies (policy-makers, academics, funders, professional associations and rural health workers). Members of the expert group are listed on pages 66-68.
The WHO background paper provides a comprehensive review of the current thinking and evidence in this area and highlights significant knowledge gaps (19). The experts used the background paper to agree on the research questions to be addressed by this report, and on the four categories of interventions. During their first meeting, they also finalized a plan of action to further supplement the evidence base, and some of the experts self-selected into a "core" expert group to undertake the additional systematic research needed. Subsequent expert consultations (two of the core group in April and October 2009, and two of the full expert group in June and November 2009) discussed the results of the additional research and proposed draft recommendations. During these consultations, members of the core expert group provided initial text for the recommendations, which were subsequently revised by the WHO Secretariat (20).
The revised draft recommendations were presented to policy-makers, academics and other stakeholders from 15 Asian countries and eight African countries during a regional workshop in November 2009 in Hanoi, Viet Nam (21). Participants had the opportunity to discuss their experiences and challenges in improving rural and remote retention and to comment on the draft recommendations.
The experts met for the final time in February 2010 to discuss again the draft recommendations, particularly with a view to rank the recommendations based on the quality of the evidence, benefits, values, and resource use. Balance worksheets were prepared for each recommendation, containing the factors taken into account in ranking the recommendations. Follow-up was done by e-mail with the core group on the final evidence tables and on the revised balance worksheets for each recommendation. The WHO Secretariat incorporated the experts' inputs and finalized the report.
Several papers that informed the development of this report were published in May 2010 in a special theme issue of the Bulletin of the World Health Organization, a peer-reviewed journal (22). In addition, two experts were commissioned to write reports on compulsory service schemes and outreach services in order to review and analyse available evidence related to these specific recommendations. Another expert conducted a "realist review" of a selection of retention studies with the aim of better understanding the influence of contextual factors and the mechanisms that make interventions work or fail. Comprehensive country case studies were also conducted in Australia, Ethiopia, the Lao People's Democratic Republic, Mali, Norway, Samoa, Senegal, Vanuatu and Zambia in order to understand country specificities and to share lessons learnt. These reports and country case studies were a significant contribution to the evidence base for these recommendations and will all be published as standalone documents and will be accessible online at: http:// www.who.int/hrh/resources/.

# Dissemination process
The document will be printed and made available on the WHO web site, as well as on CD-ROM, and will be circulated through WHO channels for adaptation and implementation at country level. It will also be translated and subsequently disseminated. The recommendations given in this document are expected to remain valid until 2013. The Health Workforce Migration and Retention Unit in the Department of Human Resources for Health at WHO Headquarters in Geneva will be responsible for initiating a review of these global recommendations at that time, based on new evidence and research and feedback from countries that have been using the recommendations. The possibility to expand the scope of the recommendations, for example, to include recruitment and retention strategies for all underserved areas, shall also be considered.
These recommendations were developed following a comprehensive review of all relevant and available evidence related to health workforce attractiveness, recruitment and retention in remote and rural areas. Much of the evidence in this field comes from observational studies, rarely from well-designed cohort studies or before-and-after studies. Unlike clinical medicine, it is quite difficult, if not impossible, to conduct randomized controlled trials to understand the effects of many of the interventions proposed in this document. These are complex interventions with multiple outcomes, and many confounders detract from the design of the interventions, and intervene during the implementation phase. The expert group considered that in this field it is equally important to understand whether an intervention works or not (effectiveness), and also why it works and how. Context is a key element that can be responsible for different outcomes or results from the same intervention and thus needs to be better captured in the research on these interventions.
All efforts were made to comply with standards for reporting, processing and using evidence in the production of WHO guidelines as required by the Organization's Guidelines Review Committee (GRC) (23). This includes using a system for assessing evidence for interventions known as GRADE (Grading of Recommendations Assessment, Development and Evaluation) and presenting the quality of the evidence in the GRADE format.
Because of the richness of the information in this field, particularly with regard to the mechanisms that make interventions work, the expert group felt that a considerable amount of valuable evidence was not being captured by GRADE. As a result, early on in the process of formulating these recommendations the experts decided to supplement the GRADE tables with an additional table to ensure policy-makers had access to summaries of all relevant existing evidence. See pages 62-65 for full details about the methodology for the literature review, additional research, evidence gathering and assessment.

# Structure of the report
This chapter provides the rationale and describes the process for the development of the global recommendations on the retention of health workers in remote and rural areas. The remaining five chapters of the report address what should be done and why, based on an extensive literature review, expert opinion and the consultative process:
- the principles and actions that should guide national strategies to improve retention of health workers in remote and rural areas (Chapter 2)
- the specific recommendations grouped in four main categories (Chapter 3) -education -regulation -financial incentives -personal and professional support
- how to select and evaluate the interventions (Chapter 4)
- the research agenda and action plan (Chapter 5)
- details of the criteria used to rank each of the recommendations presented in Chapter 3 (Chapter 6).
The annexes, which are available on CD-ROM, as well as online, include details of the evidence and information used in the formulation of the recommendations:
- the evidence profiles for the recommendations A1-A5, B1-B3, C1 and D1-D6 (Annex 1)
- a comprehensive table containing descriptive evidence not included in the evidence profiles (Annex 2)
- Resolution WHA63.16 and the WHO Global Code of Practice on the International Recruitment of Health Personnel (Annex 3).

# Principles to guide the formulation of national policies to improve retention of health workers in remote and rural areas
This chapter describes a number of interconnected principles that should underpin all efforts to improve the recruitment and retention of health workers in remote and rural areas. Commitment from policy-makers to all of the actions outlined in this chapter is essential in order for the recommendations presented in Chapter 3 to have any chance of being successful. These principles, together with the framework and questions for monitoring and evaluation presented in Chapter 4 should provide useful tools for policy-makers throughout the process of planning, implementing and evaluating the most appropriate and relevant retention strategies for their own context.

# Focus on health equity
According to the principle of health equity, all citizens should have an equal opportunity to be healthy. However, wide disparities in health status exist within many countries worldwide. Lack of access to quality health-care providers is one of the primary root causes of health inequity and is disproportionately experienced by people living in remote and rural communities. Adhering to this principle will help in the selection of the most effective retention strategies and in allocating available resources in a way that contributes to the reduction of avoidable inequalities in health.
For example, in establishing the number of health workers needed by any given community or population there is some evidence that the health needs of rural populations are greater and thus they would need a proportionately higher number of health workers (1,24).
When compared with their metropolitan counterparts, rural health workers are "extended generalists" who provide a wider range of services and carry a higher level of clinical responsibility in relative professional isolation (25). In the context of large distances, geographical factors, transport links, communications and so forth, small communities in rural or remote areas may require a larger number of generalist health-care providers that would not be justified in an urban context.

# Ensure rural retention policies are part of the national health plan
This is about the principles of alignment and policy coherence at the country level. Rural retention policies must be grounded in a costed and validated national health plan. A national health plan provides the framework for holding all partners accountable for producing tangible and measurable results; it is at the heart of health development that is country-led, country-owned, and fully aligned with national priorities and capacities. A national health workforce plan, which is an integral part of a country's national health plan, sets out the projected numbers and types of health workers needed in the future, the policies and strategies to scale up needed health workers, the strategies to retain and motivate them, and the costs of implementing all the required interventions.
Given that the ultimate goal is to improve health outcomes, it is essential that policy interventions and plans for producing and allocating the most appropriate types of health workers are developed to respond to the health needs, perceptions, expectations and health-seeking behaviours of people living in rural and remote communities.
Any retention strategy should be linked to the broader national and local health system structures and functions, to take advantage of synergies and increase efficiencies. For example, if a country has a national health plan and health sector reforms are under way, there may be an opportunity to prioritize the upgrading of rural health facilities and improve the working environment as part of a national health facility expansion plan. In contrast, a plan to expand public-or private-funded health services in urban areas may work against new strategies for attracting people to work in rural areas.

# Understand the health workforce
Before embarking on any of the recommended interventions, a clear understanding of the health workforce is necessary. This comprises an understanding of the current levels and distribution of health workers by gender, geographical region, sector and speciality. A comprehensive situation analysis and labour market analysis of current and future needs of health workers should be able to identify any potential mismatches between supply and demand factors. For example, it can identify whether large numbers of unemployed health workers are located in urban areas, or whether high remuneration differentials exist between urban and rural areas, and thus can guide appropriate interventions.
A detailed analysis of the factors that influence the decisions of health workers to relocate to, stay in or leave rural and remote areas is a key step in understanding the extent of the problem and in guiding the appropriate choice of interventions. These factors are very complex, as they tend to be related to personal aspects, health system characteristics and the overall social, economic and political environment (see Figure 2). The interplay of these factors is also complex and strongly influenced by the underlying motivation, be this economic, social, cultural, religious, etc (26)(27)(28).

# Understand the wider context
Improving the retention of health workers in remote and rural areas poses a number of complex policy challenges that cannot be tackled within the health sector alone. Broader social, economic and political factors at national, subnational and community levels that influence retention also need to be considered to ensure the choice of policy interventions are anchored in and tailored to the specific context of each country.
Government and civil-service reforms can have positive or negative impacts of retention strategies. For example, a broad public-sector reform programme may strengthen systems for posting and deployment across the public sector. But in Indonesia, one of the consequences of decentralization has been the breakdown of the health personnel information system as decision-makers at the local level thought they were no longer obligated to send data to the upper level. This had consequences on the regular payment of financial incentives and the supervision of rural health workers (30).
Making progress in raising education levels in rural and remote areas and prioritizing infrastructure and services in rural areas (roads, water, sanitation, electricity, telecommunications, etc.) will both improve people's access to health services in remote and rural areas and make working in these areas more attractive to health workers, as well as workers in other public sectors.

# Strengthen human resource management systems
A core basic requirement for any retention strategy to be effective is management capacity. Remote and rural retention strategies need to be grounded in human resource (HR) management systems, which include key components such as workforce planning, recruitment and hiring practices, work conditions, and performance management, as well as competent HR managers able to perform these functions (see Box 2).
HR management within the health sector in many countries is very weak, especially beyond the central level, and this lack of capacity is a major barrier to rolling out successful human resources for health (HRH) interventions. In addition, assessing options and championing interventions to improve rural retention of health workers will require HR management expertise at the central and local levels, while implementation of the chosen policies will require individuals with strong management and leadership skills, especially at the facility level.
Organization capacity is also important, as is the continuity in the mechanism that provides the oversight for implementing the recommendations: for example, sudden changes in administration can result in unclear rules and procedures, which in turn can delay payments of allowances and limit the intervention's effectiveness.
Most countries will need to invest in professional development programmes including training, coaching, mentoring and professional support for a strengthened HR management cadre and capacity at all levels. Many countries will need to initiate or strengthen leadership development programmes to improve supervision capacity in rural areas and create a supportive workplace environment to attract and retain health workers. At the central level, HR managers and policymakers who can engage with stakeholders, analyse and understand their power and interests, and negotiate compromises are crucial to the development of sustainable and feasible HRH strategies, including strategies to improve the retention of health workers in remote and rural areas.

# Box 2. Elements of a strong HR management system
The key functions of an effective HR management system are:
- Personnel: workforce planning (including staffing norms), recruitment, hiring and deployment
- Work environment and conditions: employee relations, workplace safety, job satisfaction and career development
- HR information: data and information for decision-making
- Performance management: performance appraisal, supervision and productivity.
The key component of a strong HR management system is professionally prepared and competent HR managers who are able to perform the HR functions described below.
- Workforce planning: Lead and support processes for effective HRH workforce planning based on sound HR information; promote data-driven decisions; link HR profiles and types of health workers needed to achieve strategic health goals (e.g. decide about issues like task-shifting, re-profiling staff, redistribution, incentives, and so on); align workforce needs with HRH strategic plans; contribute to sound overall HRH strategic planning processes; and support good costing practices so that workforce projections can be budgeted appropriately.
- HR recruitment, hiring and deployment practices: Use their knowledge of effective practices in areas like recruitment and selection, orientation, deployment, staff development and retention to promote positive change in the system by working with policy-makers to identify barriers to effective and efficient recruitment, hiring, deployment, retention, etc. This work of promoting change also links to implementing things like task-shifting, incentive packages, and so on.
- Work environment and conditions: Monitor and support workforce environment practices that contribute to high job satisfaction, including effective employee relations, workplace safety and career development.
- HR information: Integrate information and data sources to ensure timely availability of accurate data required for planning, training, appraising and supporting the workforce.
- Performance management, leadership and staff development: Ensure there is an effective performance appraisal system in place within the health system; lead and support systemic productivity improvement interventions; use knowledge of up-to-date approaches to leadership and management to promote good practices; assess the "state" of leadership and management within the system, and organize or champion improvement programmes as needed; in general, make sure health staff have the right competencies to do whatever they are required to do.

# Engage with all relevant stakeholders from the beginning of the process
Engagement of stakeholders across several sectors is a critical element for the success of rural retention policies, as it is for any type of health system or health workforce policy. In identifying and selecting the most appropriate strategies a wide consultative and coordination effort is needed. Rural and remote communities, professional associations and other relevant decisionmakers must be included in the design, development, implementation, monitoring and evaluation to obtain and maintain the support of all involved. Details about roles of various stakeholders in implementing the proposed interventions are provided in Chapter 4.

# Get into the habit of evaluation and learning
A commitment to monitoring and evaluation from the beginning is essential in order to capture valuable lessons learnt and contribute to building the evidence base, which will be of use at the country level and for countries that have similar contexts. Monitoring and evaluation will help identify challenges and limitations during implementation, assess the degree to which the objectives and goals have been achieved, and identify the need for a new intervention or the need to re-design or modify an existing one. Monitoring and evaluation should be part of the design phase and integrated into the implementation plan (see Chapter 4). In addition, continuing investment in national information systems is necessary to ensure timely and accurate data and information are available to inform the policy-making process.
Continuous learning is also critical. Applying so-called "best practices" from one country to another will not work without a clear understanding of the specific situation, needs and context. Hence the need for operational research to evaluate effectiveness and revise polices as necessary once implementation is under way. This will contribute to building the evidence base on why interventions work in some contexts but fail in others, and how they work. This evidence will help policy-makers in other countries choose the most appropriate interventions and to adapt them as necessary to fit their specific situation. The research gaps that were identified in the process of developing each recommendation are mentioned in the tables in Chapter 6. In addition, details about the quality of research in this field and how it can be strengthened and supported are provided in Chapter 5.

# Evidence-based recommendations to improve attraction, recruitment and retention of health workers in remote and rural areas
This chapter describes a range of interventions that can be combined to improve the retention of health workers in remote and rural areas. The interventions fall under four categories: education, regulation, financial incentives, and personal and professional support (see Table 3.1 below).

# Table1. Categories of interventions used to improve attraction, recruitment and retention of health workers in remote and rural areas

# Category of intervention Examples
A. Education Each recommendation in this chapter is accompanied by a statement about the quality of the evidence and the strength of the recommendation. These are requirements of GRADE (see section 1.7). As far as the evidence is concerned, the majority of the studies in this field are observational, and most do not use a control group. GRADE considers this type of evidence to be of "low" quality.
Because of the certain limitations in using GRADE criteria to assess the quality of the evidence for complex interventions with multiple outcomes, the expert group opted to give more weight to other factors in deciding on the strength of the recommendations. These criteria include the balance between benefits and harms, the variability in values and preferences, the resources needed, and technical feasibility in different contexts. Balance worksheets were used by the expert group to provide details of these factors for each of the 16 recommendations, and they are presented in chapter 6. In general, when there was low quality of the evidence, the group considered more important the values of equity and the dire need to give remote and rural populations access to health workers.
An intervention with a "strong" recommendation is associated with "moderate" or "low" quality of the evidence in the GRADE tables, general consensus on the absolute magnitude of the effects and benefits, no significant variability in how different stakeholders value the outcomes, and technical prerequisites for implementation that are feasible in most settings. Interventions with a "strong" recommendation are more likely to be successful in a wide variety of settings.
A "conditional" recommendation for an intervention implies "very low" or "low" quality of the evidence, only a small magnitude of effect over a short period of time, significantly more potentially negative effects, wide variability in values among stakeholders, and significant variability between countries in the prerequisites for implementation. A "conditional" recommendation is less likely to be successful in all settings and requires careful consideration of the contextual issues and the prerequisites for implementation, which are detailed in Chapters 2 and 4.
The current evidence and the experts' knowledge, experience, opinions and judgements are presented in this chapter in the evidence summaries and commentaries that accompany each recommendation statement. The commentaries also highlight some of the major research gaps, which are expanded on in Chapter 5. Further details can be found in the tables in Chapter 6 and in the annexes that are available on CD-ROM.
Education is the foundation for producing competent health workers. It is therefore important to select the "right" students, that is, those who are more likely to practise in remote and rural areas, and to train them in locations and using methods and curricula that are more likely to influence their future practice location. It is also important to support health workers' need to continue learning throughout their careers, particularly in isolated areas where access to knowledge and information is not easy. Use targeted admission policies to enrol students with a rural background in education programmes for various health disciplines, in order to increase the likelihood of graduates choosing to practice in rural areas.
Quality of the evidence -moderate. Strength of the recommendation -strong.

# Summary of the evidence
There is a compelling body of evidence from high-, middle-and low-income countries that a rural background increases the chance of graduates returning to practise in rural communities. Some studies have shown they continue to practise in those areas for at least 10 years (31)(32)(33)(34).
A Cochrane systematic review states: "It appears to be the single factor most strongly associated with rural practice" (35).
Several longitudinal studies tracking the practice locations of physicians in the USA have found that students with a rural background continue to practise in rural areas for an average of 11-16 years after graduation (see Box 3). In South Africa, students from rural backgrounds are three times more likely to practise in a rural location compared with their urban counterparts (32).

# Box 3. The Physician Shortage Area Programme (PSAP) of Jefferson Medical College
A multifaceted education programme aimed at producing long-serving physicians for rural areas in the USA has proven highly successful, according to the results of comprehensive longitudinal cohort studies. Researchers tracked the location and retention of graduate physicians from the PSAP in rural areas of the USA for over 20 years. They found that after 11-16 years, "68% of the PSAP graduates were still practicing family medicine in the same rural area, compared with 46% of their non-PSAP peers". Although the PSAP's class sizes are relatively small, the evidence indicates that a high percentage of its graduates serve in rural areas for many years (33).
Medical schools tend to have high education standards for admission. Countries with a lower level of secondary education in rural areas compared with urban areas may need to link specific quotas to admit students from rural backgrounds with academic bridging programmes. China, Thailand and Viet Nam are a few of the countries that have adopted this approach. The long-term solution is for governments to improve the quality of primary and secondary education in remote and rural areas.
Students from rural areas may need more financial assistance during their studies, as rural families often have significantly lower incomes than urban families. They may also need more academic and social support, because of the transition from a rural to an urban area.
When students from rural backgrounds are trained in schools also located in rural areas, using curricula that are adapted for rural health needs, they are more likely to return to work in those areas. Hence, it is important for policy-makers to bundle together at least these three interventions for a better result (A1 bundled with A2 and A3, and with B4).
More research is needed to understand whether a certain "profile" of a future rural health worker can be identified: this may be related to geographical origin, gender, specific behaviour traits, such as altruism, or other intrinsic motivation factors. Such knowledge would inform selection and recruitment policies, as well as counselling of high-school students prior to entering higher education.
There is inconclusive evidence about the extent to which gender and ethnicity are associated with practising in rural areas. This needs further research especially in countries where the demography of the health workforce is rapidly changing as a result of many more women and ethnic minorities entering medicine and nursing.
Little is known about preferential or targeted admissions of students from rural backgrounds into nursing or other health professional schools.

# Train students closer to rural communities recommendation a2
Locate health professional schools, campuses and family medicine residency programmes outside of capitals and other major cities, as graduates of these schools and programmes are more likely to work in rural areas.
Quality of the evidence -low. Strength of the recommendation -conditional.

# Summary of the evidence
Large observational studies from high-and low-income countries show that medical schools located in rural areas are likely to produce more physicians working in rural areas than urbanely located schools. For example, a recent review found that medical schools in the USA with the following characteristics tend to produce more rural physicians: located in rural states, public ownership, offering training in generalist specialties and receiving little federal research funding (36). A study in the Democratic Republic of the Congo showed that location of a school in a rural area was strongly associated with subsequent employment in the rural area (37). A study in China showed that rural medical schools produce more rural physicians than medical schools located in metropolitan centres (38). However, it is often difficult to determine the independent effect of rural location of schools, because research findings tend to be confounded by such factors as recruitment of more rural students in such schools (36). There is limited evidence that graduates from postgraduate residency programmes located in rural areas, particularly in family medicine, are also more likely to practise in a rural location, but there are some methodological limitations for this evidence (39-41) (see Table 4.2).
Complementary strategies such as distance education and e-learning approaches should be considered as they may allow urban-based schools to extend beyond their usual catchment areas and may give more rural residents access to education without having to relocate to distant cities. Combining this intervention with targeted admissions and curricula changes (A1 and A3) is likely to yield better results.
Some evidence is emerging about the benefits of locating schools for other health professions in rural areas in developing countries as well (4,42), but the effects need to be better studied.
There is emerging evidence about the importance of promoting a social accountability framework for medical education in underserved areas to better respond to the needs of these communities. For example, several need-and outcome-driven medicals schools in remote or rural areas in Australia, Canada, the Philippines and South Africa formed a network with "a core mission to increase the number, quality, retention and performance of health professionals in underserved communities" (/). The principles of social accountability underpinning the training provided by these schools are highlighted in Box 4 below.

# Box 4. Principles of social accountability underpinning the Training for Health Equity Network's (THENet) medical schools
- Health and social needs of targeted communities guide education, research and service programmes.
- Students are recruited from the communities with the greatest health-care needs.
- Programmes are located within or in close proximity to the communities they serve.
- Much of the learning takes place in the community instead of predominantly in university and hospital settings.
- The curriculum integrates basic and clinical sciences with population health and social sciences; and early clinical contact increases the relevance and value of theoretical learning.
- Pedagogical methodologies are student-centred, problem-and service-based and supported by information technology.
- Community-based practitioners are recruited and trained as teachers and mentors.
- Schools partner with the health system to produce locally relevant competencies.
- Faculty and programmes emphasize and model commitment to public service (43).

# Bring students to rural communities recommendation a3
Expose undergraduate students of various health disciplines to rural community experiences and clinical rotations as these can have a positive influence on attracting and recruiting health workers to rural areas.
Quality of the evidence -very low. Strength of the recommendation -conditional.

# Summary of the evidence
Undergraduate training, particularly for physicians, is typically conducted in tertiary care institutions using the latest available technology and diagnostic tools. Once medical studies finish, young graduates are left without skills to deal with health situations in areas where advanced technology and tools are not available. The same holds true for other health professions. Clinical placements in rural areas during undergraduate studies is one way to expose students to the health issues and conditions of service within rural communities, and give them a better understanding of the realities of rural health work.
The evidence on the effects of clinical rotations on improved retention is mixed, but it does show that exposure to rural communities during undergraduate studies influences subsequent choices to practise in those areas, even for students with an urban background (44)(45)(46)(47). These studies, which were conducted for medical, pharmacy and nursing students, also show improved competencies in dealing with rural health issues among students who completed a rural placement during their studies. However, as the rural placements are not always mandatory, there is sometimes the possibility that students from a rural background may self-select for these programmes, bringing potential confounders to the results of the studies.
Rural-based training may allow health workers to "grow roots" in such locations and facilitate the development of professional networks. It may also increase awareness of rural health, even for those who may eventually choose not to practise in a rural area on a permanent basis. The effect can be larger if this intervention is associated with A1 (targeted admission), A2 (location of schools outside major cities) and A4 (changes in curricula).
The optimum duration of the rural exposure during undergraduate studies is not known. It varies from four weeks up to 36 weeks of placement, and it can be mandatory or voluntary. The local availability of mentors, trainers and supervisors is a critical component of this intervention.
Stronger study designs are needed to better address confounders in self-selection of students in the rural clinical placement programmes. More studies are needed on other types of health workers and from developing countries.

# Match curricula with rural health needs recommendation a4
Revise undergraduate and postgraduate curricula to include rural health topics so as to enhance the competencies of health professionals working in rural areas, and thereby increase their job satisfaction and retention.
Quality of the evidence -low. Strength of the recommendation -strong.

# Summary of the evidence
Existing evidence in support of this recommendation is generally lacking, particularly in developing countries and for disciplines other than medicine. However, there is evidence that education with a primary care focus or a generalist perspective is conducive to producing practitioners willing and able to work in rural areas (48). This is because most rural health workers are generalists or primary care practitioners. In addition, some studies suggest that advanced procedural skills training (e.g. in obstetrics, emergency medicine, anaesthesia and surgery) can enhance the confidence of family medicine residents and equip them with the requisite skills for rural practice (49,50). This is because rural practitioners often lack specialist support and have a wider scope of practice.
Practising in rural areas is associated with three factors: a rural background; positive clinical and educational experiences in rural settings during undergraduate education; and targeted training for rural practice at the postgraduate level (51). However, the individual effects of each of these factors on improved retention are difficult to estimate, because of many confounders. Although there is no direct evidence that curricula changes improve rural retention, ample supportive evidence shows that rurally oriented curricula equip young students with the skills and competencies necessary to practise in those areas (52). For example, a small-scale study in Australia was able to show that when comparing mean percentages of fifth-year exam results, students from the rural curriculum course gained better results than the urban-based medical curriculum in several disciplines related to general practice, such as internal medicine, surgery, obstetrics and gynaecology, paediatrics, psychiatry and clinical examination (53).
The practice of health workers in rural areas is quite different from their urban counterparts, in the way they need to conduct the clinical assessment and management without sophisticated tools and equipment, and the way they need to collaborate with rural communities and manage the rural context. Therefore, educating students in large teaching hospitals is unlikely to equip them with the necessary skills and competencies to adequately address the health needs and the conditions of practice in rural areas. Curriculum review and renewal on an ongoing basis are needed, though the process can be time consuming. It is also important to ensure that the rural context is reflected in educational content. In addition, generalist or primary care focused curricula should include sufficient exposure to relevant specialist knowledge in order to prepare practitioners with a wider scope of practice that is often required in rural areas. More studies are needed on the direct effects of curricula changes on the retention of health workers, and particularly in relation to non-physicians.

# Facilitate professional development recommendation a5
Design continuing education and professional development programmes that meet the needs of rural health workers and that are accessible from where they live and work, so as to support their retention.
Quality of the evidence -low. Strength of the recommendation -conditional.

# Summary of the evidence
Access to continuing education and professional development is necessary to maintain competence and improve performance of health workers everywhere (1). However, it may be difficult for health workers in rural areas to access these programmes if it requires travelling to urban locations. There is limited direct evidence on the effect of continuing education programmes on retention. But there is ample supportive evidence that if delivered in rural areas, and if focused on the expressed needs of rural health workers, these programmes are likely to improve the competence of rural health workers, make them feel like they are a part of a professional group, and increase their desire to remain and practise in those areas (54)(55).
As for the previous interventions, better results are more likely with a combination of interventions. To be successful, continuing education needs to be linked to career paths (D4), as well as with other education interventions. Continuing education should be viewed from a broader perspective. Such activities are not only for knowledge acquisition or skills development, they also provide opportunities for rural health workers to interact with other practitioners and to maintain professional networks and social contacts, which may help reduce the sense of social or professional isolation (56). Distance learning by means of information and communication technologies should be used, where appropriate and available, in order to bring continuing education programmes to more remote locations.

# Regulatory interventions
Regulatory measures can be defined broadly to encompass any government control exercised through legislative, administrative, legal or policy tools. Regulatory measures range from parliamentary laws/statutes to state regulations, policies and guidelines developed by line ministries, and programme guidance. With regard to recruitment and retention in rural areas, the interventions that require regulatory measures are related to expanding the scope of practice of rural health workers, producing different types of health workers, compulsory service requirements and bonding schemes.

# Summary of the evidence
Health workers serving rural and remote communities may often have to provide services beyond the remit of their formal training, because of the absence of other more qualified health workers. In some instances this de facto enhanced scope of practice is recognized through regulatory measures (decrees, etc.) that allow certain categories of health workers to provide tasks that are beyond their training, on the assumption that this will increase access to health services for remote and rural populations.
Whether or not this expanded scope of practice has actually contributed to retention of health workers is unclear from the current evidence. There is however evidence to show that enhanced scope of practice can lead to increased job satisfaction. For example, a control study in Australia found that enrolled nurses who were allowed to prescribe reported higher levels of job satisfaction than non-medication endorsed nurses (57).
There is also compelling evidence that quality of care is not diminished when delivered by health workers with enhanced scope of practice. Indeed, one systematic review (58) found six randomized controlled trials showing that "quality of care was in some ways better for nurse practitioner consultations" when compared with physicians, although in non-rural settings. In addition, patients reported higher levels of satisfaction with nurse practitioners.
Health workers with an enhanced scope of practice can provide vital health-service delivery particularly in areas with an absolute shortage of health workers. For example, while efforts are made towards scaling-up the production of physicians, nurse practitioners and mid-level workers can be used to provide some of the services in the absence of physicians.
Ministries of health need to work with regulatory bodies, professional associations and other stakeholders in order to clearly stipulate the boundaries and guidelines for expanded scopes of practices. There may be considerable resistance from certain groups of health workers, and their concerns and arguments need to be voiced and carefully considered as part of this process. B1 is often bundled with B2 (different types of health workers). Combining this recommendation with D6 will help ensure that all those working with an expanded scope of practice are recognized for the contribution and service they are delivering in remote and rural areas. Finally, the attractiveness of relocating to a remote and rural area is likely to increase if the post includes access to further education and training (A5) and financial incentives (C1).
While it has been acknowledged that health workers with enhanced scopes of practice can contribute effectively to health-service delivery in remote and rural areas, more evidence is needed to understand whether these health workers are more likely to be retained in these areas. In addition, little is known about the type of package that is required to recruit and retain health workers with enhanced scopes of practice.

# Train more health workers faster to meet rural health needs recommendation b2
Introduce different types of health workers with appropriate training and regulation for rural practice in order to increase the number of health workers practising in rural and remote areas.
Quality of the evidence -low. Strength of the recommendation -conditional. Box 5 highlights the findings from one of the few studies investigating the retention of such workers (60). Yet there is convincing evidence to support the fact that different types of health workers can lead to improved health outcomes (61) and that many countries heavily rely on clinical officers, health assistants and other types of health workers to provide health care in remote and rural areas (62).

# Summary of the evidence

# Box 5. "Tecnicos de cirurgia" in Mozambique
Mozambique began to educate and train assistant medical officers with surgical skills called "tecnicos de cirurgia" in 1987. Twenty years later, a study found that 88% of all the "tecnicos" who graduated in 1987, 1988 and 1996 were still working in district hospitals, compared with only 7% of medical officers who were originally assigned to district hospitals after graduation. Considering that these "tecnicos" perform 92% of all major obstetrical surgical interventions in rural hospitals, the authors argue that provision of emergency obstetric care in these areas would be "impossible" without them (60).
One rationale behind creating different cadres of health workers for remote and rural areas is that their skills and qualifications may be less marketable than those of highly-trained health workers, who are also in demand in urban settings, or even outside the country. Another reason for embracing this policy is that specific types of health workers can be trained to be more receptive and reactive to local health needs, provided that quality and safety issues are also taken into account (63). In addition, types of health workers that can be trained in a relatively short period of time may be a more financially viable option in low-resource settings. For increased recruitment and retention, it is important to consider the use of financial incentives (C1) and recognition measures for these cadres (D6).
Although different types of health workers are being used in many countries, more research is needed to understand their retention in remote and rural areas, particularly in comparison with other, more traditional health cadres, such as physicians. Additionally, more sound evidence is required on the intentions and factors motivating mid-level cadres in comparison with highertrained health workers.

# Make the most of compulsory service recommendation b3
Ensure compulsory service requirements in rural and remote areas are accompanied with appropriate support and incentives so as to increase recruitment and subsequent retention of health professionals in these areas.
Quality of the evidence -low. Strength of the recommendation -conditional.

# Summary of the evidence
Compulsory service is understood as the mandatory deployment of health workers in remote or rural areas for a certain period of time, with the aim to ensure availability of services in these areas. It can be either imposed by the government (for positions that are under government employment), or linked to various other policies. For example, it can be a mandatory requirement to serve for a certain period of time in remote areas before obtaining the license to practise; or it can be a prerequisite before applying for a specialization or for career advancement.
A comprehensive review of compulsory service schemes undertaken as part of the development of these recommendations found that approximately 70 countries have previously used or are currently using compulsory service (64). The duration varies from country to country, from a minimum of one year to a maximum of nine years, and the policies have included almost all types of health workers.
Despite the popularity of compulsory service, very few evaluations have been conducted in relation to the retention of health workers either during or post their obligated service period. Studies in Ecuador (65) and South Africa (66) reveal that although physicians raised serious complaints over the management of their compulsory service scheme, they did feel that the experience improved their competencies and had been rewarding overall. In some countries, remote and rural areas are reliant upon graduates who are complying with their compulsory service obligations. In Thailand, 28 years after the implementation of a national compulsory service strategy, 49.5% of doctors in rural district hospitals were new graduates, presumably completing their compulsory service requirements (67).
Even if only for a limited period of time, health workers completing their compulsory service requirements can significantly increase the availability of health workers in underserved areas. Furthermore, compulsory service periods in remote and rural areas can increase health workers' appreciation for rural health issues, prove a valuable learning experience, and provide an opportunity to make a difference to the health of people living in underserved and disadvantaged communities.
However, there are notable challenges and risks to implementing a compulsory service requirement for health workers. In the Indian state of Kerala, for example, large and sustained strikes were organized in protest of a new three year compulsory service for medical graduates. Compulsory service can also be criticized for increasing turnover in health centres, and therefore potentially decreasing the quality of care delivered.
Support and management systems need to be in place to ensure the successful implementation of compulsory service, and participants need to be appropriately prepared prior to their compulsory service in order to be able to provide the expected standard of care (10,(68)(69)(70).
Combining compulsory service with other types of incentives (A5 and C1) and with efforts to improve the working and living environment of the locations (D1 and D2) is likely to yield better results.
As previously alluded to, more evaluations are required to understand the retention of health workers in remote and rural areas following the completion of their obligatory service period. Furthermore, research is required to evaluate compulsory service schemes for health workers other than physicians.

# Tie education subsidies to mandatory placements recommendation b4
Provide scholarships, bursaries or other education subsidies with enforceable agreements of return of service in rural or remote areas to increase recruitment of health workers in these areas.
Quality of the evidence -low. Strength of the recommendation -conditional.

# Summary of the evidence
Many governments offer students in the health professions scholarships, bursaries, stipends or other forms of subsidies to cover the costs of their education and training and in return students agree to work in a remote or rural area for a certain number years after they become a qualified health worker.
A systematic review analysed the effectiveness of financial incentives given in return for medical service in rural areas (71). It included 43 studies, of which 34 evaluated programmes based in the USA, while the rest examined programmes from Canada, Japan, New Zealand and South Africa. In these programmes, future health workers (i.e. students), or practising health workers enter into a contract whereby they receive some sort of financial incentive (either scholarships for their education, or loans to payback their education, or direct financial incentives), and in exchange they commit to serve in a rural area for a certain period of time. Usually this intervention is combined with other types of retention strategies, such as recruitment of students from rural backgrounds or training in a rurally located school (see Box 6).
These types of bonding schemes were linked to impressive retention rates in 18 studies: the proportion of participants who remained in the underserved area after completing their obligated period of service ranged from 12% to 90%. Yet numerous studies included in this systematic review had serious methodological flaws and therefore these findings should be interpreted with some caution (see related evidence profile in Annex 1 for more information).
Bonding schemes appear to be successful in placing significant numbers of health workers in rural areas, and some even appear effective in ensuring that programme participants will continue to work in other underserved areas after completing their obligatory service. However, as many offer a "buy-out" option, further information is required to understand how popular this option is in comparison with completing the compulsory service.
As with other recommendations, positive outcomes are more likely if these return-of-service agreements are combined with other interventions. For example, combining these incentives with targeted admissions (A1) is likely to have a larger effect.
Further evidence is required on education subsidies in return of service for nursing students and other types of health professional students. More needs to be known about the characteristics of students who commit to return-of-service agreements and why some graduates choose the "buy-out" option rather than completing their service. More cohort studies should be conducted to compare the retention rates of health workers who have completed their return of service with those who graduated without being part of a bonding scheme.

# Box 6. Home prefecture recruiting scheme, Jichi Medical University, Japan
The Jichi Medical University (JMU) in Japan began a new and unique home prefecture recruiting scheme in 1972 with the aim to produce rural doctors and distribute them nationwide. Students who attend JMU are fully funded by their prefecture government to study medicine and they sign a contract bonding them to working in their home prefecture medical institutions for nine years post-graduation, with five to six years of this obligation spent in rural dispatch areas chosen by their home prefecture. If a contract is breached all medical school expenses must be paid in one lump sum.
In one part of a well-designed retrospective cohort study, 1477 graduates from JMU were surveyed in 2000, 2004 and 2006. There was a 95% completion rate and on average, 69.8% of JMU graduates remained in their home prefectures for at least six years after their obligatory service. Interestingly, if settlement is defined as being in a home prefecture for at least one out of the three time points, the settlement rate of post-obligation JMU graduates rises to 76.3% (72).

# Financial incentives
In this document, financial incentives encompass all additional benefits paid or provided to health workers to entice them to work in a remote or rural area. They include monetary bonuses, in-kind benefits (a free house or vehicle), and any other benefits that reduce the opportunity costs associated with working in rural areas 1. Lost revenues because of limited opportunities for private practice in rural areas, and additional housing costs from having to keep a house in the capital city (for children's education and/or a spouse's job) are two examples of opportunity costs.
Although financial incentives have similar potential effects on the decision-making process of health workers, they may be funded through different sources and have different timelines. Fiscal sustainability is paramount for the long-term effectiveness of financial incentives.
c1 Appropriate financial incentives

# Make it worthwhile to move to a remote or rural area recommendation c1
Use a combination of fiscally sustainable financial incentives such as hardship allowances, grants for housing, free transportation, paid vacations, etc., sufficient enough to outweigh the opportunity costs associated with working in rural areas, as perceived by health workers, to improve rural retention.
Quality of the evidence -low. Strength of the recommendation -conditional.

# Summary of the evidence
Several studies point to salaries and allowances as two of the key factors that influence health workers' decisions to stay in or leave a rural workplace (27,28,(73)(74)(75)(76).
Financial incentives are widely used to recruit and retain health workers in remote and rural positions, and can be implemented relatively quickly. Yet well-designed and comprehensive evaluations of the effectiveness of financial incentives are rare, and the evidence that is available suggests mixed results. In Australia, for example, financial incentives were set up for long-serving physicians in remote and rural areas and the amount paid varied according to location and length of service (77). One of these incentive plans succeeded in achieving a 65% retention rate of physicians after five years. In the Niger, financial incentives were responsible for increasing the percentage of physicians, pharmacists and dentists working outside the capital, Niamey. But two years after implementation, the proportion of health workers choosing to go to these areas had not changed significantly (from 42% at the start to 46% after two years) (78).
Other studies have shown positive effects of financial incentives on increased attractiveness of rural areas. A survey in South Africa found that 28% to 35% of rural health workers who received the rural allowance believed it affected their career plans for the next year (79). A mid-term review of the Zambian Health Workers Retention Scheme found that within two years of implementation, the scheme had been able to attract and retain more than 50 doctors in rural areas, some to areas where there were previously no doctors available (80).
Prior to implementing financial incentives, significant work needs to be done to fully understand the opportunity costs of working in remote and rural areas as the incentives need to be carefully matched to the demands and expectations of health workers. Feasibility studies, such as discrete choice experiments and a labour market analysis, are essential to inform the design of a financial incentives scheme.
Policy-makers need to be aware of potential sensitivities surrounding giving health workers specific financial incentives and the problems this may cause between them and other civil servants (if in a civil-service system) or those health workers not covered under the scheme. For example, the rural ranking scale in New Zealand caused serious discord between physicians over the definition of "rural". Some felt they had been unfairly categorized and some even claimed they had been financially disadvantaged under the new payment system (81).
A financial incentive scheme will be more cost effective in countries with a significant surplus of health workers in major cities because underemployed or unemployed health workers could be attracted to rural areas at a lower social cost than already employed health workers. For example, providing incentives can have different results in different contexts: in the Niger, the shortage of physicians made it nearly impossible to use incentive payments as a trigger for physicians to relocate in rural areas as the urban market offered sufficient space for private practice and incomes were much higher. This is in contrast to Mali, where an oversupply of medical doctors made it attractive for unemployed young doctors to practise in rural areas when offered incentives to relocate. The differences in results of payment of incentives demonstrate the importance of including the labour market in the situational analysis (82).
The effectiveness of financial incentives could be greater if combined with other interventions, particularly targeting these at students and health workers with a rural background (A1). Consideration should also be given to combining these with B2 (different types of health workers, B3 (compulsory service), D1 (improved living conditions) and D2 (safe and supportive working environment) to ensure increased recruitment and retention of health workers.
More well-designed and comprehensive evaluations need to be conducted in order to determine the long-term impact of financial incentives on the retention of health workers in remote and rural areas.

# Box 7. Emerging evidence
There is emerging evidence on other types of financial mechanisms to improve retention of health workers in rural practice, but it is too early to draw meaningful conclusions. Future revisions of these recommendations will consider in further detail the following interventions:
- Facilitating the establishment of private practice or a public-private mix in rural and remote areas.
- The use of health financing reforms, including universal health insurance and results-based financing schemes, to strengthen the financial incentives for recruiting and retaining health workers in remote and rural areas.

# Personal and professional support
By definition, rural and remote areas often convey a sense of isolation, both from a professional and personal point of view. It is then no surprise that when asked what matters most in choosing to work in a rural location, students, young graduates and health workers all mention the need for support. This can be perceived on a personal level, where issues related to good infrastructure, opportunities for social interaction, schooling for children and employment for spouses all rank high on the preferences of health workers. On a professional level, opportunities to advance careers and to communicate and consult with peers through networks, telehealth or other approaches are equally important. For all types of health workers, public recognition of the services they provide to communities goes a long way in improving their morale, status and subsequent desire to work in rural areas.
It is reasonably clear from the evidence and country-specific experience that interventions in this area are complementary and are more likely to augment each others' impact but be ineffective in isolation. A core basic requirement for all of these and previous interventions to be effective will come from developing, deploying and motivating effective local services managers and strengthening human resource management systems, as is elaborated upon in Chapter 2. Quality of the evidence -low. Strength of the recommendation -strong.

# Summary of the evidence
The absence of direct evidence that improving rural health infrastructure and living conditions contributes to increased retention of health workers in rural areas is mainly because few largescale programmes have been implemented (83). On the other hand, there is ample supportive evidence. In studies that aim to elicit the factors that influence decisions to work in a remote or rural area, the availability of good living conditions is always mentioned as very important. This includes accommodation, roads, electricity, running water, Internet access, schools for children and employment opportunities for spouses.
A study of South African doctors listed better accommodation as one of the three most important factors that would influence them to remain in a rural area (76). A study in Bangladesh revealed that remoteness and difficult access to health centres were major reasons for health worker absenteeism, while health personnel working in villages or towns with roads and electricity were far less likely to be absent (84). Anecdotal data reinforce the results of studies indicating that the lack of appropriate housing, electricity and phone service, and inadequate schools, all act as disincentives for rural service.
Given that this intervention is always part of a larger retention package or scheme of so-called "non-financial incentives", it is difficult to isolate its individual effect on retention.
Improving rural infrastructure is part of the overall economic development of rural and remote areas.
It is an investment that, among other things, will help to improve health worker retention and have similarly beneficial effects on workers from other public sectors such as teachers and policemen. It will also create a more attractive environment for private activities in all economic sectors.

# Ensure the workplace is up to an acceptable standard recommendation d2
Provide a good and safe working environment, including appropriate equipment and supplies, supportive supervision and mentoring, in order to make these posts professionally attractive, and thereby increase the recruitment and retention of health workers in remote and rural areas.
Quality of the evidence -low. Strength of the recommendation -strong.

# Summary of the evidence
To what extent improving the working environment has directly improved retention in rural areas is unclear. However, according to a Cochrane systematic review, "questionnaire-based surveys suggest that professional and personal support may also influence health professionals' choice to work in underserved areas. Professional development, ongoing training and style of health service management were important factors influencing retention of health professionals in underserved areas" (35).
Supportive evidence from satisfaction surveys shows that health professionals are disinclined to apply for or accept assignments to practise in facilities that are in a state of disrepair and that do not have basic supplies, such as running water, gloves, elementary basic drugs and rudimentary equipment, because this dysfunctional work environment severely limits their ability to practise what they have been trained to do (29,76). In addition, supportive supervision is also a key element that contributes to improved job satisfaction, performance and subsequent retention and practise in rural areas (1,26).
Improving working conditions is likely to improve the performance and productivity of health workers, and hence the performance of health systems. But there is a risk that if pilot programmes are implemented in just some rural areas of a country, these will attract health workers from other areas, thereby re-enforcing existing imbalances.
In terms of costs, equipping and refurbishing health facilities may be resource-intensive, but benefits can be achieved for a longer period. Likewise, changes in management style and implementing supportive supervision may also require significant investment in management training courses and in effective supervision processes, but long-term benefits can be expected. Finally, holistic strategies to prevent workplace violence can also be complex and costly, but it is likely they will contribute to improved job satisfaction for the long run.

# Foster interaction between urban and rural health workers recommendation d3
Identify and implement appropriate outreach activities to facilitate cooperation between health workers from better served areas and those in underserved areas, and, where feasible, use telehealth to provide additional support to health workers in remote and rural areas.
Quality of the evidence -low. Strength of the recommendation -strong.

# Summary of the evidence
In addition to improved working conditions and supportive supervision, there is also the possibility to provide outreach support to rural health workers. One form of outreach support is when individual specialists or teams of specialists make regular visits to their rural peers to advise and assist with patient care and their professional development. Another form is telehealth, where distance-based technology is used to help rural health workers diagnose and treat patients and improve their knowledge and skills.
There is no direct evidence that outreach support programmes improve rural or remote retention. However, there is ample supportive evidence from observational studies that such programmes improve competencies and job satisfaction of rural health workers (85)(86)(87)(88). They can also contribute to improving local quality of care, reduce the number of consultation visits to specialists and lower the rate of hospital admissions (89,90).
Outreach activities can, among other things, reduce feelings of professional isolation. They are likely to be more beneficial in settings where there is a critical shortage of health workers, limited infrastructure or very sparse populations, as it provides a service that otherwise would not be available (e.g. mobile clinics or fly-in services).
Implementing outreach support activities, and particularly telehealth programmes, requires significant financial resources, as well as access to the Internet and other technologies. But rapid advances in telecommunications, in particular in the use of mobile phones, offers hope for more rapid and widespread implementation of such programmes in the near future.
More studies are needed on the role of telehealth and outreach programmes on the retention of health workers.

# Design career ladders for rural health workers recommendation d4
Develop and support career development programmes and provide senior posts in rural areas so that health workers can move up the career path as a result of experience, education and training, without necessarily leaving rural areas.
Quality of the evidence -low. Strength of the recommendation -strong.

# Summary of the evidence
A career ladder provides a sequence of posts, from the most junior to the most senior, which health workers can climb up as they advance in their jobs. This is particularly relevant in the public sector and civil service where a clear sense of hierarchy is the rule.
There is no direct evidence that setting up career ladders in rural areas can help to retain health workers. However, evidence from surveys shows that clear career prospects are important factors in the choice of health workers to practise or not in a remote or rural area (91,92). Career ladders are common in urban and hospital settings, but it is possible to develop clear and specific career paths in rural settings as well. Figure 3 below shows examples of potential steps in career ladders for various health professions.

# Figure 3. Examples of career ladders for health workers
Such interventions are likely to improve the morale and professional status of health workers, which can in turn improve their motivation, job satisfaction and work performance. However, in some instances, there may be opposition from professional bodies and/or tensions between specialists and generalists. More studies are needed on rural career ladders and their effects on improved retention.

# Facilitate knowledge exchange recommendation d5
Support the development of professional networks, rural health professional associations, rural health journals etc. in order to improve the morale and status of rural providers and reduce feelings of professional isolation.
Quality of the evidence -low. Strength of the recommendation -strong.

# Summary of the evidence
Health workers' need for continuous professional stimulation is all the more relevant in rural or remote areas, where professional isolation can negatively influence performance. Therefore, supporting professional networking and academic activities, including specialized journals with a focus on rural areas, can prove beneficial for rural health workers (93).
Some evidence shows that rural professional associations have increased the retention of health workers in rural areas. For example, in Mali, young doctors who were supported by the professional association, "Association des Medecins de Campagne", remained in rural areas for an average of four years; the retention rate was lower for those who did not have this support (4).
The "Rural Doctors Society and Foundation" in Thailand has had several positive effects on the profile and impact of rural physicians."Apart from supporting rural health services, the society has also actively supported public health movements, such as a national drug policy, an essential drugs list and tobacco control. It has also played an active role in the national movement toward democratization and political reform as well as a watchdog role to counteract corruption and inappropriate administrative behaviour"(10).
In addition to professional associations, other types of support programmes can be envisaged. For example, the "Dr Doc" programme launched in South Australia in 2006 has set up various support mechanisms such as telephone consultations, crisis support, links to urban general practitioners (GPs) who provide health care for rural GPs and their families, as well as country practice retreats to allow rural GPs some rest and relaxation. This has reportedly reduced the number of rural physicians who want to leave their practice (94).
This approach is likely to have larger effects if associated with other interventions, such as A5 (continuing education), D1 (improved living conditions) and D2 (safe and supportive working environment).
Champions may be required in countries to initiate and sustain the development of professional associations. If these associations are only supported by membership fees, they can be vulnerable to long-term sustainability issues.

# Raise the profile of rural health workers recommendation d6
Adopt public recognition measures such as rural health days, awards and titles at local, national and international levels to lift the profile of working in rural areas as these create the conditions to improve intrinsic motivation and thereby contribute to the retention of rural health workers.
Quality of the evidence -low. Strength of the recommendation -strong.

# Summary of the evidence
Recognition from managers, peers and the public is one of the main motivating factors in health care and in other industries (95). But in the case of rural health, the evidence on public recognition comes mainly from case studies of individual health workers who have dedicated their lives to serving rural communities, for which they have received numerous public recognition awards (96)(97). Whether these awards made them stay longer or whether intrinsic motivation factors contributed to their long-term service in rural areas is difficult to say. Nevertheless, it is likely that simple public recognition measures, such as titles, medals or awards can go a long way in raising the status and morale of rural health workers and thus contribute to their retention in these areas. Such public recognition measures are an occasion to focus attention on individual health workers and their achievements, thereby demonstrating political support for rural health workers and rural health work.
This intervention is relatively inexpensive and can be an important step in improving the recognition of rural health workers. Awards can be offered by health services, professional organizations, regional governments, national governments and international organizations.
Integral to the success of this recommendation is the need to promote the award or title. In addition, publishing and bringing the spotlight on rural health worker stories ensures that information on these role models is distributed throughout the population and may motivate students or new graduates to work in rural areas. This also increases the prestige of such awards to the recipient.

# Measuring results: how to select, implement and evaluate rural retention policies
This chapter presents a framework to measure results and poses five questions to prompt and guide policy-makers through the process of identifying, selecting, implementing, monitoring and evaluating rural retention interventions. The framework depicted in Figure 4.1 was prepared during the consultations of the expert group and was further developed by several members of the group (98). It takes a systems approach and builds on the traditional inputs-outputs-outcomes-impact evaluation model.
At the level of inputs, sound analytical work should underpin the subsequent choice of interventions, and this includes a situation analysis and a labour market analysis, as well as an assessment of the organizational and management capacity, as discussed in Chapter 2.
Selecting the appropriate interventions requires a process of understanding their relevance, acceptability, affordability and effectiveness, as explained in sections 4.1-4.4 below, as well as elements of context that need to be considered.
Once the appropriate interventions have been selected and implemented, they will have a direct effect on one or more of three dimensions: attractiveness of rural areas, recruitment of health workers in those areas, and retaining them for a certain period of time ("outputs"). The selected interventions also have an effect on "outcomes". These are expressed as improved health workforce and health systems performance.
The final "impact" of the interventions is expected at the level of improved health status, although health status, as well as health workforce and health systems performance, has more determinants than just these interventions. However, it should be noted that all the proposed retention strategies are complex interventions, and none of the observed effects can be attributed solely to any one single intervention, but rather to an appropriate combination or bundle.
Measuring the results of rural retention interventions can then be performed for all dimensions at each level, and specific indicators are proposed in section 4.5 below.

# Figure 4. Measuring the results of rural retention interventions

# Relevance: which interventions best respond to national priorities and the expectations of health workers and rural communities?
From a policy and planning perspective, and simply put, the availability of health workers in remote and rural areas is a function of whether or not sufficient numbers of health workers are produced, whether or not these are the right types of health workers that are needed, and whether or not there is capacity to absorb the existing health workers, i.e. is there a sufficient number of funded positions in rural and remote areas and fiscal space to accommodate all the needed health workers in the public sector. Another factor is whether or not the work locations are attractive enough for health workers to go there in the first place.
Several additional questions need to be answered at this stage, such as: Can certain groups or geographical areas be exclusively targeted without leading to labour unrest? Does the policy fit into the overall strategy of the government in the health and civil service sectors? What amount of a bonus would be sufficient enough to attract a health worker to a remote or rural area? How much do health workers value priority access to training or good management style in the workplace?
For the latter questions, specific research methods can be used to elicit the preferences of health workers for rural work and to try to calibrate the relative contribution of each potential attribute of their job in a rural area. These methods, known as stated preferences methodologies, of which the discrete choice experiment (DCE) approach is one example, are aimed at quantifying certain trade-offs that health workers would make when given hypothetical scenarios about their future jobs in a rural area (73,(99)(100)(101). They are certainly valuable tools for policy-makers when trying to understand what would be the combination of "incentives packages" that health workers would appreciate in rural areas.

# Acceptability: which interventions are politically acceptable and have the most stakeholder support?
A long-term vision, effective and sustained political commitment and political will are important for successful implementation of the chosen package of interventions. High-level political support is essential to push through planning and budgeting. Government leaders are also needed to act as champions, convene a diverse group of stakeholders and find the most equitable, feasible and sustainable solutions to improve rural retention of health workers.
Many of the interventions are crosscutting in nature and a Ministry of Health and/or individual health-care organizations cannot solve the retention challenge on its own. Engagement of stakeholders across several sectors is a critical element for the success of rural retention policies, as it is for any type of health system or health workforce policy. Ministries of civil service, finance and education, unions and professional associations, civil society, the private sector and, where appropriate, international development partners, all have a role to play.
Table 2 presents an overview of actors that need to be involved in the design and implementation of the policy interventions recommended in Chapter 3, together with their roles and responsibilities.

# Affordability: which interventions are affordable?
All interventions proposed in Chapter 3 have costs associated with them. In choosing one or the other of these interventions, information about the costs (and the level of accuracy of that information), the sources of funds for these costs and their sustainability over time is needed to make the best use of limited financial resources, and for conducting sound evaluations of policy interventions.
Understanding the costs associated with the policy intervention requires a monetary evaluation of all the various resources used to implement the intervention. This may entail, for example, actual money to subsidize education (recommendation B4), or to pay allowances and other financial incentives (recommendation C1), or to build a new school (recommendation A2). But it also includes the costs for bringing faculty to the rural areas for the newly-built schools, and their own subsequent retention, the costs of distance education programmes or of curricula development (education-related interventions), or administration costs for managing an obligatory service in rural areas (see Chapter 6 for more details).
The source and mode of financing are also important. For low-income aid-dependent countries it is particularly important to align sources of funding for retention strategies with national health budgets. This is why, from the planning stage, retention strategies must be aligned with the national health and human resources development plans, as discussed in Chapter 2. Where the money comes from and how it is channelled is tightly linked to the issue of financial sustainability, which entails an analysis of the fiscal space, the timeline and the predictability of external funding. Most low-income countries will require sustained and predictable external funding to implement the interventions, which is often difficult to secure because donors' funding cycles typically span one to three years, which is an insufficient period of time for assessing measurable effects.
A related issue is the fragmented funding of numerous small-scale or specific donor-driven initiatives, which, if not well integrated into the overall national health plan, can seriously disrupt the functioning of the health system. For example, in many countries health workers from rural areas or from the public sector are lured away by non-state providers, often driven by global health initiatives, who offer much more attractive employment conditions (including salaries and working conditions). Compounding this issue are the numerous and uncoordinated in-service training workshops for specific diseases, which often have similar content and take limited staff away from their work (102).

# Effectiveness: have complementarities and potential unintended consequences between various interventions been considered?
The answers to questions 4.1-4.3 will result in a short list of interventions that may require further prioritization, especially in low-income countries (due to resource and capacity constraints). However, it important to stress that evidence and experience show that none of the recommendations in Chapter 3 should be implemented as single interventions, but rather as an appropriate combination of strategies, or as "bundles", based on their potential complementarities. As with most public health strategies and policies, there isn't a one-size-fits-all solution and the most appropriate combination will vary considerably from country to country.
The recommendations will be ineffective in isolation because health workers do not base decisions to go to, stay in or leave remote and rural areas on one single factor. For example, health workers may place a high value on the salary paid in a rural post, but they also want access to continuing education and recognition for their enhanced scope of practice if they accept the position. Or, if a policy of preferential admission of students from rural areas is selected (on the assumption that they are more likely to serve in rural areas after graduation), then expanding the number of training schools in rural areas would be a complementary strategy.
Timing of the interventions is also an important aspect of complementarity. Some of the interventions recommended in Chapter 3 will take several years to be fully established, whereas others can be implemented relatively quickly. For instance, an allowance for rural health workers is one policy that is frequently adopted by countries as it can be apparently quick to set up. Whereas for example a compulsory service policy, a policy to create different types of cadres, or a policy to build schools or campuses in rural areas, may take far longer time to set up and implement. For the rural schools, it takes even longer before they produce a pool of potential rural health workers to be attracted and retained in rural areas. Quick-to-implement interventions with a fairly immediate impact are important to consider as they can help to attract and retain rural health workers in the short term, while other interventions can build up towards fruition. These time-to-effect variations in retention strategies are an important consideration when deciding how best to bundle interventions.
Potential unintended consequences should be considered before deciding on a policy to improve rural retention. For example, if too much emphasis is put on accelerated promotion as an incentive for working in rural areas, this could have a negative effect on organizational development which relies on promotion according to merit and potential. Likewise, indiscriminate distribution of postgraduate training awards could negate the effect of a retention strategy. And incentives for rural doctors may be met with negative reactions from other civil servants or other cadres of health workers.
: what indicators will be used to measure impact over time?
If all relevant stakeholders are clear on the intended effects of the interventions, the expected outcomes and the time it will take to implement and measure impact, then the interventions are more likely to be successfully implemented. Indicators to measure success, or at least progress, need also to be agreed upon from the early planning stage. Table 3 illustrates the questions that need to be asked when evaluating retention interventions, the proposed indicators to measure progress against the four dimensions of attractiveness, recruitment, retention and health workforce/health system performance, and the methods that can be employed in conducting such evaluations.
Detailed guidance about general principles and methods for monitoring and evaluation of human resources interventions have been developed and presented elsewhere (14). For the purpose of this document several definitions and explanations are worth considering."Retention" is defined as an increase of numbers of health workers staying in rural areas as a consequence of a specific policy intervention from within those mentioned in Chapter 3. Another way of measuring retention is to look at the duration (in years) for which health workers have remained in a rural post. However, there is no benchmark for this duration: the few studies that have measured this indicator found an average duration of four years (4,33).
Apart from traditional facility-based surveys and analysis of registry data, other methods can be used, such as "survival" curves. "Survival" curves can be used to plot the time to any non-recurrent event. The event does not have to be death, so the term survival can be misleading. In the case of retention strategies, the non-recurrent event that is plotted can be the departure of the health worker or health workers being studied from the rural area (33).  ----------------------------------------Acceptability: have all stakeholders been engaged? -----------------------------------------Affordability: have all sources of funds been identified and secured?

# Table 3. Questions and indicators for the evaluation and monitoring of interventions to increase access to health workers in remote and rural areas through improved retention
-Factors that motivate health workers to go to, stay in or leave rural areas -Stated preferences for rural job attributes - ----------------------------------------Stakeholders consultations  and engagement  --------------------------------------- ------------------------------------------Stakeholder analysis   -----------------------------------------Review of policy documents
-Did attractiveness of profession/rural/remote areas improve?
- ----------------------------------------Did recruitment of health workers in underserved areas improve?
- ----------------------------------------Did retention improve?
- -------------------------------------- -------------------------------------- -------------------------------------- --------------------------------------- --------------------------------------- As mentioned previously, each intervention has more than one outcome (or effect), and no outcome can be achieved through only one intervention. This complexity adds to the task of measuring the results and attributing the perceived effects to specific interventions.
Further details about the challenges of research in this field are given in Chapter 5. In addition, evaluation is not cheap, so not only does it have to be planned at the beginning of implementing the strategies; it has also to be budgeted for when the interventions are costed.

# Research gaps and research agenda 5.1 Research gaps
Essential for updating these recommendations in 2013 will be the considerable efforts made to fill some of the clear research and evidence gaps that have emerged through the development of this document. Specific research gaps have been identified for each recommendation and are included in the balance sheets presented in Chapter 6. However, the overarching research gaps described below were found to be common across all the recommendations.

# Study all types of health workers
Most available research and studies focus on the recruitment and retention of physicians, with some focusing on nurses and midwives, and very few on other types of health workers, such as pharmacists or clinical officers. It is essential that work is done to rectify this research gap, as a variety of health workers are responsible for the delivery of health services in rural and remote areas. A laboratory technician, for example, may have notably different values and preferences than a physician. It is thus critical to understand the needs and expectations of each cadre of the rural health workforce, individually and as a team.

# More research in low-income countries
More research is required on the recruitment and retention of health workers in remote and rural areas in all countries, but particularly in developing countries that have the severest rural-urban maldistribution of their health workforces. More studies need to be conducted in order to better understand what types of retention schemes are currently working or have failed to work in different settings. For some recommendations, such as A1, there is well-designed and compelling evidence from geographically large developed countries (mainly Australia, Canada and the USA), but very little evidence on the same issue from developing countries.

# More well-designed evaluations
Evaluations are key to help policy-makers feel confident when choosing which interventions to implement, yet there is a dearth of well-designed evaluations in this field, despite the substantial descriptive evidence highlighting the issues and challenges of working in rural areas. Methodological difficulties are one of the main reasons for this research gap, along with potential financial barriers to fund such evaluations. For example, a review of evaluated rural retention interventions found that few interventions considered collecting information on the state of affairs at the early stages of design, or matching the choice of the intervention to the preferred choices of health workers (103). Having a baseline against which to measure progress is mandatory when conducting evaluations. It is also important to have a comparison group and to compare results before and after the intervention. In addition, agreeing upon specific and relevant indicators at the beginning of the process is essential, as is the use of appropriate methods and data sources to measure these indicators (98, 103).
- 1.4 Quality of the evidence -not only "what works", but also "why" and "how"
As clearly demonstrated in the evidence profiles and the descriptive evidence tables, very little of the evidence in this field qualifies as "high quality" evidence when assessed using clinical research appraisal methods. For example, the Cochrane systematic review found no randomized controlled trials in this field, so it adjusted the criteria to include quasi-randomized trials, beforeand-after studies and observational studies (35). Despite this, the number of high-quality studies that were included in the review was very low.
Unlike clinical medicine, it is quite difficult, if not impossible, to conduct randomized controlled trials to understand the effects of many of the interventions proposed in this document. These are complex interventions with multiple outcomes, and many confounders intervene and may influence the observed outcome of a certain intervention. Using clinical research appraisal criteria for assessing the methodological quality of the evidence in this domain may at best yield "low" or "very low" quality of research, which by itself can deter policy-makers from taking any further action.
Much more needs to be considered by policy-makers when deciding the type of interventions to use in their own situation, and the existing evidence needs to be explained in clearer and simpler ways (36). Therefore, the expert group considered that in this field, as in many areas of health systems strengthening, it is equally important to understand whether an intervention works or not (effectiveness), but also "why" it works and "how". Context is a key element that can be responsible for different outcomes or results from the same intervention and thus needs to be better captured in the evaluations of these interventions. Innovative research methods need to be explored and applied in this field. For example, theory-based methods (such as realist review) have the potential to provide insights into the mechanisms and the contextual issues that made the same intervention work in certain contexts and fail in others (82).

# Research agenda
Work has already begun to identify which members of the expert group are willing and able to contribute to filling in the above research gaps, whether through working directly with the WHO Secretariat or through their own independent research projects. A research action plan survey was shared with all experts and seen as an appropriate mechanism to systematize research efforts in the coming years towards the revision of the recommendations in 2013. The survey will establish which areas of research each expert would be able to contribute to, whether as principal or co-investigator, adviser, research assistant or peer reviewer. The survey also will field suggestions for expanded research areas the recommendations should address when they are revised.
A key part of moving forward the research agenda will also be the work done with various pilot countries that have expressed interest in taking forward these recommendations. The WHO Secretariat and the group of experts will provide technical support for these countries, whether at the planning, designing, implementation or evaluation stage of their retention strategies. In addition, further country case studies and reports are already planned in the coming years.

# Deciding on the strength of the recommendations
Table below explains the criteria that were used for deciding whether a recommendation was ranked strong or conditional (see section 3.1 for a description of these terms). The WHO Secretariat produced similar tables (also called balance worksheets) for each recommendation in Chapter 3, which were subsequently reviewed and revised by the expert group by email. The information provided in the tables was taken from the GRADE evidence profiles reported in Annex 1, as well as additional descriptive evidence reported in Annex 2.
As the tables show, and as stressed several times already in this document, quality of the evidence as judged by GRADE was only one of the criteria used to decide on the strength of a recommendation. Experiences and opinions of expert group members have further informed the discussions on the evidence as well as on values and preferences, benefits and disadvantages, resource use and feasibility. Policy-makers need to consider all of these criteria when deciding on and implementing the recommendations through wide stakeholder consultation, and within the specific country context.
An intervention with a "strong" recommendation is associated with "moderate" or "low" quality of the evidence in the GRADE tables, general consensus on the absolute magnitude of the effects and benefits, no significant variability in how different stakeholders value the outcomes, and technical prerequisites for implementation that are feasible in most settings. Interventions with a "strong" recommendation are more likely to be successful in a wide variety of settings.
A "conditional" recommendation for an intervention implies "very low" or "low" quality of the evidence, only a small magnitude of effect over a short period of time, significantly more potentially negative effects, wide variability in values among stakeholders, and significant variability between countries in the prerequisites for implementation. A "conditional" recommendation is less likely to be successful in all settings and requires careful consideration of the contextual issues and the prerequisites for implementation, which are detailed in Chapters 2 and 4. Population: this is the target population to which the intervention is applied Intervention:: insert a very brief description of the intervention

# Factors Decision Explanation
Quality of the evidence
 High  Moderate  Low  Very low
The higher the quality of the evidence, the stronger the recommendation.
However, when "low" or "very low" quality, consider more carefully the other criteria below in deciding the strength of the recommendation.

# Values and preferences
 No significant variability  Significant variability
This refers to values placed by health workers, policy-makers, patients and other stakeholders on the intended outcomes of interventions.
If there is wide variability between values and preferences of various stakeholders, it is less likely to have a strong recommendation.
Absolute magnitude of effect Disadvantages should consider the potentially negative effects of the intervention, as well as the unintended effects.
The less potentially negative effects, the more likely to have a strong recommendation Resource use
 Less resource- intensive  More resource- intensive
The resource needed for implementing the recommendation may comprise financial resources, human resources, and infrastructure or equipment. Ideally, the benefits of the intervention should come at reasonable, affordable and sustainable costs. One should consider that capital costs, such as for infrastructure development, even if initially high, may yield benefits in the long run.
The higher the incremental or recurrent costs, all other things being equal, the less likely it is to have a strong recommendation.
 Yes, globally  Yes, conditionally
All interventions require political commitment and wide stakeholder engagement as a prerequisite. In addition, "technical" feasibility requires functional organizational and institutional structures necessary to manage, follow through, and monitor the implementation of the recommendation. This comprises inter alia workforce planning and information systems, personnel management systems, regulatory frameworks, and monitoring and evaluation processes.
The elements of technical feasibility vary widely by country or context, but if these elements are likely to be functional in a wide variety of settings, the more likely is to have a strong recommendation.

# Overall ranking:
- Consider types of health workers and settings for which there is a lack of evidence - Consider stronger study designs and methods - Consider potential synergies between interventions.

# Table 4.1 Recommendation A1 -Targeted admission policies RECOMMENDATION A1
Use targeted admission policies to enrol students with a rural background in education programmes for various health disciplines in order to increase the likelihood of graduates choosing to practice in rural areas.
Population: students of health professions institutions Intervention: targeted admissions of students from a rural background into health profession schools

# Factors Decision Explanation
Quality of the evidence
 High  Moderate  Low  Very low
- Systematic review showed that rural background of students was associated with subsequent rural practice in 10 of 12 studies.
- Students from rural backgrounds are on average 2-3 times more likely to practice in rural areas.
- Long-term effects in increasing rural retention of graduates from rural background reported from high-income countries.
Values and preferences  No significant variability  Significant variability
- Preferential admissions policies may be seen as discriminatory and inequitable in some countries and may face legal or other challenges.
- However, affirmative action is an accepted practice in many countries. - Benefits: students from rural background are given opportunities for professional development and communities can benefit from supporting their own members.
- Disadvantages: students from rural areas may need special assistance, such as academic bridging, upgrading programmes or financial assistance, so they can compete with their urban counterparts for admission to medical schools and education programmes for other health disciplines.
Resource use - Potential extra costs for financial assistance, upgrading academic level of rural students (bridging programmes) and for supporting students once admitted, but such costs should be considered an investment.
Feasibility (or local factors that influence the translation of evidence into practice)
 Yes, globally  Yes, conditionally
- Some countries have put in place quota systems for recruiting students from rural backgrounds.
- Regulations are required, but no major controversies foreseen.

# Overall ranking: STRONG RECOMMENDATION

# Research gaps:
- most of the research comes from developed countries, and is about medical students - more studies from low-and middle-income countries needed - more longitudinal cohort studies are needed. - Only observational studies, few with large effects size.
- Graduates from rurally located medical schools or family medicine residency programmes are more likely to practice in rural areas.
- Rurally located medical schools produce more rural physicians than urban-based schools.
- But, difficult to determine the independent effect, as results may be confounded by significantly higher recruitment of students from rural backgrounds in these schools.

# Values and preferences
 No significant variability  Significant variability
- If dealing with limited resources, stakeholders may place more value on placing medical schools in their capitals or areas with a pre-established infrastructure, whereas others place higher value on these schools being more accessible, socially accountable and close to the rural areas. - Benefits: local training is likely to produce graduates whose competencies are appropriate and more relevant to local health needs.
- Disadvantages: results may only appear after a long lag time. Some concerns over the quality of the training, given that retention of faculty in rural areas is also an issue.
 Less resource- intensive  More resource- intensive
- Major resources required for the school's infrastructure.
- Additional costs needed for retention strategies for faculty/teachers/ tutors.
Feasibility (or local factors that influence the translation of evidence into practice)
 Yes, globally  Yes, conditionally
- Requires involvement and commitment of Ministry of Education.
- In some instances, regulation may be in favour: e.g. in some countries no more schools are allowed in the capital and so must be opened elsewhere.

# Overall ranking: CONDITIONAL RECOMMENDATION

# Research gaps:
- most studies are on medical doctors -need to study effects for other health professional schools - evidence is mostly from high-income countries -need for studies in low-and medium-income countries - stronger study design needed to assess role of confounders - need to assess the quality of training. - One cohort study showed that graduates with rural exposure were more likely to practice in rural areas.
- Many observational studies describe positive outcomes, but have no comparison groups.
- Confounders not addressed (self-selection of students from rural background for the rural placements).
Values and preferences - Benefits: rural-based training may allow health workers to "grow roots" in such locations, facilitates the development of professional networks, and increases awareness of rural health, even for those who may not eventually chose to practice there on a permanent basis.
- Disadvantages: it is not known how long the exposure should be, some may be very short experiences. - In some countries rural rotation and service is already mandatory and so this is a minor barrier.
Feasibility (or local factors that influence the translation of evidence into practice)
 Yes, globally  Yes, conditionally
- The system has to be in place -rural clinical placements require preceptors or mentors, which may not be available in many rural areas.
- Regulations have to be enacted to require clinical exposure.

# Overall ranking: CONDITIONAL RECOMMENDATION

# Research gaps:
- most studies are on medical doctors -need to study effects for other health professional schools - evidence is mostly from high-income countries -need for studies in low-and medium-income countries - stronger study design needed to assess role of confounders. - Observational studies highlight the importance of primary-care orientation in the production of rural health workers and show that those following rural curriculum perform equally, if not better, in medical examinations when compared with mainstream students.
- One study found that the best predictor for rural practice was a combination of rural background and interest in family medicine.  Yes, globally  Yes, conditionally
- Requires stakeholder engagement towards policies to change curricula.
- Requires acceptance of academic institutions.

# Overall ranking: STRONG RECOMMENDATION

# Research gaps:
- evaluations of impact of curricula changes on retention. - Difficult to assess direct effect because of confounders.
- Moderate evidence proving that it can lead to improvements in quality of care.
- Indirect evidence that continuing education programmes influence the desire to remain in rural practice. - Benefits: continuing education programmes are not only for knowledge acquisition and sharing but also for potential networking and reducing professional isolation.
- Disadvantages: programmes may be difficult to set up because of infrastructure and equipment required (distance education).
Resource use - Also travel costs if training is outside the practice location (this is one reason why programmes should be made available without leaving the rural area).
- Requires tutors and supervisors.
Feasibility (or local factors that influence the translation of evidence into practice)
 Yes, globally  Yes, conditionally
- Distance education is conditional on Internet access and availability of equipment.
- Needs to be linked with career paths to be more attractive to health workers.

# Overall ranking: CONDITIONAL RECOMMENDATION

# Research gaps:
- evidence mainly for physicians and nurses - evidence only from high-income countries - stronger study designs are needed. - No direct evidence on retention.
- Evidence from randomized controlled trials that quality of care and competence of nurses with enhanced scope of practice was similar if not better when compared with physicians in non-rural settings.
- Evidence from observational studies that advanced procedural skills training can enhance the confidence of family medicine residents in rural areas and improve their competence.

# Values and preferences
 No significant variability  Significant variability
- Nurses in rural areas would feel more valued and appreciated if recognition was given for the enhanced scope of practice which they get by default because of absence of physicians in rural areas.
- But, likely to have opposition from some professional bodies.
Absolute magnitude of effect  Large effect in the long term  Small effect for short duration - It is likely to have larger effects and for longer periods of time as nurses and other non-physicians clinicians are less inclined to leave rural areas compared with physicians.
- For a potentially larger effect, this intervention should be combined with B2 (production of different types of workers), A4 (adaptation of curricula for rural settings) and C1 (appropriate financial incentives).
- Additional effects can be obtained from association with recommendations in D category (personal and professional support).

# Balance of benefits versus disadvantages
 Benefits clearly outweigh disadvantages  Benefits and disadvantages are balanced  Disadvantages clearly outweigh benefits
- Benefits: in areas with absolute shortages of health workers, allowing an enhanced scope of practice to non-physician health workers can reduce the shortage gap while scaling-up the supply of highertrained health workers.
- Disadvantages: there is a risk that if interventions in D category are not associated, the increased scope of practice will then enable nurses or other types of workers to leave rural areas, and also a risk of dissatisfaction of professional associations and regulatory bodies.  Yes, globally  Yes, conditionally
- Supervision required.
- Regulatory framework needs to be agreed among all stakeholders (this may be highly political).

# Overall ranking: CONDITIONAL RECOMMENDATION

# Research gap:
- evidence needed on the direct effects of retention for an enhanced scope of practice in rural settings. - One observational study found higher retention rates for mid-level cadres compared with physicians, as well as similar if not better quality of services.
- Anecdotal evidence that non-physician clinicians are being produced in and are heavily relied upon in many sub-Saharan African countries, mainly for rural and remote areas, but no impact assessment.

# Values and preferences
 No significant variability  Significant variability
- Mid-level workers recognized and trusted by the community.
- Community members may place a higher value on the more permanent presence of mid-level staff than an irregular presence of a physician.
- Their skills are less "marketable", hence less likely to migrate.

# Absolute magnitude of effect
 Large effect in the long term  Small effect for short duration - It is likely to have larger effects over longer periods of time, because it takes less time to produce these types of workers, and they are less likely to leave rural areas, especially if they were specifically developed for these areas.
- For a potentially larger effect, this intervention should be combined with B2 (production of different types of workers), A4 (adaptation of curricula for rural settings) and C1 (appropriate financial incentives).
- Additional effects can be obtained from association with recommendations in D category (personal and professional support).

# Balance of benefits versus disadvantages
 Benefits clearly outweigh disadvantages  Benefits and disadvantages are balanced  Disadvantages clearly outweigh benefits
- Benefits: in places with an absolute shortage of health workers this intervention can fill the shortage gap while highly-trained health workers are being produced; and in many countries these new cadres have been specifically produced to serve rural areas as they receive training more specific to local health needs.
- Disadvantages: opposition from professional organizations (hence the need for wide stakeholder engagement).  Yes, globally  Yes, conditionally
- Supervision required.
- Regulatory framework needs to be agreed among all stakeholders (this may be highly political).

# Overall ranking: CONDITIONAL RECOMMENDATION

# Research gaps:
- evidence needed on the direct effects of retention for mid-level health workers compared with physicians - more solid evidence on intentions to leave of mid-level cadres compared with higher-trained health workers. - One retrospective cohort study showed high retention rates of obligated physicians after the completion of their mandatory service.
- As a recruitment measure, most studies reported high completion rates, particularly if intervention combined with other incentives.
- Few observational studies reported improvements in competence and job satisfaction during the obligated service, as well as service delivery.
- Although intervention implemented in many countries, there is a lack of analysis of the actual effects on long-term retention.

# Values and preferences
 No significant variability  Significant variability
- In free-market economies, higher value is placed by health workers on their freedom to choose the practice location, while policymakers would need immediate solutions to address acute shortages in rural areas.
- On the other hand, underserved communities would appreciate the presence of health workers in their settings, even if only for shorter periods of time (i.e. the duration of the obligatory service).
- In societies where high value is placed on altruism and equity, health workers may value the opportunity to "give back" to the community.
Absolute magnitude of effect  Large effect in the long term  Small effect for short duration
- Large size of effect expected for the duration of the obligatory service if it is an enforced national policy.
- If combined with incentives (such as licensing or rapid entry to civil service, or scholarships), programmes are likely to have higher recruitment and completion rates.
- If combined with A1 (targeted admissions) and C1 (financial incentives) is likely to have larger effects for a longer period of time. - Supervision required.
- Administration-heavy (matching of candidates, close monitoring to ensure adherence).
- Many stakeholders need to be engaged.

# Overall ranking: CONDITIONAL RECOMMENDATION

# Research gaps:
- no evidence on compulsory service for nurses and other types of health workers - need further evaluations conducted on retention of health workers after completion of obligatory service. - A systematic review found that financial-incentive programmes for physicians had an average recruitment rate of approximately 70% (14 studies) and retention rates between 12% to 90% (18 studies) of those students who entered the programme
- But serious confounders due to self-selection of participants in the programmes.
- All studies about physicians, and most of them from United States of America (the rest from Canada, Japan, New Zealand and South Africa).

# Values and preferences
 No significant variability  Significant variability
- Programmes may have equity benefits if they preferentially place physicians in areas of greatest needs.
- However, if programmes applied to first-year medical students, preferences for work in rural areas usually change during medical school, so investment may be lost, particularly if there is a buy-out option.
Absolute magnitude of effect  Large effect in the long term  Small effect for short duration - In general, programmes are successful in placing physicians in rural areas for only the duration of the obligatory service.
- But, if combined with A1 (targeted admissions), is likely to have larger effects for a longer period of time. - Benefits: availability of health workers in remote or rural communities with absolute shortages of health workers, even if for shorter periods of time.
- Disadvantages: in the long-run, high turnover and consequently decreased continuity and quality of care. If a buy-out option exists, there is the risk that participants pay their way out of the programme. If applied to first-year medical students, their preferences for practice location may change at the end of the studies. - Very little information on costs of programmes, but is likely that scholarships and bursaries would require significant upfront costs, and subsequent recurrent costs for maintaining the programme. This may be an issue in low-income countries.
Feasibility (or local factors that influence the implementation)
 Yes, globally  Yes, conditionally
- Supervision required.
- Administration-heavy (matching of candidates, close monitoring to ensure adherence).
- It may not be affordable for middle-and low-income countries, but opportunities for using external aid should be examined.

# Overall ranking: CONDITIONAL RECOMMENDATION

# Research gaps:
- no evidence on financial incentives for return of service for nurses and other types of health workers. - One before-and-after and one observational study found that retention rates following financial incentives schemes increase moderately and for only short term.
- Supportive evidence shows that financial incentives are always one key element influencing preferences for work in rural areas, but not the most important.
- Supportive evidence shows that the amount of the financial incentives provided must outweigh the opportunity costs of living in rural areas.
Values and preferences - The incentive package should include both monetary benefits (bonuses) and non-monetary benefits (grants for children's education, loans for housing, starter medical kit).
- It can be more cost-effective if there is a surplus of health workers in urban areas (labour market analysis required).

# Feasibility (or local factors that influence the implementation)
 Yes, globally  Yes, conditionally
- It is one intervention that is most readily adopted by policy-makers, as it can provide a short-term solution to the problem. However, long-term implications and sustainability need to be carefully considered.  - Apart from possible high investment costs, no major barrier to implementation.

# Overall ranking: STRONG RECOMMENDATION

# Research gaps:
- studies on the effects/impact on recruitment and retention, but also need to implement more such interventions to be able to assess their effectiveness. - Benefits: improving working conditions is likely to improve also the performance and productivity of health workers, and hence the performance of health systems.
- Disadvantages: small-scale pilot projects may attract health workers from other areas with shortages, thus further exacerbating the imbalances (need a coordinated approach).
Resource use  Less resourceintensive  More resourceintensive
- Equipment and refurbishing of health facilities may be resource intensive, but benefits can be achieved for a longer term.
- Changes in management style and implementing supportive supervision may also require significant investment in management training courses and in effective supervision processes.
- Holistic strategies to prevent workplace violence can also be complex and costly.
Feasibility (or local factors that influence the implementation)  Yes, globally  Yes, conditionally
- Should be part of wider health systems strengthening efforts.

# Overall ranking: STRONG RECOMMENDATION

# Research gaps:
- studies on the effects/impact on recruitment and retention, but also need to implement more such interventions to be able to assess their effectiveness. - No direct evidence on recruitment or retention.
- A Cochrane systematic review (only nine good quality studies included) found that specialist outreach can improve access to care, certain health outcomes and service use, especially when delivered as part of a multifaceted intervention, involving collaboration with primary care, education or other services. However, most studies were from urban settings, and small size of effects was reported in terms of improved access to care.
- Controlled studies on virtual outreach (telehealth) show increased access to specialist services, and reduced referral times, but small size of effect.

# Values and preferences
 No significant variability  Significant variability
- It can be valued by rural health workers as providing additional support, but it can also be perceived as limiting the "individuality" of rural practice.
- Not all remote or rural areas are attractive for specialists to provide outreach services (select the ones closest to the cities).
Absolute magnitude of effect  Large effect in the long term  Small effect for short duration
- Small-scale projects, often difficult to scale up at national level.
- It is likely to be more beneficial in settings where there is limited infrastructure or with very sparse populations, as it provides a service that otherwise would not be available (e.g. mobile clinics or flying-in services). - Access to technology can be a serious limitation in low-income countries, although rapid advances in technology can be seen even in these settings (particularly on the use of mobile phones).

# Overall ranking: CONDITIONAL RECOMMENDATION

# Research gaps:
- studies on the effects/impact of outreach support activities on the recruitment and retention of physicians and other types of health workers (outreach support activities may include: outreach specialist support, mobile health clinics, telehealth, use of ICT and e-health). - It requires regulations of the civil service and negotiations with professional associations to define the career paths.

# Overall ranking: STRONG RECOMMENDATION

# Research gaps:
- studies on the effects/impact of career development programmes on recruitment and retention. - Benefits: reduced feeling of isolation, improved access to continuous education. In some instances the rural professional associations can provide support/lobby for larger health system or public health reforms.
- Disadvantages: if professional associations supported by membership fees, it can be vulnerable to long-term sustainability.
This section presents an overview of the methods used in developing these recommendations.

# Literature review
As mentioned in Chapter 1, the process to develop these recommendations started off with a literature review conducted by the WHO Secretariat in 2008, which formed the basis of the first expert consultation in February 2009. For this review, peer-reviewed publications as well as "grey" literature were examined.
Electronic searches were conducted in August and September 2008 in PubMed, the Cochrane database, Embase and LILACS. Reference lists of the retrieved studies were also searched to complement the final list of articles. Further evidence was gathered from experts in the field of human resources for health, hand searches of the journal Human Resources for Health and the Journal of Remote and Rural Health, as well as from grey literature, through searches in GoogleTM, the Human Resources for Health (HRH) Global Resource Centre and various government ministries' websites.
The following subject headings and text words and a combination thereof were used: doctors, nurses, midwives, mid-level health workers, community health workers, health managers, lab technicians, health worker, health professional, human resources for health, health workforce, health technician, clinical engineer, health teams, physician in combination with: rural, remote, underserved, rural/urban imbalances, maldistribution; retention, recruitment, retention, retention strategies/retention strategy, retention scheme; financial incentive, monetary incentive, non-financial incentive, non-monetary incentive, allowances, salaries, benefits; compulsory service, bonding scheme; rural pipeline, professional development, professional support, telemedicine; vacancy rates, motivation, patient satisfaction, utilization of services, duration in service; and evaluation, impact, programme result. Although the main search was conducted in English, substantial efforts were made to gather studies in French, Portuguese, Spanish and Scandinavian languages, with the support of consultants in those regions.
The review included articles that were published between 1995 and September 2008, from both developed and developing countries and covering all types of health workers. The inclusion criteria stated that the study must report on the results/effects of an intervention, have a focus on remote or rural areas, and have a clear description of the study design and methods used. News and editorials were excluded as they did not report on effects of interventions.
The background paper summarizing this initial literature review was presented at the first full expert group meeting in February 2009 and served as a basis for the group to agree on the research questions, to establish the scope of the guidelines, and to identify the research gaps (19).
Based on these gaps and the plan of action agreed upon by the expert group, additional research was commissioned to some of the experts in the group (104). These are presented in one of the following sections below. Subsequent ad hoc searches were conducted through early 2010 to ensure no essential studies were missed during the expert group's work on the recommendations.

# Available/published systematic reviews
Several systematic reviews already available in this field were particularly instrumental in helping collect evidence and complete the GRADE evidence profiles.

# Grobler et al. (2009) Interventions for increasing the proportion of health professionals practising in rural and other underserved areas (Review)
This comprehensive systematic review was developed for the Cochrane library and aimed to assess the effectiveness of interventions targeted towards increasing the proportion of health professionals working in rural and underserved areas (35). No studies were found that met their inclusion criteria (randomized controlled trials, controlled trials, controlled before-and-after studies and interrupted time series evaluating the effects of recruitment or retention strategies). Of the six studies that did meet the study design criteria, all were excluded as they only reported indirect outcomes. Despite this, the authors did provide examples of current strategies to address the maldistribution of health professionals, all of which correspond to the four categories of interventions used in these recommendations. According to the authors: "Strategies that have shown promise include selection of students with a rural background, the establishment of university departments and/or teaching clinics in rural areas, rural and scarce skills allowances and enhanced professional and personal support."
As with most other reviews conducted in this area of health policy research, the review concludes that better designed evaluations are required to understand the impact and effectiveness of these strategies. The WHO Secretariat used this review as key evidence for the background paper, yet chose to conduct an additional comprehensive review (as detailed above) in order to capture and analyse the many studies that did not meet the inclusion criteria of this review. It was decided that complex interventions, such as recruitment and retention strategies, warranted further analysis of descriptive and observational studies, as these can often provide essential information and insight into why and how certain retention strategies work in some contexts and not in others.

# Wilson et al. (2009) A critical review of interventions to redress the inequitable distribution of health-care professionals to rural and remote areas
This study built on the above Cochrane review and expanded the scope of the search to include studies it had excluded, with the aim of providing a more comprehensive overview of studies in this domain, and present the findings in a simpler way for policy-makers to understand (36). A total of 110 articles were included in the final study. The authors chose to present the findings under given intervention categories: selection, education, coercion, incentives and support. None of the evidence included in the review was rated as "convincing", although evidence in the "selection" category was rated as "strong". The majority of evidence retrieved was from high-income countries. The review calls for more scientifically rigorous evaluations to be conducted. This study was conducted independently of the work of the expert group, but the findings were quite similar with the WHO background paper and the deliberations of the expert group. Subsequently, one of the co-authors of this study was co-opted in the expert group.

# Gruen et al. (2009) Specialist outreach clinics in primary care and rural hospital settings (Review)
This systematic review sought to describe and assess the effectiveness of specialist outreach clinics on various outcomes, including access and patient satisfaction (86). It was largely consulted for recommendation D3, highlighting that outreach services to rural populations did lead to an increase in specialist consultations and a higher proportion of patients receiving correct breast-cancer care. Although 73 outreach interventions were identified in this review, only nine met the study design inclusion criteria and most were urban outreach programmes. There were no outcomes reported in relation to the retention or recruitment of health personnel. One of the experts complemented this Cochrane review with another review on this topic, providing further details and explanations of the different outreach support schemes that have been or are currently being implemented in different countries, and particularly those in remote and rural areas.
adequate descriptions of the characteristics of complex interventions used to recruit and retain health workers. And unlike strictly clinical interventions, using controls or fixing for all variables or confounding factors is extremely challenging and sometimes impossible for complex health policy interventions and is absent in many studies (105).
The evidence profiles in Annex 1 present the grading of the evidence for each recommendation and more information on the GRADE methodology. A GRADE evidence profile was prepared for each recommendation with the intention of presenting the highest quality evidence available for that recommendation (additional evidence is captured in the descriptive evidence tables in Annex 2).
The following GRADE criteria were used for assessing the quality of the evidence: In general, the higher the quality of the evidence, the stronger the recommendations are. However, as noted in Chapters 3 and 6, the quality of the evidence was only one of the criteria used to determine the eventual strength of the recommendations. Due to the limitations of using GRADE to assess the quality of the evidence for health policy interventions (as opposed to clinical interventions) equal consideration was given to other criteria such as balance between benefits and risks, values and preferences, and resource use. As a result, there are some strong recommendations associated with low-quality evidence.

# Descriptive evidence tables
Detailed evidence tables can be found in Annex 2. These were developed at the request of the expert group in order to provide additional evidence to that captured in the GRADE evidence profiles. These descriptive tables present short summaries of approximately 100 studies that were considered in the development of these recommendations. The tables include all the studies that appear in the GRADE evidence profiles, plus additional descriptive studies, papers that analyse the factors that influence health workers' decisions to go to, stay in and leave remote and rural areas, and regional or global literature reviews related to the recruitment and retention of health workers.
Policy-makers in all countries, regardless of their level of economic development, struggle to achieve health equity and to meet the health needs of their populations, especially vulnerable and disadvantaged groups. A shortage of qualified health workers in remote and rural areas impedes access to health-care services for a significant percentage of the population, slows progress towards attaining the Millennium Development Goals and challenges the aspirations of achieving health for all. This document proposes a set of evidence-based recommendations to increase the recruitment and retention of motivated health workers in rural and remote areas and provides guidance on how to implement retention strategies in both developed and developing countries.
Steve J. Reid, University of Cape Town, Cape Town, South Africa Felix Rigoli, WHO Regional Office for the Americas, Washington, DC, USA Eric de Roodenbeke, International Hospital Federation, Ferney-Voltaire, France Salif Samake, Ministere

# Barnighausen et al. (2009)
Financial incentives for return of service in underserved areas: a systematic review 2 This systematic review retrieved 43 studies, 34 from the USA and the remaining studies from Canada, Japan, New Zealand and South Africa (71). All studies retrieved were observational studies and considered various types of loans and scholarship schemes. This systematic review provided the majority of the evidence for recommendation B4. Although there were weaknesses in the study designs acknowledged, there were encouraging results reported in relation to retention in underserved areas following the initial obligation period.
Further details about the findings of these four reviews are presented in the GRADE evidence profiles (Annex 1).

# Country case studies and commissioned reports
As mentioned above, additional research was commissioned by the WHO Secretariat to fill in the evidence gaps identified during the first expert consultation. This included specific systematic reviews and a series of country case studies.
Three reviews were commissioned. One review examined the impact of compulsory service on the recruitment and retention of health workers in rural areas, which formed the basis of recommendation B3 (64). The second review was a "realistic" evaluation, which applied theory-based methods to the original findings of the WHO background paper with the aim of understanding why and how certain interventions worked (82). Finally, the third review was on the role of outreach support on the recruitment of health workers in remote and rural areas (90). This review built on and expanded the original review by Gruen et al., by providing more examples of outreach support activities in rural areas.
The country case studies were commissioned in order to better understand the contextual elements that influence retention strategies in different situations. They used a common template developed by the expert group, and included the following countries: Australia, China, Ethiopia, the Lao People's Democratic Republic, Mali, Norway, Samoa, Senegal, Vanuatu and Zambia. Some of the case studies are still ongoing, but it is expected that it will be possible to draw comparative lessons from the various contexts of these countries on the planning, implementation and evaluation of different retention strategies.
These country case studies and the three reports mentioned above will all be published as standalone documents and will be accessible online at: /.

# GRADE evidence profiles
The GRADE (Grading of Recommendations, Assessment, Development and Evaluation) methodology was used in the development of these recommendations. GRADE presents a systematic and transparent way of assessing and grading the quality of the evidence.
WHO staff from the Health Workforce Retention and Migration Unit were trained and acquired skills in using the GRADE methodology and prepared the GRADE evidence tables, with support from a member of the GRADE working group. There were certain challenges in using GRADE for these recommendations. For example, GRADE does not allow for consideration of key contextual issues or provide space for the inclusion of