## Implementation Research In Health A Practical Guide

©WHO
David H. Peters, Nhan T. Tran, Taghreed Adam

## Implementation Research In Health: A Practical Guide

David H. Peters, Nhan T. Tran, Taghreed Adam

## Who Library Cataloguing-In-Publication Data

Implementation research in health: a practical guide / David Peters … [et al]. 1.Health services research - standards. 2.Health policy. 3.Delivery of health care. 4.Research design. 5.Health plan implementation. I.Peters, David. II.Tran, Nhan. III.Adam, Taghreed. IV.Alliance for Health Policy and Systems Research. V.World Health Organization. ISBN 978 92 4 150621 2	 	
	
	
(NLM classification:  W 84.3)

## © World Health Organization 2013

All rights reserved. Publications of the World Health Organization are available on the WHO web site (www.who.int) or can be purchased from WHO Press, World Health Organization, 20 Avenue Appia, 1211 Geneva 27, Switzerland (tel.: +41 22 791 3264; fax: +41 22 791 4857; e-mail: bookorders@who.int). Requests for permission to reproduce or translate WHO publications –whether for sale or for non-commercial distribution– should be addressed to WHO Press through the WHO web site (www.who.int/about/licensing/copyright_form/en/index.html). The designations employed and the presentation of the material in this publication do not imply the expression of any opinion whatsoever on the part of the World Health Organization concerning the legal status of any country, territory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries. Dotted lines on maps represent approximate border lines for which there may not yet be full agreement. The mention of specific companies or of certain manufacturers' products does not imply that they are endorsed or recommended by the World Health Organization in preference to others of a similar nature that are not mentioned. Errors and omissions excepted, the names of proprietary products are distinguished by initial capital letters. All reasonable precautions have been taken by the World Health Organization to verify the information contained in this publication. However, the published material is being distributed without warranty of any kind, either expressed or implied. The responsibility for the interpretation and use of the material lies with the reader. In no event shall the World Health Organization be liable for damages arising from its use. The named authors alone are responsible for the views expressed in this publication. Printed by the WHO Document Production Services, Geneva, Switzerland Designed by Corrales Creative Suggested citation: David H. Peters, Nhan T. Tran, Taghreed Adam. Implementation research in health: a practical guide. Alliance for Health Policy and Systems Research, World Health Organization, 2013.

## Acknowledgements

This Guide was developed by the Alliance for Health Policy and Systems Research (AHPSR) with support from the Implementation Research Platform (IRP). The IRP was formed in recognition of the need for reliable, relevant research to inform the implementation of health policies and programmes, and to ensure that needed interventions are made widely available within health systems through effective scale-up.  The AHPSR is the host of the IRP Secretariat and leads its work. The AHPSR's main goal is to promote the generation and use of health policy and systems research as a means to improve health and health systems in low- and middle-income countries. Although the Guide has benefited from the contributions and advice of many individuals, responsibility for the views expressed and for any errors of fact or judgment rests with the authors. In particular, the authors would like to acknowledge Irene Agyepong for her contributions to the planning, case-studies, and review of the Guide; George Pariyo for his contributions to the planning and review of the Guide; and Sally Theobald for her contributions to the case-studies and review of the Guide. The authors would also like to thank the following individuals for their review and feedback on the Guide: Garry Aslanyan, Rajiv Bahl, Neal Brandes, Somsak Chunharas, Soraya Elloker, Abdul Ghaffar, Lucy Gilson, Margaret Gyapong, Luis Huicho, Jose Martines, Karstein Maseide, Garrett Mehl, Olumide Ogundahunsi, Kelechi Ohiri, Enola Proctor, Krishna Rao, Suzanne Reier, Abha Saxena, Jim Sherry, Rajendra Shukla, and Jeanette Vega. Finally, we would like to acknowledge Gary Humphreys for his help in bringing this project together on the page.

## Preface

One of the greatest challenges facing the global health community is how to take proven interventions and implement them in the real world. Research on health systems, such as implementation research, is crucial to meeting that challenge, providing a basis for the context-specific, evidence-informed decision-making needed to make what is possible in theory a reality in practice. The World Health Organization (WHO) has long played a leading role in promoting health policy and systems research (HPSR)—including implementation research, with notable recent initiatives including the 2011 report: Implementation Research for the Control of Infectious Diseases of Poverty and the 2012 publication of its first strategy on HPSR called 'Changing Mindsets', which advocated for greater embedding of research into decisionmaking and called for more demand-driven research. With this Guide, WHO continues its support for this area, offering an introduction to the often challenging field of implementation research. Because implementation research is a relatively new and somewhat neglected field, there is a need to bring it into sharper focus, defining exactly what it is and what it can offer. As such, this Guide presents an introduction to basic concepts used in implementation research and describes the range of approaches and applications that it can be used for. The main aim of the Guide is to support the development of and demand for implementation research that is problem-focused, action-oriented and above all aligned with health system needs. Research on implementation requires the engagement of a wide range of stakeholders and draws on multiple disciplines in order to address complex implementation challenges. As this Guide points out, at its best, implementation research is often a collective and collaborative endeavor and in many cases it is people working on the front line of health care, whether running specific programmes, or working in health systems, who ask the questions around which it is built. It is therefore essential that all stakeholders understand the importance of collaboration in the implementation research endeavor. It is our hope that this Guide will encourage that collaboration, and facilitate the coming together of stakeholders across the broad spectrum of health systems, all of which, on a daily basis, wrestle with the challenge of implementation.

## Foreword

Interest in implementation research is growing, largely in recognition of the contribution it can make to maximising the beneficial impact of health interventions. As a relatively new and, until recently, rather neglected field within the health sector, implementation research is something of an unknown quantity for many. There is therefore a need for greater clarity about what exactly implementation research is, and what it can offer. This Guide is designed to provide that clarity. Implementation research engages a wide range of interventions in a health system, and for the purposes of the Guide we consider a wide variety of policies, programmes, as well as individual practices and services intended to improve people's health.  Regardless of the way these interventions work - be it through the prevention of disease, promotion of good health, or treatment and/or palliation of illness - research on their implementation is crucial to understanding how they work in the real world. Intended to support those conducting implementation research, those with responsibility for implementing programmes, and those who have an interest in both, the Guide provides an introduction to basic implementation research concepts and language, briefly outlines what it involves, and describes the many opportunities that it presents. The main aim of the Guide is to boost implementation research capacity as well as demand for implementation research that is aligned with need, and that is of particular relevance to health systems in low- and middle-income countries (LMICs). Research on implementation requires the engagement of diverse stakeholders and multiple disciplines in order to address the complex implementation challenges they face. For this reason, the Guide is intended for a variety of actors who contribute to and/or are impacted by implementation research. This includes the decision-makers responsible for designing policies and managing programmes whose decisions shape implementation and scale-up processes, as well as the practitioners and front-line workers who ultimately implement these decisions along with researchers from different disciplines who bring expertise in systematically collecting and analysing information to inform implementation questions. It is also our hope that the Guide may appeal to educators who teach implementation research, to funders of health research and health programmes who may be interested in supporting this type of research, and to civil society groups interested in health programmes and research who may want to use this evidence to promote good clinical and public health practice. 

While for the sake of simplicity we refer in the Guide to implementers and researchers, we recognize that these are notional groupings, and that many people play both roles. Indeed it is one of the core contention of the Guide that the interests of implementation research are often best served where there is active participation by people working in the field, since it is those people who see where implementation is going wrong and who are therefore likely to ask the most pertinent questions. We encourage implementers to ask those questions, and to take a lead in demanding better research. We also encourage researchers to be more practical in their work, focusing on the issues that matter to implementers. If research is to improve implementation, research design needs to reflect the specific implementation problems implementers are addressing as well as the contexts in which they occur. The opening chapters make the case for why implementation research is important to 
 
decision-making. They offer a workable definition of implementation research and illustrate the relevance of research to problems that are often considered to be simply administrative and provide examples of how such problems can be framed as implementation 
 
research questions. The early chapters also deal with the conduct of implementation research, 
 
emphasizing the importance of collaboration and discussing the role of implementers in the planning and designing of studies, the collection and analysis of data, as well as in the dissemination and use of results. The second half of the Guide detail the various methods and study designs that can be used to carry out implementation research, and, using examples, illustrates the application of quantitative, qualitative, and mixed-method designs to answer complex questions related to implementation and scale-up.  It offers guidance on conceptualizing an implementation research study from the identification of the problem, development of research questions, identification of implementation outcomes and variables, as well as the selection of the study design and methods while also addressing important questions of rigor.  

## Executive Summary

A key challenge faced by the global health community is how to take proven interventions and implement them in the real world. Affordable, life-saving interventions exist to confront many of the health challenges we face, but there is little understanding of how best to deliver those interventions across the full range of existing health systems and in the wide diversity of possible settings. Our failure to effectively implement interventions carries a price. Each year more than 287,000 women die from complications related to pregnancy and child birth, for example, while approximately 7.6 million children, including 3.1 million newborns, die from diseases that are preventable or treatable with existing interventions. 

## Understanding Implementation In The Real World

Implementation issues arise as a result of a range of factors including 'real world' contextual factors that are either overlooked or not captured by other research disciplines. Implementation research shines a light on those factors, providing the basis for the kind of context-specific and evidence-informed decision-making that is crucial to making what is possible in theory a reality in practice. Because implementation research is embedded in reality, people working in the real world (practitioners as opposed to people 'doing research') often ask the questions that are the starting point for new thinking. Making sure that those questions are heard, and that the research undertaken is directed at finding answers to the questions asked rather than at the topics researchers themselves may find interesting is one of the key challenges implementation researchers face.

## A Practical Tool

Embedded in the real world, implementation research is also a powerful tool for capturing and analysing information in real time, allowing for the assessment of performance, for example, and facilitating health systems strengthening. Implementation research is particularly important in supporting the scale-up of interventions and their integration into health systems at the national level. Too often interventions that work in small-scale pilot studies fail to live up to expectations when rolled out in national strategies, or fail to transfer from one country to another as a result of contextual differences. Implementation research not only helps to clarify why that happens, but can be used to support the process of re-iterative refinement needed for successful adaptation. The same capacities make implementation research a useful tool for helping organizations develop the capacity to learn, enabling them to assimilate and put into effect knowledge developed on an iterative basis. 

## A Collaborative Endeavor

Implementation research is often at its most useful where implementers have played a part in the identification, design and conduct phases of the research undertaken. It is for this reason that the fostering of collaborative ties between key stakeholders involved in policy generation, programme management, and research is so important. One way to support collaboration between researchers and implementers is to integrate implementation research into policy and programmatic decision-making processes right from the beginning, rather than an endeavor that is carried out separately from the implementation process. In this way, scientific inquiry can also be integrated into the implementation problem-solving process in an iterative, and continuous manner. Implementation research can also play an important role in acting as a vehicle for grass roots communities by identifying neglected issues, exposing performance shortcomings and increasing the accountability of health organizations. In all of these collaborative endeavors, implementation researchers are called upon to be receptive and flexible in the work they do. Indeed the understanding of context and systems, and the flexibility to identify appropriate methodological approaches, can be as important as or even more important than adherence to a fixed research design.  

## A Broad Research Spectrum

Broadly speaking, the term implementation research describes the scientific study of the processes used in the implementation of initiatives as well as the contextual factors that affect these processes. It can address or explore any aspect of implementation, including the factors affecting implementation (such as poverty, geographical remoteness, or traditional beliefs), the processes of implementation themselves (such as distribution of fully-subsidised insecticide-treated bednets (ITNs) through maternal health clinics, or the use of mass vaccination versus surveillance-containment), and the outcomes, or end-products of the implementation under study. Implementation research is applicable and relevant to many different research domains,  and to different degrees, depending on the subject under study. For example, basic research into new medicines typically involves no implementation issues, while ensuring that those medicines are available to the people who need them does. Implementation research often focuses on the strategies needed to deliver or implement new interventions here called 'implementation strategies', a term used to distinguish them from clinical and public health interventions. In order to study implementation processes, a framework for conceptualizing and measuring implementation outcomes based on 
 
variables such as 'acceptability', 'appropriateness' and  'feasibility', can be used to 
 
understand how well a given implementation process is actually working. 

## A Wide Range Of Approaches

Because it draws on a wide variety of research approaches and disciplines, it makes little sense to talk in terms of a set of 'implementation research methods.' However, certain research approaches and designs - including pragmatic trials, effectiveness–implementation hybrid trials, quality improvement studies and participatory action research -  are particularly useful because they generate actionable intelligence, are good at capturing the subtleties of context over time, and offer the iterative flexibility needed to respond to change.  While such tools are vital to the implementation researcher, it is important to bear in mind that in implementation research, the "question is king" which is to say that it is the question that determines the method used, rather than the method that determines the kinds of questions asked. Implementation research questions are often complex, reflecting the wide array of contextual factors that can influence implementation, producing unpredictable effects, and requiring continuous adaptation by implementers.  Embracing that complexity requires considerable flexibility on the part of researchers, particularly in regard to the complex and dynamic nature of the subject under study.  

## Aligning Research With Need And Ensuring Quality

Ideally, implementation research should be aligned with need, both in the sense that it addresses the concerns of the intended audience, and is also responsive to the particularities of the subject under study. A key consideration in this regard is the level of certainty required regarding results or projections. A policy-maker working with his or her own constraints, for example, may be looking for strong indications that a given intervention will work, but may not necessarily have the time required for multi-year studies that could generate a higher level of certainty. Responding to the needs of different audiences, may have important implications for the basic design of research, and for budget and scheduling. In order to ensure that implementation research is aligned with need and of high quality, it is helpful to ask the following key questions:

k
k Does the research clearly address a question concerning implementation?
k
k Is there a clear description of what is being implemented (e.g. details of the 
 
practice, programme, or policy)?
k
k Does the research involve an implementation strategy? If so, is it described and 
examined appropriately?
k
k Is the research conducted in a real-world setting? If so, are these conditions described in sufficient detail?
k
k Does the research appropriately consider implementation outcome variables?
k
k Does the research appropriately consider context and other factors that influence 
implementation?
k
k Does the research appropriately consider changes over time, and the level of complexity of the system?
k
k Does the research clearly identify the target audience for the research and how it 
can be used?

## Getting More Out Of Implementation Research

Despite the importance of implementation research, it continues to be a neglected field of study, partly because of a lack of understanding regarding what it is and what it offers, and partly because of a lack of investment in implementation research activities. We spend billions on health innovations, but very little on how best to use them. This problem affects everyone, but in particular populations in low- and middle-income countries where the implementation challenges are greatest. This guide is an attempt to redress the deficit in understanding of implementation research and to encourage programme personnel and implementers to take a greater interest in the subject, recognizing that implementation research is in fact an integral part of programme planning and execution, rather than something that happens once programmes are up and running, and conducted largely for the benefit of other researchers. For their part implementation researchers can do much more to engage with implementers and programme personnel in research process. Only by coming together can implementers, with their intimate understanding of context, and researchers, with their understanding of the methods and science of inquiry, hope to advance our understanding of the implementation issues that compromise so many of our public health endeavors. 

## Why Is Research On Implementation Needed? 1 Key Points

k
k
Despite abundant evidence of the efficacy of affordable, life-saving interventions, there is little understanding of how to deliver those interventions effectively in diverse settings and within the wide range of existing health systems. 
k
k
Implementation issues often arise as a result of contextual factors that policy-makers and health system managers may not even have considered. 
k
k
Implementation research is crucial to improving our understanding of the challenges we face in confronting the real world by broadening and deepening our understanding of these real-world factors and how they impact implementation.
k
k
Implementation research is of immense value in shining a light on the often bumpy interface between what can be achieved in theory and what happens in practice. 

## Why Is Research On Implementation Needed? " Neglecting Implementation Challenges Costs Lives And Money."

lages and vaccinating the people who had not yet caught the disease, effectively ring-fencing the hot spots with a barrier of immunity.  Realizing that some people, many of whom might be unaware that they were infected, were going to be moving around, notably to the markets where they bought and sold food and goods, Foege also mapped out local transportation routes and the markets they served. Then, using the remainder of the vaccine available to him, he built rings of resistance in the areas he had identified as being most likely to see new cases. 

Despite abundant evidence of the  
efficacy of life-saving interventions, there is little understanding of how to deliver those interventions effectively.

In December 1966 Doctor William Foege drove into the Eastern Nigerian bush to investigate a reported smallpox outbreak. A great deal was already known about how to deal with the disease at that time; indeed a global campaign to eradicate smallpox had been running since 1959, and there had been several breakthroughs in vaccine production and delivery, notable among which the development of freeze-dried smallpox vaccine, and the use of a simple bifurcated needle to deliver an effective dose. But at the end of 1966 smallpox was still circulating in 31 countries and territories, and there were concerns that the goal upon which the whole campaign was based - mass vaccination - was unattainable. In reality mass vaccination meant vaccinating 80% of the population concerned, a level at which so-called herd immunity is achieved.  But even with good quality vaccine and the simple magic of the bifurcated needle, achieving that level of coverage was proving extremely difficult [1]. There was a problem of implementation.

This surveillance-containment strategy -  as it came to be known - shut down smallpox outbreaks in Eastern Nigeria in just five months, and was achieved by vaccinating just 750 000 of a population of around 12 million [2]. 

Foege was by no means the first to use the method, indeed a similar approach had been used to control outbreaks in England in the 19th century, but he was the first to use it in the demanding conditions of sub-Saharan Africa, and the achievement resonated. The problem of achieving herd immunity had been shown to be irrelevant. The power of the approach was soon confirmed in other African countries (see Figure 1), and even more impressive results were achieved elsewhere, notably in the Indian state of Tamil Nadu, where, the following year, D. A. Ramachandra Rao led a single team on a surveillance-containment campaign which halted smallpox transmission among the 41 million inhabitants in just six months [2]. Global scale-up of surveillance-containment followed, and smallpox was officially declared eradicated in 1979.  
Then came Nigeria.  Having confirmed that the reported outbreak was indeed smallpox, Foege set about mounting a response, and realized that he did not have sufficient vaccine to achieve the mass vaccination of the population, only 35% of which was currently vaccinated against the disease. Meanwhile, fresh supplies of vaccine and the extra vehicles needed to transport it were several weeks away. He needed a backup plan.

Of course the implementation of the surveillance-containment strategy was not, strictly speaking, implementation research or what was then known as 'field research' - in many ways a more descriptive term. Foege was simply responding to an outbreak. However, the results of his efforts were of immense value for research, and, as In the end, that plan involved going to the affected vilthis Guide will show, some of the most significant breakthroughs in implementation knowledge have come from people who were not 'doing research' at the time of their discovery.

Very often it is the person in the field - the doctor in the remote rural clinic or the midwife working in the local community - who, facing some particular problem, asks the questions that are the starting point for new thinking. 

Roughly half a century since Foege made that drive into the Eastern Nigerian bush, we are still struggling to make the best use of the vaccines available to us, even if the challenges we face have, generally speaking, changed. 

At present, LMICs seeking to implement vaccination strategies are more likely to be struggling to introduce new vaccines or integrate immunization programmes into the services routinely provided by their health systems. In doing so, they face a range of obstacles to effective implementation that include managerial, systems, socio-behavioural, and financial challenges,  any one of which can limit the impact of a vaccination programme and hinder progress towards better health. 

One of the central messages of this Guide is that contributions to implementation research can be made by people both inside and outside academia, and that very often it is the person in the field - the doctor in the remote rural clinic or the midwife working in the local community - who, facing some particular problem, asks the questions that are the starting point for new thinking. Making sure that those questions are heard, and that the research undertaken is directed at finding answers to the questions asked rather than at the topics researchers themselves may find interesting is a subject that we will return to. 

Just as it underpinned efforts to scale-up Foege's surveillance-containment approach to smallpox immunization, effective implementation research can play a vital role in improving our understanding of these challenges and 

## Chapter 1 - Why Is Research On Implementation Needed?

support the implementation of routine immunization programmes within health systems. In this way, immunization, instead of being delivered only through special campaigns, can be delivered on a sustainable basis as part of routine new-born infant and child health services. And the same is true of many other interventions, which, like immunization, often fall short of their promise through failures of implementation. Despite the medical and public health advances of the past century, each year more than 287 000 women die from complications related to pregnancy and child birth, and approximately 7.6 million children, including 3.1 million newborns, die from diseases that are preventable or treatable with existing interventions [3]. More often than not, these deaths are the result of problems with Source: Mandala et al., 2009 [5]
implementation, problems that can be elucidated with carefully designed, planned and executed implementation research. As stated in the introduction, the central concern of this Guide is implementation research as it relates to LMICs, where, despite abundant evidence of the efficacy of affordable, life-saving interventions, there is little understanding of how to deliver those interventions effectively in diverse settings and within the wide range of existing health systems [4]. We know that insecticide-treated bednets reduce malaria transmission rates, but need more research into the most effective and financially sustainable way to deliver those bednets and how to ensure that they are properly used. We know that oral rehydration therapy (ORT) is extremely effective in treating diarrhoearelated disease, but struggle to achieve adequate levels of ORT use. Similarly, we know that anti-retroviral treatment (ART) programmes can prolong the lives of people living with HIV, but too often fail to ensure that everyone who needs treatment gets it. As shown in Figure 2 ,only a small proportion - less than 4%, of women who test positive for HIV during pregnancy in Zambia - are actually initiated on ART in order to prevent mother-to-childtransmission (PMTCT) of HIV [5].    
Properly conducted implementation research, with its all-important focus on context can help implementers foresee and anticipate problems. 

Both the United Republic of Tanzania and Ghana have polyester textile mills capable of making nets for ITN purposes, but only in the United Republic of Tanzania did these mills convert some of their textile production into nets, largely as a response to direct appeals by the government and the business opportunity provided by the initial social marketing for the ITN programme. In Ghana, private sector businesses produced locally-sewn nets. Unfortunately, during the roll-out of the voucher scheme in Ghana, the informal sector was overlooked, the scheme's designers preferring to work through the formal private sector which was not only not being supplied by the domestic polyester textile mills, but had limited capacity to import ITNs into the country. As a consequence, the formal private sector was unable to meet the demand generated by the voucher scheme, and mothers found themselves presenting vouchers that could not be honoured. 

The reasons for these failures are all too familiar to implementers and programme managers: human resource shortages, supply-chain issues, inefficient distribution channels, and barriers to patient access that include obvious obstacles such as high out-ofpocket costs or lack of conveniently located facilities, to more subtle barriers such as socioeconomic or gender discrimination, or cultural values and preferences that may prevent communities from accessing and/or benefiting from the interventions available. In some cases implementation issues arise outside the health system, occurring as a result of contextual factors that policymakers and health system managers may not even have considered. This was certainly true of the voucher scheme to support the distribution of ITNs that was first piloted in Ghana in 2004, for example [6]. 

While it has often been remarked that hindsight is a wonderful thing, it is probably fair to say that properly conducted implementation research, with its all-important focus on context, would have helped implementers foresee and anticipate the problems that arose in Ghana, notably by highlighting the role of the informal private sector in supplying ITNs. It is because of its capacity to illuminate contextual issues that implementation research is such an important tool for implementers at the planning stage and there are numerous examples of implementation research supporting effective policy-making and programme design. The development of Ghana's Dangme West Health Insurance Scheme is a case in point, the scheme's designers making full use of different implementation research tools to anticipate likely challenges (Box 1) [7, 8].

The basic idea of the scheme was to give each pregnant woman a voucher for an ITN at her first visit to an antenatal clinic.  The voucher entitled her to a discount of approximately US$ 4.20 on the purchase of an ITN available through formal sector retail outlets. A similar scheme had been launched in the United Republic of Tanzania with considerable success at the end of the 1990s.  In Ghana however, the scheme ultimately failed, the main reason for the difference in outcomes being the nature of ITN production and distribution in the two countries. 

## Box 1.  Ghana'S Dangme West Health Insurance Scheme

In the late 1990s, the needs of the people of Dangme West District, a poor rural district just outside of Accra, Ghana, were served by just four public sector rural health centers, none of which had doctors. To make matters worse, user fees, introduced in 1985, had resulted in many poor people foregoing even basic health services. There was thus increasing pressure to find a viable alternative method of financing health that did not include out-of-pocket fees imposed at point of service use. Interest in the possibility of developing health insurance was qualified by the fact that most people in the district were either very poor or worked outside the formal sector. Research to test the feasibility of a health insurance scheme in Dangme West was conducted using a consultative and iterative process of formative research involving the District Health Management Team (DHMT) in collaboration with the District Assembly (local government) and community members of the Dangme West District. Extensive discussions, at both the community and individual level were used to arrive at a feasible design, drawing on examples from other LMICs, as well as a study of the history of health insurance and its evolution in high-income countries. Practical details such as the constituents of a suitable benefit package, payable fees, and likely costs were the subject of broad consultation as was the challenging issue of how beneficiaries were to be identified and how access to care was to be ensured, given the limited service availability in the district. After protracted consultation, the decision was taken to introduce a benefit package covering all outpatient care in the health centers in the district. The health centers were also assigned a gatekeeper role, providing referrals for patients needing hospital care to the four hospitals in the surrounding districts. All household members had to register with the scheme and the premium was set per member rather than per household.  Many of the findings and experiences of this research eventually informed the Ghana National Health Insurance scheme development, which was passed into law in 2003 as a national scheme.

Implementation research is, then, of immense value in shining a light on the often bumpy interface between what can be achieved in theory and what happens in practice. Engaging with the real world, and drawing much of its strength from real-world practitioners and the communities they serve, implementation research generates context-specific insights that are simply not available from narrower research perspectives. How exactly the potential of implementation research can be realized in different applications is the subject of the next chapter. 

## How Is Implementation Research Used? 2 Key Points

k
k
Implementation research is vital to understanding context, assessing performance, informing implementation and facilitating health systems strengthening.
k
k
Implementation research is particularly important in supporting the scale-up of interventions and integrating them into health systems at the national level.
k
k
Implementation research can also be used to help organizations develop the capacity to learn.
played a central role in overcoming the challenges of reducing maternal and newborn mortality [10].

## How Is Implementation Research Used? " Implementation Research Takes What We Know And Turns It Into What We Do."

As noted in the previous chapter, implementation research is particularly valuable in shining a light on the sometimes subtle cultural barriers which may escape strictly quantitative information gathering. For example, gender roles and household decision-making authority can determine the extent to which individuals access needed health services, notably where decisions relat-

## Even When Interventions Are Designed In Similar Ways, Implementation Occurs Differently In Different Contexts, And With Many Different Effects.

Having made the broad argument for the importance of implementation research, we now turn to some of the specific areas where it can be of enormous value to a range of stakeholders from ministerial-level decisionmakers, who may use implementation research to support health policy formation, to programme managers seeking to understand context-specific issues, and health providers looking to assess performance, make changes, or introduce innovations.  For all these stakeholders, implementation research offers a window into the practical challenges presented by the provision of health services in the real world. 

## Understanding Context

ing to health seeking are deferred to the male heads of households; women often delay seeking care—even in emergencies and at the detriment of their own health and the health of their children, if the man is not home. Where such barriers do exist, implementation researchers and programme managers can play an important part in changing the approach used to inform communities about the care available; for example, employing messaging about alternate decision-makers for health seeking when male heads of households are not home. This sort of initiative ensures that pregnant women do not delay accessing emergency obstetric services—a significant contributor to high maternal mortality rates in some 
 
areas. 

As the Ghana ITN example cited in the previous chapter demonstrates, implementation research has an important role to play in elucidating the contextual factors that can influence the outcomes of interventions.  This is important because even when interventions are designed in similar ways, there is evidence to suggest that implementation occurs differently in different contexts, and with many different effects [4]. For example, it has been amply demonstrated that the Integrated Management of Childhood Illness (IMCI) strategy, a systematic approach to child health which puts equal emphasis on curative care and disease prevention, is hugely dependent on local health system characteristics such as the basic skills of health workers, the supply of drugs, and the effectiveness of supervision and referral [9]. IMCI is also dependent on the extent to which patients avail themselves of the services on offer, and implementation research can be of particular value in identifying and describing barriers to access, as evidenced by research on the "three delays model" (deciding to seek care, getting transport to care, and receiving quality care once at a health facility), which Thus implementation research can offer crucial insights at a number of levels for implementers who, generally speaking, recognize that implementation goes beyond simply reapplying the same template in country after country [11]. The value of such insights is illustrated by work going on in Andhra Pradesh, India, where implementation research based on micro-planning has been used to focus on understanding the local context, including the perceptions and needs of vulnerable populations (Box 2) [12].  

## Box 2.  Context-Specific Research Becomes Context-Specific Implementation

Efforts to collect data on India's HIV/AIDS epidemic, have greatly sharpened the overall epidemiological picture, but significant challenges remain to generating the evidence needed to allocate resources efficiently and implement an effective AIDS response. In the state of Andhra Pradesh, USAID's Samastha Project, which is designed to support the implementation of India's HIV strategy, has been working to refine such a response. The project's strategies were based on findings from a needs assessment carried out in Andhra Pradesh in 2006 which identified a number of gaps in HIV health services that compromised the quality of care, and left many unaware of available services. To address these shortcomings, Samastha mounted a four-pronged response focusing on: improving access to quality clinical care in facilities; instituting a system of self-assessment and monitoring to prevent infection and improve quality; setting up a computerized management information system (CMIS) that tracks individuals to support programme planning and monitoring; and consolidating community outreach services by using micro-planning and support groups. Micro-planning is a context-specific "bottom-up" tool for developing, implementing, and monitoring activities tailored to the needs of local communities. Communities wishing to improve local HIV services can use micro-planning to identify vulnerable populations, analyze the availability and accessibility of services, and prioritize service delivery according to the available services. Micro-planning is also invaluable for tracking clients' use of services. A key aspect of the micro-planning effort was the use of peer outreach workers (PORWs), people living with aids (PLHIV) who were chosen from the community to conduct outreach, referrals, and follow-up in the target areas. These PORWs played a critical role in that they initiated and maintained contact with the project's target populations, and also provided essential details about the local context, and notably the needs and characteristics of the community in their coverage area. The PORWs used a range of tools to help them identify PLHIV and vulnerable individuals, develop action plans to address each person's needs, and make sure that clients used the necessary services. As a result of the micro-planning effort, registration of PLHIV increased from 610 in 2007 to 5 907 in 2011. Utilization of ART services also increased sharply, the number of those receiving ART or pre-ART services nearly quadrupled, while the number of eligible PLHIV who were not enrolled for ART services diminished from 228 to 18.

Source: Sankar, 2013 [12]

## Assessing Performance

Implementation research can also be used to gauge performance over time and to serve as the basis for projections into the future. Where appropriate, it can also be used to look at the way similar organizations or programmes have performed, noting any disparities in outcomes. It is worth noting here that monitoring and evaluation (M&E) - activities that many implementers and managers already engage in on a routine basis - play a significant part in this kind of implementation research, often helping to define important research questions. This was certainly true of the work done in Afghanistan after the collapse of the Taliban regime in 2002 when a simplified 'scorecard' was used for M&E on health system performance and to establish annual priorities for improvement (Box 3) [13-15]. The approach revealed a number of constraints in delivering basic services that needed to be addressed through reallocation of funding, contract decisions, training, and further research, and formed a basis to assess several new implementation strategies to finance and deliver health services. 

## Box 3.  The Role Of Implementation Research In Assessing And Improving Performance

After decades of war and with the fall of the Taliban regime in 2002, Afghanistan's health sector was in shambles.  The physical infrastructure was destroyed, there were few health workers and managers, no functional health information systems. The meagre health services provided were largely managed by nongovernmental organizations (NGOs).  The new government produced a national health policy and strategy for the delivery of a Basic Package of Health Services (BPHS) to be implemented across the country with the help of donor organizations and local and international NGOs. With inputs by government officials, donor organizations, NGOs, and front-line providers, and facilitation by an external evaluation team, a Balanced Scorecard of BPHS was developed to provide a summary of performance of the BPHS in each of the 34 provinces on an annual basis. The scorecard fits on a single sheet of paper, and covers six main domains (1. Perceptions of patients and community; 2. Staff perspectives; 3. Capacity for health services; 4. Service provision (quality and volume); 5. Financial systems; and 6. Overall vision on equity for women and the poor) and 29 indicators. Data were collected from observations made of over 700 randomly selected health facilities, over 7000 observations of randomly selected patient-provider interactions and exit interviews. There were also over 2000 interviews with randomly selected health workers. The Balanced Scorecard has been used to highlight areas for training and allocation of resources, to provide bonuses on contracts or to cancel contracts, to motivate staff and provide accountability to the public, and as a basis for testing new policies. It also led to the removal of user fees at primary clinics and the extension of performance based payment schemes.  Overall, trends have been improving in all areas on a national basis.

Sources: Peters et al., 2007 [13]; Hansen et al., 2008 [14]; Edward et al., 2011 [15]

## Supporting And Informing Scale-Up

The crucial importance of context in the successful implementation of interventions clearly has implications for the way interventions are scaled up. Interventions that work and can be shown to work in small-scale pilot studies, too often fail to live up to expectations when rolled out in national strategies.  We know what works, but do not always know how to make it work in the real world. The successful implementation of even simple interventions entails an understanding of the different ways in which the intervention is affected when the real world (community, health system, economy) interacts with it. Of course many new interventions are not simple at all, and may involve the introduction of a new diagnostic tool or new information/communication technology. Innovation may also take the form of an organizational change designed to support, for example, a new outreach approach. Then again, the innovation might involve a 

## Interventions That Work In

small-scale pilot studies, too often fail to live up to expectations when rolled out in national strategies.  
change in a process in clinical, or administrative services, or a new project, programme, strategy, or policy. Whatever form the innovation does take, it is clear that simply bolting on a new component in the hope that things will improve is not sufficient. Actions have consequences and it is the job of the implementation researcher to identify and understand them. Nor is scale-up a simple matter of doing the same thing on a bigger scale. Achieving scale-up is generally equated with increasing geographic coverage from a limited study area. In fact this kind of scale-up is more accurately described as horizontal scale-up, or 'spreading', which typically involves replicating an intervention, and stands in contrast to vertical scale-up, which is defined as involving the institutionalization of an innovation through policy, regulatory, budgetary, or other health system changes - in other words, the complex process of embedding an innovation in the institutional structure of a health system [16]. Scale-up can also concern the expansion of the organizational, financial, and technical capabilities of a health system. Needless to say, each of these forms of scale-up - and there are others, present particular challenges for decision-makers and the researchers who support them. To make that support effective implementation researchers need to ask a number of key questions, including:  

k
k What are the projected effects of introducing and 
scaling-up the new component (both the intended and unintended consequences)?
k
k What lessons can be learned from other initiatives to introduce and scale-up (or not scale-up) a similar intervention?
k
k What are the main obstacles to/opportunities for 
scaling-up the intervention?
k
k How will key stakeholders be affected? 
k
k How are different stakeholders disposed towards 
scaling-up (e.g. as facilitators, blockers, etc.)?
k
k How well do different strategies for dealing with 
the different stakeholders work over time?
k
k What are the projected costs of scaling-up?

## Supporting Quality Improvement And Health Systems Strengthening

The benefits of implementation  
research are maximized where  
research is answering the questions that decisions-makers and  
practitioners are asking, or should  
be asking.

With regard to supporting quality improvement and health system strengthening, as with all health policy and systems research - of which implementation research is a form - the core concern is to ask questions that are relevant to the challenges faced. Implementation research can yield many benefits but those benefits are maximized where research is answering the questions that decision-makers and practitioners are asking, or should be asking. Health policy and systems research is often referred to as the brains of the health system, but it is also the eyes and the ears, the only mechanism that decision-makers can rely on for the constant stream of information they need to be able to adapt to changing circumstances optimally [11].  For this reason, some of the best implementation research is often supported, if not actually conducted, by practitioners in the field - the 

Unknown (2446 
samples)
3408
Number of unsatisfactory samples 
41 (1.7%)
14 (0.4%)
Turnaround time from clinic to laboratory (days)
23
9
Turnaround time from laboratory to clinic (days)
27
11
Number of women followed up with colposcopy for positive Pap 
22 (24%)
196 (100%)

Source: Agurto et al., 2006 [17]
doctors, nurses, and managers who are confronted with quality issues on a daily basis. Implementation research is also of great value where it allows for an iterative approach to improvement, as exemplified by the case of El Salvador, where, in 2002 a Pan American Health Organization and Ministry of Healthsponsored study sought to enhance organizational features of an early detection, cytology-based programme. El Salvador's cervical cancer programme had many problems back in 2002: few women were getting screened with PAP smears, many laboratory samples were unsatisfactory, and follow-up colposcopy for those with positive tests was rare. The use of quality improvement cycles (plando-study-act) led to the training of outreach workers to identify women needing screening, to support access to screening, and encourage follow-up visits. After just one year, results improved dramatically (Table 1) [17]. The process was picked up by the Ministry of Health to expand the programme. for decision-making learn much of their craft through ap-
A key driver of quality improvement, and indeed of health prenticeship and the tacit knowledge they gain in addisystem strengthening generally is the learning capacity of tion to the formal methods they apply.  
organizations.  Learning organizations have been defined as those structured in such a way as to facilitate learning as well as the sharing of knowledge among members or 

employees [18, 19].  Like learning people, learning orga-
This chapter has attempted a brief overview of the broad nizations are better placed to anticipate problems, and applications of implementation research, including its develop responses, thus becoming more effective. Key to value in elucidating the challenges and opportunities the learning organisation's ability to learn is the capacthat arise when interventions are moved from the pilot ity to assimilate and put into effect new knowledge that study to the real world. The next chapter will discuss what develops iteratively as knowledge is tested and new lesexactly implementation research is, offering a practical sons learned [20]. Implementation research has an imdefinition that is applicable across the different research portant role to play in helping decision-makers ensure areas that it covers. It will also consider the implementathat the organizations for which they are responsible are tion strategies that support improved delivery of services, able to learn, notably by asking questions such as: what programmes and policies.  Finally, it will describe the approaches and processes can be used to create/build implementation outcome variables that can be used to a learning organization; how can a health organization characterize the various ways in which implementation interact with its various stakeholders (e.g. governing bodoccurs. 

ies, financing bodies, regulators, beneficiaries, internal staff) to improve learning and organizational effectiveness? Needless to say, this is a complex issue and one to which we will return.

## Tacit Knowledge

Having discussed the importance of implementation research and outlined some of its key applications, it is perhaps worth acknowledging that implementation research cannot provide all the knowledge needed for successful implementation. Implementation know-how is also acquired through apprenticeship and experience involving observation and practice.  This kind of know-how, sometimes referred to as 'tacit knowledge', is an important part of learning for individuals and organizations [21]. That said, it is clear that the boundaries between tacit knowledge and the kind of formal knowledge derived from implementation research often overlap. For example, research on the tacit knowledge of health managers may provide important insights about implementation. Those who conduct implementation research or use research 

## What Is Implementation Research? 3 Key Points

k
k
Implementation research can address any aspect of implementation, including the factors affecting implementation, the processes of implementation themselves and the outcomes, or end-products of the implementation under study.
k
k
Implementation research is applicable and relevant in many different domains and, depending on the subject under study, is applicable and relevant to different degrees, with certain research questions being implementation-light and others implementationheavy.
k
k
Implementation research often focuses on the strategies needed to deliver or implement new interventions, which are referred to as 'implementation strategies'.
k
k
In order to understand implementation processes, it is essential to use a framework for conceptualizing and measuring implementation outcomes. Implementation outcome variables are the building blocks of this framework and serve as indicators of how well a given implementation is actually working.
gies to scale, and importantly, how to sustain these strategies over the long term.  

## What Is Implementation Research?

" The basic intent of implementation research is to understand not only what is and isn't working, but how and why implementation is going right or wrong, and testing approaches to improve it."
Implementation research, as it applies specifically to health, is a type of health policy and systems research concerned with the study of clinical and public health policies, programmes, and practices, with the basic intent being to understand not only what is and isn't working, but how and why implementation is going right or wrong, and to test approaches to improve implementation. As noted at the outset, very often it is concerned with the problems arising when an initiative is rolled-out or scaled-up. 

## Implementation Research Is The

For the purposes of this Guide we propose a very simple and very broad definition of implementation research that can be used across research communities, and that has meaning for health sector practitioners, and policymakers, as well as for the interested public: implementation research is the scientific inquiry into questions concerning implementation.

## Scientific Inquiry Into Questions Concerning Implementation.

Implementation research is a growing field of study with roots in many disciplines and research traditions. It addresses a wide range of implementation problems in diverse contexts. In many ways this is the great strength of implementation research - its capacity to tap into different sources, bring to bear multiple perspectives and offer multisectoral insights - but it also presents some obvious taxonomic challenges of the kind that academics sometimes shy away from. It is perhaps not surprising then that there is some confusion regarding nomenclature as well as significant debate regarding the  scope of implementation research [22, 23]. 

Under this definition implementation research can address or explore any aspect of implementation, including the factors affecting implementation (such as poverty, geographical remoteness, or traditional beliefs), the processes of implementation themselves (such as distribution of fully-subsidised ITNs through maternal health clinics, or the use of mass vaccination versus surveillance-containment), and the outcomes, or end-products of the implementation under study. As described above, implementation research may focus on issues such as: identifying common implementation problems; understanding the factors that hinder or facilitate access to health interventions; developing and testing solutions to tackle implementation barriers, either within a specific context or across a range of environments; and determining the best way to introduce innovations into a health system, or to promote their large scale use and sustainability.

Broadly speaking, the term implementation research describes the scientific study of the processes used in the implementation of initiatives as well as the contextual factors that affect these processes. One major purpose of implementation research is to support and promote the successful application of interventions that have been demonstrated to be effective - a drug that is known to kill malaria parasites, a diagnostic test that identifies who has tuberculosis, or a strategy to prevent the transmission of HIV from mother to child.  It can be used to figure out how to deploy human resources so that geographically remote communities can access care when needed, and identifying how to eliminate financial barriers that prevent vulnerable populations from getting needed services. It is also about reducing costs and making organizations more efficient and accountable.  Finally, implementation research is about learning how to bring promising strate-

## Implementation Strategies

While implementation research may not be concerned with discovering new health products, or testing the safety or efficacy of clinical interventions, it often deals with the strategies needed to deliver or implement those products or interventions. These strategies are sometimes referred to as 'implementation strategies', a term used to implementation outcomes is essential. Such a framework also allows for much-needed studies of the comparative effectiveness of implementation strategies. In order to conceptualize and evaluate the success or failure of implementation, it is useful to employ some form of consistent taxonomy that allows us, for example, to talk about different aspects of implementation response - the acceptability of an intervention, say, or the extent to which an intervention has been taken up or adopted. 

distinguish them from clinical and public health interventions [24]. For example, while outreach clinics and supervision checklists are implementation strategies commonly used to improve the coverage and quality of immunization programmes, the provision of the vaccine itself is considered the health intervention. Implementation strategies may also be designed to improve the sociocultural aspects of implementation, for example by improving the acceptability or adoption of the intervention, or may affect things like the quality and cost of the services pro-
These characteristics can be seen as the outcomes of implementation, and are referred to as implementation outcome variables. Implementation outcome variables serve as indicators of how well a given implementation is actually working. The implementation outcome variables - acceptability, adoption, appropriateness, feasibility, fidelity, implementation cost, coverage and sustainability - can also be seen as intermediate factors that contribute to other important outcomes such as satisfaction with health care or health status [25, 26].  
vided. Implementation research may focus on the implementation strategy itself, or incorporate consideration of the implementation strategy into a broader study of the health intervention. As Table 2 shows, one way of talking about implementation strategies is to group them in terms of the actor or stakeholder using them. Typical implementation strategies include: (1) enhancing the capabilities of government (public policy, oversight and financing agencies); (2) improving the performance of implementing and provider organizations; (3) strengthening the capabilities and performance of individual providers and front-line workers; 
Not all implementation outcome variables are of equal importance in the delivery of an intervention, or for research on  implementation (see Table 3) [25].  With regard to a novel intervention, for example, the main focus might be on issues of acceptability, adoption, appropriateness, and feasibility.  For existing interventions, the degree to which the intervention is implemented as it was originally designed, or is faithful to the original (measured by the fidelity variable) is often very important, as are costs and coverage.  Although sustainability issues should be considered from the earliest phases of an intervention, they are frequently neglected in research on health interventions [27].

(4) empowering communities and households; and (5) supporting multiple stakeholders engaged in improving health [4].

Each variable represents an important aspect of implementation that can be studied through implementation research. 

## Implementation Outcomes

In order to advance our understanding of implementation processes, and enhance the efficiency of implementation research, a framework for conceptualizing and measuring 

Main Actor and Areas of  
Intervention
Implementation Strategy Examples
Government 
- Policy-making,  
oversight and regulation
- Public financing
- Policy reviews - Governance strengthening and corruption reduction strategies - Contracting with performance measures - Decentralize public service provision - Public financing incentives and rules (ways to raise revenues, pool funding, and 
payment mechanisms)
- Public education, behaviour change communication
Implementing and Provider Organizations
- Organizational  
improvement and accountability
- Quality improvement/quality assurance/performance management strategies: team 
problem-solving; develop and apply guidelines and standard operating procedures; regular supervision
- Provide financial incentives for teams and individuals based on performance - Reorganize and/or integrate services - Human resource management systems - Facility management and logistics systems strengthening - Strengthen financial management - Marketing health services and products
Individual Providers and Front-line Workers
- Individual practices
- Continuing education and training - Peer learning and support - Job aids
Communities and Households
- Empowerment,  
participation, education
- Individual practices
- Community information and education: training community health workers; training of community members such as youth, mothers (in groups, home, mass media); social marketing and demand creation
- Strengthen inclusion and participation: community-managed services; community 
partnerships and co-management; community-owned services
- Strengthen local accountability: joint monitoring; provider accountability schemes; 
community-based information systems
- Local organizational capacity building: community mobilization; community boards 
and structures to oversee and manage
- Financial empowerment: community financing; in-kind subsidies and vouchers; 
community participatory budgeting; incorporation with income generating and micro-financing schemes
- Peer support for health services and healthy behaviours
Multiple Actors
- Assess needs and constraints: constraints reduction plans - Obtain broad-based support of stakeholders: engage powerful interest groups; 
coordinate with community organizations 
- Flexible management processes and modification through stakeholder feedback

Adapted from: Peters et al., 2009 [4]

Implementation 
Outcome
Working Definition*
Related terms**
Acceptability
The perception among stakeholders (e.g. consumers, providers, managers, policy-makers) that an intervention is agreeable 
Factors related to acceptability: (e.g. comfort, relative advantage, credibility)
Adoption
The intention, initial decision, or action to try to employ a new intervention
Uptake, Utilization, Intention to try
Appropriateness
The perceived fit or relevance of the intervention in a particular setting or for a particular target audience (e.g. provider or consumer) or issue
Feasibility
The extent to which an intervention can be carried out in a particular setting or organization
Practicality, Actual fit, Utility, Suitability for everyday use
Fidelity
The degree to which an intervention was implemented as it was designed in an original protocol, plan, or policy
Adherence, Delivery as intended, Treatment integrity, Quality of programme delivery, Intensity or dosage of delivery
Implementation cost
The incremental cost of the delivery strategy (e.g. how the services are delivered in a particular setting). The total cost of implementation would also include the cost of the intervention itself.
Coverage
The degree to which the population that is eligible to benefit from an intervention actually receives it. 
Reach, Access, Service Spread or Effective Coverage (focusing on those that need an intervention and its delivery at sufficient quality, thus combining coverage and fidelity), Penetration (focusing on the degree to which an intervention is integrated in a service setting)
Sustainability
The extent to which an intervention is maintained or institutionalized in a given setting.
Maintenance, Continuation, Durability, Institutionalization, Routinization, Integration, Incorporation

*The original definitions referred to individual "innovations or evidence-based practices".  This table uses the term "intervention" so that the definitions are more broadly applicable to programmes and policies. **Other terms are more commonly found in implementation literature on large-scale programmes and policies (Peters et al 2009; Rogers 2003; Carroll et al. 2007, Victoria et al 2005) ***Provides the numerator for related measures of efficiency and measures of cost-utility, cost-benefit, or cost-effectiveness.  Many cost analysis examine the total cost of implementation of an intervention, including the cost of the intervention itself, as well as the costs of implementing a particular delivery strategy in a given setting.

Relevance, Perceived fit, Compatibility, Trialability, Suitability, Usefulness, Practicability Marginal cost***

## The Continuum Of Implementation Research

As discussed earlier, one of the criticisms sometimes levelled at implementation research is that it lacks definition as a field of study. This is partly because it is applicable and relevant in so many different domains and partly because, depending on the subject under study, it is applica-
Examples: Basic science; 
Qualitative studies unrelated to implementation issues (e.g. perceptions of illness)
as secondary issue ble and relevant to different degrees. In order to grasp this idea, it is helpful to think about implementation research in terms of a continuum, with certain research activities being Implementation-light and others Implementationheavy. In Figure 3 we represent this idea as a flow chart, with questions becoming more implementation intensive or heavy as we move downstream.

tory research without necessarily seeking to understand what may be influencing that progress (or lack of it). Monitoring is very often the starting point for implementation research when it extends to using research methods to investigate issues beyond the routing monitoring. Research occurring in controlled, non-practice settings, with highly selected samples that do not represent the intended population for the intervention, and where implementation outcome variables and factors affecting implementation are not relevant or are controlled in attempts to eliminate their effects, can be said to be implementation-light. 

Thus, on the left side of the flow chart, we find research that does not involve implementation issues at all (such as basic research into a Zivoudine as a means of preventing mother-to-child-transmission of HIV). While at the right of the chart, we find research that is primarily concerned with question about implementation in real-world settings (for example, how to ensure that pregnant women who test positive for HIV in low-income countries get the prophylactic Zivoudine treatment needed to reduce the risk of transmission of HIV to her baby). As described below, as it focuses more on implementation strategies and studies implementation variables, the research becomes more implementationheavy.

## Research Is Implementation-Heavy When It Addresses Questions Focused On Implementation, Occurs In Real-World Settings, And At Least Considers Factors That Affect Implementation

Finally, it is important to note that this flow chart represents a significant simplification of a much more complex reality and is used here merely to illustrate the basic idea. In reality, health systems rarely deal with new innovations one at a time, for example, and often have to accommodate multiple innovations 
(and other disruptions/changes) simultaneously. Thus the process of absorption and adaptation is much messier and more complex than has been presented here. 

It is important to note that research that is not concerned with implementation can become highly relevant for implementation research. For example research into freeze-dried smallpox vaccine meant that the vaccine no longer had to be kept cold which had major implications for the implementation of the smallpox eradication campaign. Thus basic research may yield new products that can later be tested as an intervention that improve health. Similarly, discoveries in the field (oral rehydration salts can be mixed on site and dispensed by non-specialized personnel, for example) can have implications further up the research chain (focus on developing non-caking oral rehydration salts). 

This chapter has presented some of the basic ideas that constitute implementation research, including a practical definition that can be used across research traditions. It has also attempted to offer a typology of implementation barriers and problems, a description of implementation strategies, and a description of implementation outcome variables that can be used in research to describe various aspects of the ways in which implementation occurs. In the next chapter we will be looking at the role of collaborations and why they are necessary to understanding implementation in real-world settings, reflecting pertinent contextual factors.  
Research is implementation-heavy when it addresses questions focused on implementation, occurs in realworld settings, and at least considers factors that affect implementation, if not actually testing implementation strategies and implementation outcome variables. It is the focus of implementation research on context and the interaction between the real world and the intervention being studied that sets it apart from, say, routine monitoring, which measures progress in a specific area over time, 

## Who Should Be Involved In Implementation Research? 4 Key Points

k
k
Good implementation research is collaborative research, and often most useful where implementers have played a part in the identification, design and conduct phases of the research undertaken.
k
k
The fostering of collaborative ties between key stakeholders involved in policy generation, programme management, and research is essential.
k
k
Implementation research should be integrated into policy and programmatic decision-making so that scientific inquiry becomes a part of the implementation problem-solving process.
k
k
Implementation research can play an important role in "speaking truth to power", by identifying neglected 
issues or by demonstrating performance and increasing accountability of health organizations.  
k
k
Understanding of context and systems, and the flexibility to identify appropriate methodological approaches, can be as important as or even more important than adherence to a fixed-research design.

## Who Should Be Involved In Implementation Research? " Successful Implementation Research Begins And Ends With Successful Collaboration."

Implementation research is most likely to be useful to its audience where that audience is not just a passive recipient of results.

taken, or for whom it may be relevant. Specifically, implementation researchers need to bear in mind the needs and/or limitations of their intended audience - the people or organizations that are going to be using the results. As noted in the last chapter, implementation research covers a broad range of subjects, and the audience for this research is similarly broad, including, for example, health managers and teams working on quality improvement (QI) strategies, policy-makers looking to effect a change in policy or to introduce an entirely new policy, and practitioners who need support in using interventions that are based on the best evidence available. Each of these potential audiences has different requirements and different perspectives, and for implementation research to be of optimal utility it needs to take those differences into consideration. More often than not, the audience for this research is not another researcher, or academic, but a non-specialist in need of clear, evidence-based analysis uncluttered by jargon, which can form the basis of future decisions. 

As highlighted in the previous chapter, one of the defining aspects of implementation research is that it seeks to understand the reality of implementation in real-world contexts. Unlike other forms of research, it does not seek to filter out the extraneous or accidental; indeed, in many ways it is precisely such factors that are of interest to the implementation researcher. This approach implies a readiness on the part of the implementation researcher to embrace the unpredictable and otherwise problematic on occasion, in a way that other researchers might not. It also implies using study subjects in all their complexity and in their natural environments. This means working with populations that are actually going to be affected by an intervention, for example, rather than selecting populations on the basis of narrow eligibility criteria (for example, choosing volunteers who have only one health condition when it is people with co-morbidities who are the target for the intervention). The considerations of context that are relevant to implementation research include the social, cultural, economic, political, legal, and physical environment, including the prevailing demographic and epidemiological conditions. They can also include the institutional setting, comprising the particular institutional structures in place, and the various stakeholders working within them. The way health systems are structured, and the roles played by government, the private sector and non-governmental organizations (NGOs) are also of interest.

Though there are exceptions, which we will discuss below, implementation research is most likely to be useful where implementers have played a part in the identification, design and conduct of the research undertaken, and are not just a passive recipients of results. As already noted, people on the front line of health care, whether running specific programmes, or working in health systems have a great deal to contribute to the information-gathering endeavor (Box 4). Routine monitoring, for example, is often the starting point for many implementation research questions, and relies on the routine collection and analysis of current administrative and health data. There are a number of ways of encouraging feedback from the field, one of which being so-called Participatory Action Research (PAR), which, as the name suggests, involves participation of the research subjects, and has been described as research "carried out with and by local people Another key consideration for implementation researchers is the audience for whom the research is being under-

## Box 4.  The Importance Of Implementers In Implementation Research

Given the nature of the questions implementation research asks, it is clear that implementers have a crucial part to play in its conduct, and should take an active role rather than being participants or passive partners in the overall research process. Indeed in certain cases - quality improvement (QI) studies, for example - implementers are ideally placed to lead this type of research, taking responsibility for almost all aspects of the research cycle. Implementers can make a number of important contributions, starting with the planning stage where they are best placed to identify implementation barriers and propose implementation research questions, as well as to identify solutions that can be tested. They can also play a role in designing studies, notably by facilitating understanding of context and the contextual factors that impact implementation, and contributing to frame research questions to reflect the existing situation.  In terms of data collection, implementers are clearly in a privileged position with regard to accessing data sources, and interviewing respondents, while in terms of analysis (particularly for qualitative data), implementers can play an important role in making sense of and interpreting data. Finally implementers have a key part to play in the dissemination of results, notably by incorporating lessons learned into programme practice.

rather than on them" [28]. PAR was not designed specifically to answer questions about implementation, but implementation is an obvious subject for participatory research. PAR also creates opportunities for communities that may not otherwise have a voice to "speak truth to power", for example by offering an account of exploitive or abusive service provision. We will return to PAR, and other research methods in more detail in Chapter 5. The importance of both researchers and implementers coming together in the conduct of implementation research is of course of considerable importance in situations where the core issues relate to quality improvement and the scale-up of a programme, both of which impact many stakeholders. As noted in the previous chapter, scale-up takes many forms and may involve not only the expansion of services, but also the development of the organizational, financial, and political capabilities of implementing organizations, and learning processes that engage implementing bodies, beneficiaries, funders and officials [29]. Such endeavors are necessarily multifaceted, and thus require multifaceted research studies. Such studies are best carried out where there is strong collaboration. Above all, regardless of the subject under study, it is important for both researchers and implementers to recognize the value of coming together in what is in fact a symbiotic relationship - a relationship in which implementers generate feedback from the front lines, while researchers provide expertise in research methods needed for trustworthy studies. Only by coming together in this way, can stakeholders ensure that the knowledge generated is valid, and aligned with need. 

## The Importance Of Partnerships In Implementation Research

Given the importance of collaboration in implementation research, the skills needed to build and maintain partnerships are a critical consideration. This includes the fostering of collaborative ties between key stakeholders involved in policy generation, programme management, and research. One interesting example of the way collaboration can work is the above-cited case from Ghana, where researchers and implementers came together in the late 1990s to consider the optimal approach to developing a national health insurance scheme (Box 1) [7]. Beginning with what is known as formative research, which involves studying the community for which a given intervention is being planned, the researchers and implement-

## Chapter 4 - Who Should Be Involved In Implementation Research?

ers worked in close collaboration, making joint decisions about the design of the research to ensure that it not only had the capacity to provide evidence that was sound, but would do so while respecting the time constraints faced Implementers generate feedback from the front lines, while  
researchers provide expertise in cation with implementation researchers at an early stage; implementation research needs to be embedded in the overall design, planning and decision-making endeavor. This embedding can be achieved in three ways: first, by integrating the funding for research and programmatic activities; second, by systematically applying research and scientific inquiry as part of problem-solving; and, third, by using joint decision-making in the research and implementation process. We consider each of these imperatives in turn.

research methods needed for 

## K K Integrating Funding Into Research And Programme Activities

trustworthy studies.

by the District Health Authorities [7]. The experience in Ghana shows not only the value of collaboration, but the importance of an evolving, iterative approach to implementation research.

## Embedding Implementation Research

A major driver of the way implementation research is conducted is the way it is funded. Research funding generally flows through separate channels from programme funding. This is true for both international donors and governments that typically have one budget for programmes and one for research.  As a result of this separation, research funding cycles are not always aligned with programme needs. Similarly, a good deal of research is awarded on a competitive basis, often on the initiative of individual investigators, whereas programme funding typically is not competitive, and may require teams or programme managers to identify the questions of relevance. This too creates mismatches between programme needs and research objectives, and creates a disincentive for researchers to link their work to the actual barriers and challenges that are encountered during the implementation of programmes. Moreover, the process and timeline for issuing calls for research proposals and selecting studies often exceeds the time available to those responsible for programmes to make important decisions about implementation. 

One way to improve collaboration and encourage partnerships in implementation research is to integrate it into policy and programme decision-making. Because implementation research often flows from well-established programme activities and is of direct benefit to programmes, it makes sense to include it as an integral part of programme processes from the beginning rather than a tangential activity that is then used to provide context/ comparison that may inform programme development at a later date. Had implementation research been an integral part of Ghana's 2004 ITN pilot from the very beginning, the differences between Ghana and the United Republic of Tanzania with respect to ITN production and distribution would have been identified early enough for the pilot's designers to make the necessary adjustments, thus saving time and vital resources [6]. In order to effectively integrate implementation research into the decision-making processes related to implementation it is not enough to simply open lines of communi-
Where funding for IR is integrated within programme budgets, there are greater opportunities to align the research with programme needs.  This is the approach that has been adopted by Bloomberg Philanthropies in two major global initiatives on tobacco prevention and road safety.  In both instances, the foundation provided funding for a consortium of partners, ranging from civil society, 

## Chapter 4 - Who Should Be Involved In Implementation Research?

academic researchers, and technical agencies to collaborate on the implementation of large scale public health interventions.  Each consortium partner plays a role and contributes to different aspects of the implementation. 

 
In the case of the tobacco initiative, the focus was on legislating tobacco control and research on the costs of tobacco control and policy implications was paired with advocacy efforts [30].  Similarly, for the Road Safety in 10 Countries (RS-10) initiative, researchers are working with implementing partners to support surveillance efforts, monitor and evaluate progress, as well to provide critical insights related to perceptions of stakeholders and communities on the interventions being implemented [31]. This collaborative approach not only gives important recognition to the individual roles played by different partners in the implementation process, it enables alignment of research questions with programme needs while also facilitating the engagement of multiple stakeholders in the research process—thus ensuring that the evidence generated is used to inform implementation. 

## K K The Systematic Application Of Research And Scientific Inquiry In Programme Activities

In addition to integrating research and programme funding, it is crucial that the systematic application of scientific research be institutionalized within programme decisionmaking so that implementation research becomes a core part of the problem-solving process.  This can be achieved in a number of ways. To begin with, establishing protocols and/or processes for decision-making related to implementation and scale-up that explicitly refer to research is a good way of ensuring that problems and questions that need investigation are addressed in a systematic manner. The WHO/ExpandNET framework for scale-up is a useful example of how research and scientific inquiry can be integrated into processes. The framework includes questions that need to be answered - sometimes through implementation research - as part of the nine steps that implementers need to consider when scaling-up a programme [16]. Integrating implementation research questions into a framework for scale-up effectively embeds research into decision-making related to the scale-up of interventions.  The WHO/ExpandNet framework also suggests a multidisciplinary approach using different actors to guide the scale-up process so that there is partnership and collaboration at all stages, including during the conduct of implementation research [16]. Another way in which research and scientific inquiry can be integrated into programme decision-making is through mandatory M&E, the value of which as a basis for implementation research has already been noted [32].   Requiring a link between the monitoring and implementation of programmes ensures that problems and challenges, some of which may be addressed through implementation research, are identified on a regular basis.  Required evaluation of programmes also serves to enhance accountability and longer term learning. It also helps to systematize the way in which implementation challenges are understood, and to ensure the relevance of implementation research for the programme.  An interesting example of the value of this type of embedding is the legislation requiring the systematic evaluation of all publicly-funded social programmes in Mexico, which has helped to de-politicize policy-making while at the same time increasing the relevance of research evidence to implementers and other policy-makers [32, 33]. 

## K K Shared Responsibility For Decision-Making

The final aspect of embedded research is perhaps the most important, and also the most difficult to achieve. Programme implementation and scale-up both demand a certain level of flexibility and often develop under time pressure. Thus, any research study undertaken in support of these activities needs to be responsive to these realities. This means that decisions about study designs, methods, and outcomes need to be informed not just by the perspectives of researchers, but must also reflect the views of implementers and other stakeholders.  For example, researchers alone may desire evidence that is probabilistic and which can be substantiated through statistical analyses, whereas time pressures may dictate a study design which simply generates evidence regarding, ent actors can be given due consideration.

## The Challenges Presented By Partnerships

for example, whether a strategy has resulted in a certain level of coverage in a population or whether the quality of services meets certain standards (sometimes referred to as an adequacy statement) without necessarily taking into consideration whether outside influences caused the change [4, 34].  
Even in this brief presentation of embedded implementation research, it should be clear to the reader that while collaborative approaches offer a number of opportunities, they also entail challenges. This is partly a reflection of the complexity of health systems and the way that the multiplicity of actors working within them interact. While it is commonplace to talk about critical actors in terms of their basic roles, breaking them down as decision-makers, programme managers, front-line health workers, and the patients they serve, for example, it is important to recognize that each role can be played out at different levels and that the boundaries between participants are sometimes blurred. 

Similarly the questions that are the subject of implementation research need, in many cases, to be jointly developed by researchers and decision-makers to reflect their different perspectives. As already noted, implementers and researchers often come at problems from slightly different angles, implementers focusing on the specific barriers and challenges to implementation, and researchers looking for ways to formulate questions that are suitable for study and can be answered through research. This difference in agenda was apparent in the roll-out of the study on the use of visibility enhancement materials for motorcycle safety in Malaysia (Box 5). Researchers 

## Collaborative Approaches Present A Number Of Opportunities For Implementation Research, But They Also Present Challenges.

Thus, decision-making that sets the agenda and leads to the crafting of policies and programmes may initially occur at the national level of a given health system, but when central decisions are handed down for implementation, problem-solving or process-enhancing initiatives may lead to modification of the originally planned implementation arrangements and of the central decisions. As such, while front-line workers are often predominantly responsible for implementing already agreed decisions and policies, because of these local effects they can also end up being the setters of the decision-making agenda and the formulators of related policies and programmes. From this perspective, implementers, at all levels of health system operations, have a crucial part to play—sometimes even the leading role (particularly in quality improvement studies), in the implementation research endeavor.  A good example of implementers engaging with the research process is provided by the district health management teams (DHMT) in Ghana, Uganda and the United Republic of Tanzania, who, supported by the PERFORM research group, used the Action Research (plan, act, observe and reflect) approach to develop and then test context-specific management strengthening processes focused on improving workforce performance (Box 6) [36].

and decision-makers at different levels worked together to come up with research questions and a study design that generated evidence (on the feasibility of scale-up at a district level), which responded to the decision-makers' needs, while employing quasi-experimental methods which ensured rigor and allowed for the independence and objectivity that the researchers were comfortable with [35]. The Malaysia example is evidence that different agendas can be satisfied, given a readiness to compromise, but it should not be seen as typical. The fact is sharing responsibility for decision-making is not always easy, nor will every decision be agreed upon by all. However, it does show that decisions can be informed by multiple perspectives and that the expertise and insights of differ-
It is apparent then that the flow of information and the formation of ideas is often fed by feedback loops, and is not a strictly linear process. Moreover, because of the importance of process in implementation research, an understanding of context and systems, and the flexibility and creativity to identify appropriate methodological approaches, can be as important as or even more important than adherence to a fixed research design informed by a particular disciplinary perspective. The clinical randomized controlled trial, for example, depends on a fixed and reproducible intervention such as the taking of a pill, whereas implementation may involve interventions that 

## Box 5.  Policy-Makers And Researchers Come Together On Road Traffic Injuries In Malaysia

Despite the clear link between knowledge and action, interactions between those who generate information and those who are expected to use and apply that information are the exception rather than the rule, especially in LMICs. The trial of preventive measures designed to reduce motorcycle crashes and deaths in Malaysia is a good example of how this kind of collaboration can be made to work. In July 2005, a Department of Road Safety (DRS) was established within the Ministry of Transport (MoT) and meetings between DRS and other stakeholders were set up to assess policy-makers' appetite to engage in new research regarding the possible benefits to be derived from using reflective materials to improve visibility of motorcyclists. Crucially, researchers and policy-makers came together very early on to agree on their shared objectives, and how to attain them.  Policy-makers initially had a sceptical view of research, believing it took too much time to conduct and that they could resolve most of the challenges related to the district wide implementation of an intervention using visibility enhancement materials (VEMs).  At the same time, the research team had initially envisaged a more complex study design of a longer duration in order to obtain a high degree of generalizability.  Through the stakeholder consultations, discussions, and compromises, the group agreed upon a study field trial using a quasiexperimental design that would enable the scale-up of the VEM intervention in an entire district, as it was not politically feasible to randomize motorcyclists to receive the intervention within a district.  This design would also enable the use of routinely collected police and hospital data for the analysis which reduced the time required for the baseline assessment; it also strengthened the link between research and programme activities. A field trial was particularly appealing to policy-makers because it presented an opportunity to achieve immediate, concrete results.  It also offered the opportunity to understand how such an intervention could be scaled-up in the Malaysian context and how it could be sustained once the trial was finished. Of course the potential downside of conducting what would be very public research was that the VEM might turn out to be useless, but researchers were able to convince the DRS that even negative findings would result in savings given that the cost of the research would be far less than the cost of a national VEM campaign that was ineffective. In the end the trial was given a public launch to raise awareness of the issue and the project was eventually branded the "Be Seen and Be Safe" campaign for motorcycle safety. 

change frequently and are not strictly reproducible. A fixed research design like a randomized controlled trial, while appropriate for studies of efficacy and effectiveness, may not be suitable for answering questions related to implementation. Another challenge inherent in collaboration is the sometimes competing priorities of participants.  For example, researchers may be under pressure to publish in highimpact journals that often favour specific disciplinary approaches, while implementers may be under pressure to resolve the problem in the shortest time possible. Such differences, and there are many others, are not insurmountable, but it is clear that without compromise and in some case sacrifices on both sides, collaboration will be problematic at best. One way for implementation researchers to improve their chances of successful collaboration with partners in the field is by getting out into the field. The best understand-

## Box 6.  District Health Teams Use Implementation Research To Build Human Resources Capacity In Africa

The dearth of competent, motivated health workers is a major impediment to improving health and saving lives in Africa. To address this deficit, a two-pronged strategy is required, ensuring the training of new health personnel and improving the performance of the existing workforce. A number of complex factors affect workforce performance, including staffing levels and distribution, the organization of work and required resources, working conditions and remuneration. Understanding the nature of these factors and developing appropriate responses to the challenges faced offers the opportunity to not only improve the performance of the existing workforce but also to reduce staff losses. PERFORM, a research group currently working in Ghana, Uganda and the United Republic of Tanzania, focuses on the building of local capacity and ownership for processes supporting human resource management and related health systems activities. Specifically, PERFORM supports district health management teams (DHMT) who, through a series of workshops and review meetings use so-called Action Research (plan, act, observe and reflect) to develop and then test context-specific management strengthening processes focused on improving workforce performance. The DHMTs identify areas of health workforce performance to be improved, implement integrated human resource and health systems strategies feasible within the existing context to improve health workforce performance, and monitor the implementation of the strategies, evaluate the processes and impact on health workforce performance and the wider health system. 

The DHMTs then **plan** their own 'bundles' of human resource and health systems strategies, act to implement these strategies and **observe** the impact of the strategies they have developed on health workforce performance. They then **reflect** on how well their plans have been achieved and if necessary, revise the plan or address new challenges thus beginning to embed the process within their districts. The main outputs of this ongoing work will include developed and tested district-level interventions to improve both health workforce performance and the management of decentralised health systems more generally in the three African countries. These interventions can be potentially replicated in other districts in Ghana, Uganda and the United Republic of Tanzania and in other countries with decentralised health systems. The outcomes of the research will contribute to the body of knowledge of how strengthening management in sub-Saharan Africa can improve workforce performance and the wider health system.  In addition, the action research approach will contribute to the development of skills and abilities of participating managers to resolve other management problems in the future, with the possibility of scaling-up this approach if it proves successful.

ing of context and others' perspectives comes from some experience of that context. Ideally, implementation 
 
researchers should spend some time living and working in the context (and the organizations) they intend to study, thus gaining insights that can inform their research design, and working methods. This kind of immersion also helps in the development of listening skills, understanding others' perspectives, engaging in dialogue, negotiation 

and collaborative problem-solving. Unfortunately, while such skills are critical aspects of implementation research, they are not generally recognized as research skills.  

## Implementation Research In The Face Of Opposition

This chapter has sought to identify the core issues related to implementation research, focusing in particular on the importance of conducting implementation research in real-world settings, paying attention to context, and being mindful of the needs of the audience for whom the research is intended.  It has also emphasized the importance of collaboration and the need for partnership between implementers, researchers and other stakeholders. In the next chapter we will focus on the principles behind common methods used in implementation research, and look at the ways in which research questions provide a basis for the research methods used. We will also be looking at ways in which methods can be designed to address the problem of complexity that is so often a defining characteristic of implementation.

While the kind of immersion just described is helpful, there are situations where embedding research in public policy processes is simply not possible. Policy-makers, managers, and funding agencies do not always want to know how their programmes are being implemented, unless of course they can be shown to be doing well. They may have invested considerable political and financial capital in a policy, and be afraid of not producing the desired results or of poorly managing resources. Funders are frequently resistant to research that might highlight sustainability issues or the negative unintended consequences of their programmes, such as the human resource distribution problems arising as a result of hiring people for single purpose projects, an issue often encountered with HIV projects, among others [27]. Similarly, the concerns of minority groups may not be of interest to those groups in power, particularly if there are social and political sensitivities. Areas where this kind of problem arises include issues related to men who have sex with men (MSM), the treatment of aboriginal groups, the provision of abortion services, and pervasive dangerous practices such as female genital mutilation, etc. Implementation researchers who collaborate with disadvantaged groups or civil society organizations may find themselves unable to collaborate with those who oppose them.  This can be a particular problem when research is conducted in an area suffering from ongoing civil conflict. In some cases participatory action research may even be considered revolutionary to the existing power structures. In these circumstances, an important aspect of implementation researchers' work is to find ways to get their research into agenda-setting processes to influence policy. This may also require approaches that rely more on advocacy strategies that can make use of well-designed research.

## What Approaches And Methods Are Appropriate For Implementation Research? 5 Key Points

k
k
Implementation research, like all research, is governed by two broad principles: that its findings should be warranted, and that its methods should be transparent.
k
k
Because it draws on a wide variety of qualitative, quantitative, and mixed-method research approaches, it makes little sense to talk in terms of a narrow set of 'implementation research-methods'
k
k
In implementation research, the "question is king", and it is the question that determines the method used, rather than the method that determines the kind of questions asked. 
k
k
The questions asked are often complex, reflecting the 
complexity of the real world. A wide array of contextual factors influence implementation, producing unpredictable effects that require continuous adaptation by implementers.
is used, these principles need to be borne in mind. 

## What Approaches And Methods Are Appropriate For Implementation Research? " The Question Is King."

Because implementation research draws on a wide variety of qualitative, quantitative, and mixed-method research approaches, it makes little sense to talk in terms of a narrow set of 'implementation research-methods'. There are however a number of research approaches that are particularly useful to the implementation researcher because they are inherently practical and generate actionable intelligence; are good at capturing the subtleties of context, and in particular context as it changes over time; and offer the iterative flexibility needed to respond to change and evolve. A short description of a selection of these approaches is presented below. 

## Pragmatic Trials Implementation Research Draws On A Wide Variety Of Qualitative, Quantitative, And Mixed-Method Research Approaches

In discussing different approaches to implementation research it is helpful to keep in mind its basic goals, which, as we have already discussed, are to understand how, and why clinical and public health policies, programmes, and practices work, or fail to work in real-world settings, and to learn how to make them work better. More specifically implementation research can be used to: assess change in real-world contexts, drawing on past experience, where appropriate; understand complex phenomena; generate and/or test new ideas; and predict, or at least help anticipate what may happen in the future as a result of a particular innovation or change. It also plays an important role in informing stakeholders, thereby improving understanding, transparency and accountability [37]. Last, but certainly not least, the goal of implementation research is to make a difference, to improve the effectiveness, quality, efficiency and equity of policies, programmes and services.

so it makes little sense to talk in terms of a narrow set of 'implementation research-methods'.

Tests or trials of health interventions are generally described as either explanatory or pragmatic. The terms were originally coined to distinguish between trials designed to help choose between options for care, and trials designed to test underlying causal hypotheses. Thus, explanatory trials generally seek to understand and explain the benefit produced by an intervention under controlled conditions, often using carefully selected subjects in a research clinic, whereas pragmatic trials focus on the effects of the intervention in routine practice. In contrast to explanatory trials, pragmatic trials seek to maximize the variability in the way the intervention is implemented, (e.g., in terms of settings, providers or types of patients), in order to maximize the generalizability of results to other settings [39].  In this way, pragmatic trials can provide strong evidence of the effectiveness of an implementation strategy in 'real-world' conditions.  
Before discussing some of the possible  research approaches that can be used to achieve these goals, it is worth noting that, while implementation research may in some ways be different to other forms of research, like all research, it is governed by two broad principles. The first of these is that findings should be warranted, that is to say backed by sufficient evidence. The second is that its methods should be transparent, that is to say sufficiently explicit for others to be able to judge whether the processes are adequate and justify the conclusions reached, and can be repeated [38].  Whichever approach Pragmatic trials usually include an extensive formative period involving implementers and policy-makers to design the intervention strategy, which may sometimes generate a false sense of confidence that the design is robust and suitable for the setting in which it is implemented.  Therefore,  unless additional research methods are added to the trial, the kind of changes that happen in the 'real world' - changes in implementation strategies, changes in implementation outcome variables, or other non-random changes in contextual factors - may not be 

## Box 7.  A Pragmatic Trial In South Africa

One of the biggest obstacles to improving access to antiretroviral therapy (ART) in LMICs is the lack of trained medical staff needed to administer it. In South Africa shortages of doctors have tended to restrict access to the treatment and researchers at the Knowledge Translation Unit of the University of Cape Town Lung Institute in Cape Town, South Africa used pragmatic trials to demonstrate that health workers other than doctors were capable of meeting the demand for care. Specifically, the trial focused on the Streamlining Tasks and Roles to Expand Treatment and Care for HIV (STRETCH) programme, which provides educational outreach training of nurses to initiate and re-prescribe ART, and to decentralise care. Thirty-one primary care clinics were randomly assigned to either the nurse-run program or the usual, 'standard' care. The study followed over 8000 patients in the nurserun programme and 7000 patients in the standard care group for one and a half years, and found that mortality rates, viral suppression rates, and other measures of quality of care did not differ, or were actually higher in the nurse-run programme.

Source: Fairall et al., 2012 [40]

## Effectiveness-Implementation Hybrid Trials

Effectiveness-implementation hybrid trials combine elements of effectiveness and implementation research in order to assess both the effectiveness of a health intervention, and the implementation strategy used to deliver it. Whereas pragmatic trials do not try to control or ensure the delivery of services to meet a realistic standard in normal practice settings, effectiveness-implementation hybrid trials also intervene and/or observe the implementation process as it actually occurs, for example by assessing implementation outcome variables [24]. One recent paper proposes three basic types of effectiveness-implementation hybrid research designs, based captured. Ideally, the design of the intervention in a pragmatic trial, and the outcomes the research is designed to generate, should be developed in collaboration with the participants, funders, and practitioners who are making decisions about the intervention, and who are directly affected by the outcome. The value of pragmatic trials in LMIC settings is well documented, one good example being a recent study undertaken by researchers in South Africa (Box 7) [40]. 

largely on the priority given to the effectiveness or implementation components in the research aims [24].  

k
k Type 1 designs test the effects of a health intervention on relevant outcomes while observing and gathering information on implementation. In this kind of research patient functioning or symptoms in response to a health intervention are measured, while at the same time the feasibility and acceptability of the implementation approach taken is evaluated through qualitative, process-oriented, or mixed-methods.
k
k Type 2 designs involve the dual testing of health 
interventions and implementation strategies. 
 

## Chapter 5 - What Approaches And Methods Are Appropriate For Implementation Research?

k
k Type 3 designs test an implementation strategy 
while observing and gathering information on the health intervention's impact on relevant outcomes. Type 3 designs primarily test the implementation strategy using measures of adoption of and fidelity to health interventions.
Effectiveness-implementation hybrid trials offer a number of benefits.  Instead of taking a step-by-step approach to problem-solving, starting with a randomized clinical trial to determine if an intervention implemented under controlled conditions works, and then moving on to designs such as cluster randomized controlled trials in order to determine the best way to introduce the intervention in real-world settings, effectiveness-implementation hybrid approaches allow researchers to simultaneously evaluate the impact of interventions introduced in real-world settings and the implementation strategy used to deliver 

## Box 8.  Effectiveness-Implementation Research Applied In Bangladesh Newborn Care Study.

In order to capture the full range of qualitative and quantitative data needed to evaluate the cluster randomized controlled trial that was designed to test a homecare and community health worker intervention and a community-care intervention in comparison to the usual neonatal care provided, researchers used a range of research methods, including:

- Quantitative household survey research, which provided estimates of existing neonatal mortality and levels of 
skilled birth attendance, which motivated the need to intervene and also provided baseline levels; 
- Formative qualitative research, which was used to explore home practices that put newborns at risk of death, 
and the barriers for safe delivery and postnatal care. It was then used as the basis for the development of the home-based newborn package of care ("participant enrichment"), and to design quantitative research instruments ("instrument validity");
- Observations of newborn care were conducted to show that community health workers could diagnose newborn illness;
- Household surveys and in-depth interviews were used to show that the intervention was being implemented 
as planned, while surveys, observations, and in-depth interviews were also used to demonstrate whether the new newborn practices were actually being implemented ("implementation fidelity");
- End-line household surveys were used to assess both neonatal mortality and service coverage levels, while 
qualitative research was used to explain in detail how and why delivery and postnatal practices changed, largely because of the engagement of the local community in the programme, and supportive supervision of the community health workers ("meaning enhancement"). 
Effectiveness-implementation hybrid trials offer a number of benefits,  
including speeding up the translation of knowledge into action.

them. Such designs not only speed up what may otherwise be a very time-consuming process, they also allow researchers to identify important intervention-implementation interactions. These can then be used to inform decisions about optimal implementation approaches. The testing of the newborn care packages in Sylhet, Bangladesh offers a good example of the different purposes for which effectiveness-implementation hybrid trials are appropriate (Box 8) [41-44]. 

## Quality Improvement Studies

As highlighted by the example of cervical screening in El Salvador (Table 1), the study of quality improvement (QI) in health care presents three main challenges: first, the evaluation of quality is inherently context-dependent; second, quality is something of a moving target - with improvement interventions being repeatedly modified in response to outcome feedback; third, conducting research into quality also generally involves taking into account complex, multi-component interventions. All of this has implications for research design [17]. In order to reflect the iterative, 'moving-target' nature of QI, studies typically involve a set of structured, cyclical processes, governed by a paradigm referred to as the Adapted from: Brassard et al., 1994 [46]
plan-do-study-act (PDSA) cycle or a variant thereof [45]*. The PDSA cycle allows for the application of scientific methods on a continuous basis to formulate a hypothesis or plan to improve quality, implement the plan, analyze and interpret the results, then generate a plan for what to do next.  A detailed description of the range of quality improvement tools that can be used in PDSA studies is beyond the scope of this Guide, though some examples are shown in Figure 4, listed according to the stages of the PDSA cycle [46]. Another instructive example is the PDSA work done in El Salvador in relation to cervical cancer screening, which is discussed in Chapter 2. design and report on research involving QI interventions, Typically, PDSA QI research assesses whether the interreaders should consult the Standards for Quality Improvevention being studied - usually a change in process - 
ment Reporting Excellence (SQUIRE) guidelines [45].  
produces a significant improvement in an outcome. The results are then used to effect changes in the intervention on a cyclical, iterative basis. PDSA interventions thus 

## Participatory Action Research

typically involve repeated testing over time. PDSA studies All research on human subjects involves human participaare often referred to as quasi-experimental because the tion, but the particularity of participatory action research experimenter lacks complete control of the study, nota-
(PAR) is that it assigns power and control over the rebly with respect to his or her ability to randomly allocate search process to the subjects themselves. Thus, PAR rethe intervention to specific subjects [47]. There is a wide fers to a range of research methods that typically involve range of quasi-experimental research designs, some comiterative processes of reflection and action "carried out mentators identifying as many as sixteen different types, with and by local people rather than on them" [28]. 

and noting their different strengths and weaknesses with respect to issues of internal validity (whether the inter-
The kind of "bottom-up" approaches that involve locally vention in the specific study actually made a difference), defined priorities and perspectives are set out in Table 4 
and external validity (asking to which populations, set-
[28]. Although most of the PAR methods involve qualitings, treatment and outcome variables can the observed tative techniques, increasingly quantitative and mixedeffect be generalized) [47]. 

methods techniques are being used, as, for example, in participatory rural appraisal or participatory statistics [50] 
PDSA research designs are often used for QI in health 
[51]. Guidelines for conducting and reporting participasystems, offering as they do the assessment of responses tory action research are available, though the emphasis measured repeatedly and regularly over time, either in a tends to be on local action by participants rather than single case or with comparison groups [48]. Research deways in which external researchers can work with them signs typically include: time series studies, with baseline 
[52, 53]. 

and post-intervention assessments of levels and trends in results; multiple time series studies, where the intervention and baseline are repeated at different times: time series studies where the interventions are lagged across different groups at different times: and factorial design studies where the intervention is randomized to groups to compare time series. The data for these quasi-experimental designs may come from routine health management information, or from special surveys to specifically measure the outcomes of interest. For standardized guidance on using good quality health information systems and health facility surveys, readers can go to the Lindelow and Wagstaff report on data and measurement issues arising in the assessment of health facility performance [49]. For general advice on how to 

|                                           | Participatory Action Research          | Conventional Research                     |
|-------------------------------------------|----------------------------------------|-------------------------------------------|
| What is the research for?                 | Action                                 | Understanding with possible later action  |
| Who is the research for?                  | Local people                           | Institutional, personal, and professional |
| interests                                 |                                        |                                           |
| Whose knowledge counts the most?          | Local people's                         | Scientist's                               |
| Who chooses the topic?                    | Local priorities                       | Funding agency, institutional agendas,    |
| professional interests                    |                                        |                                           |
| Methodology is chosen for what reasons?   | Empowerment and learning               | Disciplinary convention, "objectivity",   |
| "truth"                                   |                                        |                                           |
| Who takes part in the stages of research? |                                        |                                           |
| Problem identification                    | Local people                           | Researcher                                |
| Data collection                           | Local people                           | Researcher, data collector                |
| Interpretation                            | Local concepts and frameworks          | Disciplinary theories and frameworks      |
| Analysis                                  | Local people                           | Researcher                                |
| Presentation of findings                  | Locally accessible and useful          | By researcher to academics and funding    |
| agency                                    |                                        |                                           |
| Action on findings                        | Integral to process                    | Usually separate or may not happen        |
| Who takes action?                         | Local people, with or without external |                                           |
| support                                   |                                        |                                           |
| External agencies                         |                                        |                                           |
| Who owns the results?                     | Shared                                 | The researcher or funder                  |
| Emphasis of process or outcomes?          | Process                                | Outcomes                                  |

Source: Cornwall and Jewkes 1995 [28]  
A number of PAR-based initiatives have been undertaken in LMICs in recent decades, one notable example being the work of the Indian nongovernmental organization, 

## Box 9.  Participatory Action To Improve Neonatal Health Care

The success and sustainability of community-based programmes for improving maternal and neonatal health require the active involvement of women, families and community health-care workers, yet the strategies used to engage these groups are often externally driven and top-down in character. Since 2005, the Indian nongovernmental organization known as Ekjut has sought to reverse this trend by helping women's groups to improve maternal and neonatal health in tribal areas of the Indian states of Jharkhand and Odisha. Local female facilitators guide women's groups through a cycle of activities involving participatory learning and action, during which women identify, prioritize and analyse local maternal and neonatal health problems and subsequently devise and implement strategies to address them. The Ekjut intervention was initially evaluated in a cluster randomized controlled trial carried out between 2005 and 2008 in 36 largely tribal clusters of three contiguous districts of Jharkhand and Odisha. A recent study reported significant falls in neonatal mortality in those districts as a result of the interventions and concluded that community mobilization through women's groups can produce a sustainable and reproducible improvement in neonatal survival in rural areas of India.

Ekjut, which helps women's groups to improve maternal and neonatal health in tribal areas of the Indian states of Jharkhand and Odisha (Box 9) [54].

## Realist Review

the key questions of the review. This might include questions such as: Which parts of the programme are working and which are not? For whom? Under what circumstances? Why?  Finally, the results of the review are shared in ways that provide a stronger link to those who commissioned the review or who can use the review, preferably as part of a policy dialogue [56].  
The aim of realist review is to enable decision-makers to reach a deeper understanding of the intervention and how its potential can be maximised in different settings. The approach can be extremely helpful in dealing with policy and programme interventions, where complexity and variation in implementation are significant factors [55]. 

## Realist Review Provides

explanatory analysis focused on  
what works for whom, in what  
circumstances, in what respects,  
and how.

The 2011 WHO report, Realist Review and Synthesis of Retention Studies for Health Workers in Rural and Remote Areas, by Dieleman and colleagues, is an example of how this type of review can be applied to understand contextual factors and the main mechanisms that underpin many retention strategies [57]. This report adopts an analytical framework that is centered around a 'context-mechanism-outcome' which describes how an intervention—in this case retention strategies for workers, interacts with a specific context—in this case rural and remote areas in LMICs, and how it results in certain outcomes.  Through this approach, the underlying theory supporting the intervention are clarified and tested using scenarios where the intervention has been implemented [57]. 

## Mixed-Methods Research

Used to address complex social interventions or programmes, realist review is anchored in particularity, providing explanatory analysis focused on what works for whom, in what circumstances, in what respects, and how [55]. That said, it is clear that synthesis requires some form of generalisation and in realist review that generalisation comes from the identification of underlying assumptions. Thus, the first step in realist review is to discover and make explicit the underlying assumptions of the intervention in question, revealing how it is meant to work in theory. This is an iterative process that involves sharpening the focus of questions around the nature of the intervention by assessing the integrity of the underlying theory, comparing rival theories, and assessing the same theory in different settings. Realist review then seeks empirical evidence in the literature that supports, contradicts or modifies the underlying programme assumptions, combining theoretical understanding and empirical evidence, while focusing on the relationship between the context in which the intervention is applied, the mechanisms by which it works and the outcomes which are produced. 

Mixed-methods research, as the name suggests, uses both qualitative and quantitative methods of data collection and analysis in the same study. While not designed specifically for implementation research, mixed-methods research is particularly suitable for these research activities because it provides a practical way to understand multiple perspectives, different types of causal pathways, and multiple types of outcomes - all of which are common in implementation settings. Mixed-methods research approaches can be extremely useful and are applicable for a range of purposes - as many as 65 according to The search and review of publications eventually leads to a formal audit trail of the literature reviewed, and a final search is then conducted once the review is nearly complete. Individual articles are assessed for their relevance and methodological rigour, and synthesized according to 

## Chapter 5 - What Approaches And Methods Are Appropriate For Implementation Research?

one study [58]. These can be boiled down to four main rationales [59]*. 

k
k Describe the justification for using a mixed 
 
methods approach to address the research question.
k
k Describe the design in terms of the purpose, 
 
priority and sequence of methods.
k
k Participant enrichment: to gain the most information from a sample of participants (e.g. by administering a standard survey questionnaire and then asking for in-depth explanations). 
k
k Describe each method in terms of sampling, data 
collection and analysis.
k
k Instrument validity: to make sure that the instruments used are appropriate and useful (e.g. using focus groups to identify items for a questionnaire or testing its validity).
k
k Describe where the integration has occurred, 
how it has occurred, and who has participated in it.
k
k Describe any limitation derived from associating 
one method with another method.
k
k Implementation fidelity (treatment integrity): to 
assess whether the intervention or programme is being administered as intended.
k
k Describe any insights gained from mixing or 
 
integrating methods.
k
k Meaning enhancement: to maximize the interpretation of the findings, such as by using qualitative measures to explain the statistical analysis or vice versa.

## The Importance Of The Research Question

While the different research approaches described above may be said to constitute a kind of basic toolbox for implementation researchers, it is important to remember that in implementation research it is the question posed that determines the tool to be used and not the tool that determines the shape of question. Put simply, in implementation research the question is king. This does not mean that it is an entirely adhoc endeavor and that there is no room for overarching concepts; indeed certain theoretical constructs are of value in helping to think about implementation processes, notable among them are theories of change that try to explain what is needed to achieve a long-term goal such as improved health. 

A number of different schemes exist that describe different types of mixed-methods research, based on the emphases used in different approaches, the sampling schemes used for the different parts of the study, the timing and sequencing of the qualitative and quantitative methods, and the level of mixing between the qualitative and quantitative methods [60, 61].  Fully mixed-designs use qualitative and quantitative methods in each stage of the research, including sampling and data collection, analysis, and drawing of inferences. Interested readers can consult Tashakkori and Teddlie's handbook which outlines 35 different designs, or Onwuegbuzie and Collins' article describing 24 different sampling schemes [55, 61]. Broad guidance on the design, conduct, and reporting of mixed-methods designs are available by several authors [59, 60, 62, 63].  A simple scheme for good reporting of a mixed-methods study (GRAMMS) is listed below [64].

A theory of change should describe a logical sequence or pathway with sets of outcomes along the way to the goal, and should reflect the assumptions about the changes made. Often the steps along the way and the assumptions about why they occur are the subject of research for 

## Chapter 5 - What Approaches And Methods Are Appropriate For Implementation Research?

verification, testing, or further explanation. It is also worth noting the valuable insights that have come from some of the work done on implementation theory, notable among 

## Box 10.  Implementation Theory

A number of theories have been developed to promote the effective implementation of health interventions. Many try to explain individual or group behaviours around implementation issues.  For example the RE-AIM (reach, efficacy, adoption, implementation, and maintenance) framework is commonly used in health promotion interventions, and provides a practical approach to evaluating the effects of health interventions through changes in individuals, organizations and communities. The Diffusion of Innovations Theory seeks to explain how innovations spread, highlighting the importance of the perceived attributes of the innovation (relative advantage, compatibility with current approaches, ability to observe results, ability to test the innovation, and its complexity), innovativeness of the adopter, the social system, individual processes for adoption, and the system for diffusion. The Consolidated Framework for Implementation Research (CFIR) was developed as a way to consolidate the various theories and terms used to support further development of theory and testing on how to translate health interventions that have been shown to be effective. The CFIR comprises five elements: 1. Intervention characteristics; 2. Outer setting; 3. Inner setting; 4. Characteristics of the individuals involved; and 5. The process of implementation. Constructs like the strength and quality of the evidence are related to the intervention domain, whereas issues like patient needs and resources are part of the outer setting. Constructs related to inner setting of the organization include its culture and leadership engagement. Individual attitudes, beliefs and capabilities also play a prominent role, whereas other influential factors are related to the process of implementation itself (e.g., plan, evaluate, and reflect).

which the Consolidated Framework for Implementation Research (CFIR), a useful template for organizing key concepts in implementation research (Box 10) [65-67]. tion is being introduced, as was the case with the ITN 

## Matching Methods With Research Questions

voucher programme in the United Republic of Tanzania Given the importance of the research question, it is worth or the insurance scheme in Ghana. 

taking a moment to consider the kinds of questions that As noted in the previous chapters, implementation reare likely to arise, and the research methods that may be search is of greatest use where it is conducted in realappropriate to answering them. One way of going about world settings and assesses the context and other factors this is to break down research questions into a limited influencing implementation, with some research specifinumber of categories based on the core objective of the cally testing implementation strategies and/or measuring research to be undertaken, as we have done in Table 5 
implementation outcome variables [68].

[4, 34, 58]. Thus, in the second example, one of the key questions 

asked with a view to describing implementation phe-
This chapter has sought to identify the defining characternomena, is: 'What describes the main factors influencing istics of research approaches selected on the basis of their implementation in a given context?' This is followed by usefulness to implementation researchers, and to suggest the kinds of research methods likely to generate the inways in which those methods can be applied to specific formation required, a list that includes both quantitative implementation research questions. In the next chapter 
(cross-sectional or descriptive surveys, network analysis) 
the focus will be on aligning implementation research and qualitative methods. What is noticeable about the with need, both in terms of the response imposed by the methods proposed is the importance given to qualitative material under study and the needs of the intended audiresearch, and in particular to research designed to absorb ence. 

a richness of detail through, for example, case-studies, key informant interviews, and focus groups. In the predictions example, the research questions intended to generate predictions draw on prior knowledge or theories to forecast future events. The research may rely on a variety of quantitative research methods, including agent-based modeling, which relies on computational models to simulate the actions and interactions of autonomous agents such as organizations or groups with a view to assessing their effects on the system as a whole, and data extrapolation and sensitivity analysis. These methods may be supplemented by qualitative research such as scenario building exercises and so-called 'delphi' techniques which rely on a panel of experts making predictions, usually based on a questionnaire, and conducted over several rounds between which a facilitator provides an anonymous summary of forecasts. Such forecasts are obviously of particular value where a significant innova-

| Objective                                                        | Description           |
|------------------------------------------------------------------|-----------------------|
| Explore                                                          | Explore an idea or    |
| phenomenon to                                                    |                       |
| make hypotheses or                                               |                       |
| generalizations from                                             |                       |
| specific examples                                                |                       |
| What are the possible factors                                    |                       |
| and agents responsible for                                       |                       |
| good implementation of a                                         |                       |
| health intervention? For en-                                     |                       |
| hancing or expanding a health                                    |                       |
| intervention?                                                    |                       |
| Describe                                                         | Identify and describe |
| the phenomenon                                                   |                       |
| and its correlates or                                            |                       |
| possible causes                                                  |                       |
| What describes the context in                                    |                       |
| which implementation occurs?                                     |                       |
| What describes the main fac-                                     |                       |
| tors influencing implementa-                                     |                       |
| tion in a given context?                                         |                       |
| Influence                                                        | Test whether an       |
| intervention pro-                                                |                       |
| duces an expected                                                |                       |
| outcome.                                                         |                       |
| Is coverage of a health                                          |                       |
| intervention changing among                                      |                       |
| beneficiaries of the interven-                                   |                       |
| tion?                                                            |                       |
| With Adequacy                                                    | With sufficient       |
| confidence that                                                  |                       |
| the intervention                                                 |                       |
| and outcomes are                                                 |                       |
| occurring                                                        |                       |
| Is a health outcome plausibly                                    |                       |
| due to the implemented                                           |                       |
| intervention rather than other                                   |                       |
| causes?                                                          |                       |
| With Plausibility                                                | With greater          |
| confidence that the                                              |                       |
| outcome is due to                                                |                       |
| the intervention                                                 |                       |
| Is a health outcome due                                          |                       |
| to implementation of the                                         |                       |
| intervention?                                                    |                       |
| With Probability                                                 | With a high (calcu-   |
| lated) probability                                               |                       |
| that the outcome is                                              |                       |
| due to the interven-                                             |                       |
| tion                                                             |                       |
| How and why does implemen-                                       |                       |
| tation of the intervention lead                                  |                       |
| to effects on health behaviour,                                  |                       |
| services or status in all its                                    |                       |
| variations?                                                      |                       |
| Explain                                                          | Develop or expand     |
| a theory to explain                                              |                       |
| the relationship                                                 |                       |
| between concepts                                                 |                       |
| and the reasons                                                  |                       |
| for the occurrence                                               |                       |
| of events, and how                                               |                       |
| they occurred?                                                   |                       |
| Predict                                                          | Use prior knowledge   |
| or theories to fore-                                             |                       |
| cast future events                                               |                       |
| What is the likely course of                                     |                       |
| future implementation?                                           |                       |
| Quantitative: Agent-based modeling; Simulation and forecast-     |                       |
| ing modeling; Data extrapolation and sensitivity analysis (trend |                       |
| analysis, econometric modeling)                                  |                       |
| Qualitative: Scenario building exercises; Delphi techniques from |                       |
| opinion leaders                                                  |                       |
Qualitative methods:  Grounded theory, ethnography, phenomenology, case-studies and narrative approaches; key informant interviews, focus groups, historical reviews Quantitative: Network analysis, Cross-sectional surveys Mixed methods: Combining qualitative and quantitative methods.
Quantitative: Cross-sectional (descriptive) surveys, network analysis Qualitative methods:  Grounded theory, ethnography, phenomenology, case-studies and narrative approaches; key informant 
interviews, focus groups, historical reviews Mixed methods: Both qualitative and quantitative inquiry with convergence of data and analyses
Before-after or time-series in intervention recipients only; Participatory action research
Concurrent, non-randomized cluster trials: health intervention implemented in some areas and not in others; before-after or cross-sectional study in programme recipients and non-recipients; Typical quality improvement studies
Partially controlled trials: Pragmatic and cluster randomized trials; Health intervention implemented in some areas and not in others; Effectiveness-Implementation Hybrids
Mixed methods: Both qualitative and quantitative inquiry with convergence of data and analyses Quantitative: Repeated measures of context, actors, depth and breadth of implementation across subunits; network identification; can use designs for confirmatory inferences; Effectivenessimplementation hybrids; Qualitative methods: Case-studies, phenomenological and ethnographic approaches with key informant interviews, focus groups, historical reviews Participatory action research

## How Should Implementation Research Be Conducted? 6 Key Points

k
k
Implementation research should be aligned with need, both in the sense that it meets the requirements of the intended audience and is also responsive to the particularities of the subject under study.
k
k
Research designs need to be responsive and capable of capturing changing elements at multiple points in time.
k
k
There are no fixed rules for justifying the selection of a particular research method for implementation research, other than the fact that the methods used should reflect the questions asked.

## How Should Implementation Research Be Conducted? " Be Responsive To The Demands Of Your Subject And Your Audience."

Having looked at ways in which the question in implementation research can be seen to be 'king', in this chapter we look at the importance of doing research that is aligned with need, both in the sense that it meets the requirements of the intended audience and is also responsive to the particularities of the subject under study. Although there is a role for implementation research in generating theory, most often the evidence generated by implementation research is produced to be used in the real world, rather than consumed by other researchers. This means that the implementation researcher needs to be aware of the use to which his or her work will be put. A key considerations in this regard is the level of certainty required regarding results or projections. A policy-maker, for example, may be looking for strong indications that a given intervention will work, but may not have the time required for protracted studies that could generate a higher level of certainty. The quasi-experimental field trial of the kind described in the previous chapter about reflective materials and motorcycle injuries may be adequate and more importantly, more appropriate for answering the research questions being asked. Such differences have important implications for the basic design of research, in terms of sample size, the need for concurrent comparison groups and randomization of participants within these groups, as well as the cost and time requirements for the research to be undertaken [4, 34]. Different research approaches also have implications for budget and scheduling. Simple exploratory or descriptive studies seeking to establish what is happening may not require long-term observation, for example. However, studies assessing the influence of an intervention usually need observations from at least two points in time, while more frequent measurement may be needed if changing implementation factors are to be assessed. 

If the main research question calls for an explanation of how or why implementation is developing in a particular way, the demands on research design become that much greater, requiring multiple observations to capture changes from different types of respondents. Thus, in a situation where lack of compliance with guidelines on, say, antimicrobial prescription is the main concern, a quantitative study of antimicrobial prescription levels would need to be accompanied, at the very least, by research capturing levels of awareness regarding rational drug use, and doctors and patients attitudes and interactions. But even there, the implementation researcher would probably miss essential contextual factors without digging a little deeper, or throwing the net a little wider. For example, where public sector generalists are paid per consultation, they may find it quicker and thus more profitable to prescribe the antimicrobial that the patient is demanding rather than spend time informing the patient about the dangers of antimicrobial resistance [69]. Simply looking at questions of awareness, would miss the essential part played by health system financing.

## The Importance Of Flexible, Responsive Research

Health systems are complex. A complexity that is aggravated by the complexity of its main actors including decision-makers, implementers and people who are the ultimate beneficiaries of health services.  Both health systems and its actors are constantly changing and adapting to new actions, often reacting in unpredictable ways [70].  In addition, a wide array of contextual factors typically influence implementation and these factors often change over time, producing unpredictable effects that require continuous adaptation by implementers. This has profound implications for the research methods used in studying implementation, particularly in regard to the need for flexibility and creativity in response to changes in the subject under study. In addition, since implementation of policies, programs, and practices is rarely a static or linear process, research designs need to be responsive and capable of capturing changing elements at multiple 

## Chapter 6 - How Should Implementation Research Be Conducted?

health systems, the focus of research turns to implementation. Needless to say, this is solely a theoretical model and in reality the process is neither linear nor completed in one iteration, since health systems are constantly adapting and incorporating multiple innovations with different entry points.  
points in time. Research designs that depend on having a single and unchangeable intervention, such as a typical randomized controlled clinical trial, are poorly adapted to study phenomena that change, especially when that change is unpredictable.  The challenge posed by complexity is all the greater where the research design itself is complex, calling for multiple methods and different sources of information.  
While it is clear that a degree of subjective judgment is necessary to assess any particular study, as a general rule, good implementation research should be able to address each of the following questions: 
Contextual factors can influence implementation and these factors 

k
k Will the research answer a relevant and important implementation problem?  
can also change over time, producing 

k
k Is the new knowledge potentially worth the cost 
of the research? 
unpredictable effects that require  
continuous adaptation  
by implementers.

k
k Are there clear research objectives and questions 
related to implementation, and does the proposed research design match them (see Table 5) 

## Justifying The Selection Of Research Methods

k
k Does the research fit with a theory of change or 
causal chain in a coherent way?  If not, what is the potential for generating new theories or questions?
k
k Will the research produce results that can be 
acted on in a timely way by the intended audiences?
k
k Does the research design reflect an understanding about whether the intervention is stable and simply replicable, or whether the intervention is expected to change? 
k
k Does the research adequately capture changes 
that occur over time and place in both the intervention, the context, and the effects?
As in other areas of research, there are no fixed rules for justifying the selection of a particular research method for implementation research, other than the fact that the methods used should reflect the questions asked. An understanding of the research objective and specific research question is a good starting point, and should be informed by the proposed theory of change. Figure 3 offers a useful template for thinking about the range of research questions and methods along the continuum of implementation research, with implementation becoming more important with the testing of proof of implementation and the consideration of implementation outcome variables. 

k
k In complex environments, can the research identify the main components of the health system and their relationships, as well as the unintended consequences that are likely to occur from an intervention?
Generally speaking however, proof of implementation usually concerns a clinical or public health intervention that has already been shown to work. As the issues move further downstream to questions of scaling-up coverage and integration and sustainability of interventions in 

k
k Does the research appropriately consider context 
and other factors that influence implementation?

## Assessing The Quality Of Implementation Research

Standards for assessing the quality of conventional quantitative and qualitative methods of research are largely 

k
k Does the research appropriately consider changes over time, and the level of complexity of the system?
the same when the research involves implementation. 

 
Guidelines already exist to help with the design and re-

k
k Does the research clearly identify the target audience for the research and how it can be used?
porting of health research, many of which are catalogued by the EQUATOR "Library for Health Research Reporting" 
It is our hope that, using this guidance, practitioners, 
(at: http://www.equator-network.org/). These include policy-makers, researchers and users of research will such guidelines as the CONSORT statement on randombegin to have the confidence to assess whether impleized controlled trials, STROBE guidelines for observational mentation research is conducted and reported in a transstudies, PRISMA for systematic reviews and meta-analyparent way and warrants the conclusions drawn through sis, and COREQ for qualitative research. 

valid and reliable methods.

Because none of these guidelines focuses on the particu-

lar issues related to implementation research, we propose the following set of questions that can be used as a sum-
This chapter has sought to identify key considerations for mary guide to assessing the issues particularly related to implementation researchers seeking to optimise the imimplementation research, which can be used in addition pact of the work they do. In the next chapter we will look to the conventional guidelines.  It should be noted that at the challenges faced by the implementation research these guidelines are designed to help with the reporting field itself, and discuss ways in which both implementers and assessment of research reports, but they can also be and researchers can do more to support this crucial area adapted for purposes of assessing research proposals or of study and make better use of the potential it offers.

the design of implementation research.  The key questions include:

k
k Does the research clearly address a question concerning implementation?
k
k Is there a clear description of what is being implemented (e.g. details of the practice, programme, or policy)?
k
k Does the research involve an implementation 
strategy? If so, is it described and examined appropriately?
k
k Is the research conducted in a real-world setting? 
If so, are these conditions described in sufficient detail?
k
k Does the research appropriately consider implementation outcome variables?

## How Can The Potential Of Implementation Research Be Realized? 7 Key Points

k
k
Without implementation research we are at best committing valuable resources to implementation in the hope that things will work out.
k
k
Despite the importance of implementation research, it continues to be a neglected field of study for two reasons: a lack of understanding regarding what it is and what it has to offer; and a lack of funding for implementation research activities.
k
k
Implementation research should be seen as a core function of programme implementation, which means embedding it into the programme cycle.
k
k
More funding is needed for implementation research, and this funding should be aligned with funding for 
programmes.
k
k
More opportunities for researchers and implementers in LMICs who want to undertake implementation research are needed.  

## How Can The Potential Of Implementation Research Be Realized? " Implementation Research Is Not On The Whole An Expensive Pursuit So Investments In It Go A Long

on health innovations, yet only a fraction of this is spent on how best to use those innovation [72]. The problem affects everyone, but where implementation challenges are greatest, notably in LMICs, the adverse consequences of neglecting implementation research are most keenly felt. Even when products programmes and services are specifically designed for LMICs, too often they never get to the end user because of failures in implementation.

way."

## Without Implementation Research We Are

Given the need for implementation research, why isn't more of it done? And why does the implementation research that is undertaken not always have the impact we might expect? There are several reasons for the lack of implementation research, among the most important being the lack of understanding of what it is and the role it can play in maximising the benefit of the interventions at our disposal. Another obvious constraint is the lack of funding to support this type of research. 

## At Best Committing Valuable Resources In The Hope That Things Will Work Out

Implementation research is essential to ensuring that the benefits accruing from the effective implementation of policies, programmes, and services are realized. Without it, and the knowledge it generates, we are at best committing valuable resources in the naive hope that things will work out rather than applying the kind of evidenceand experience-informed decision-making that implementation research makes possible. 

This approach comes with a cost. Every time a programme fails because insufficient attention was paid to 'real-world' or context-specific factors not foreseen or anticipated by programme designers, there is a cost both in terms of wasted resources and of unnecessary human suffering. Sometimes this cost can be enormous. 

As it is clear from the various approaches described in this Guide, implementation research is not on the whole an expensive pursuit, and certainly not expensive compared to investment in cutting edge biomedical engineering. So investment in implementation research goes a long way. Moreover it is possible to argue that we now have many of the interventions and technologies needed to reduce morbidity and mortality, and should focus more on making better use of them. That is not to say that investment in basic science should stop, and it is worth noting that the above-cited Cooksey report stressed the importance for the UK of maintaining the level of investment in so-called 'blue sky' projects. Nor is it simply a question of raising new funds for implementation research, but of making better use of the funds available, channelling them towards research that is aligned with need, by, for example, increasing opportunities for implementers and programme personnel to access this funding. 

But despite the importance of implementation research, it continues to be a neglected field of study, a fact borne out by numerous studies, including, for example the 2006 report produced by Sir David Cooksey on United Kingdom (UK) health-care research, which noted "perverse incentives that value basic science more highly than applied research" and reported that two thirds of public and charitable research funding went into basic as applied research, including research into implementation [71]. Needless to say private sector research funding was even more heavily skewed to developing new products for new markets, rather than to making the most of products already on the shelf. And what is true of the UK is true of most developed countries. Billions of dollars are spent 

## Chapter 7 - How Can The Potential Of Implementation Research Be Realized?

To address these different challenges, it is necessary to support and promote implementation research on a number of different fronts, which we here break down into an agenda for action.

Action #1: Implementation research should be seen as a core part of programme implementation. This means embedding research into the programme cycle in an iterative progression that allows for continuous learning and, where necessary, adaptation. One way of supporting this change would be to encourage ownership or primary responsibility by implementers or programme personnel with support and guidance from specialist academics. Bottom line: implementers need to take a more active role in implementation research. 

Action #2: To ensure that implementation research becomes more accessible, researchers should be encouraged to engage in programme activities. This will include entering into dialogue with implementers and pursuing initiatives that embed implementation research in the real world such as living in the field in order to better appreciate the complexities of implementation. 

Action #3: Implementers need to make programmes more accessible to researchers and invite researchers to participate in programmes. There are times when it is the implementers who resist collaboration. Knowledge generation activities should be made a part of programme implementation and those with the expertise to carry out research should be engaged to support this process. 

Action #4: Make more funding available for implementation research and align this funding with funding for programmes. To bring this realignment about it is essential that funding for implementation research be made available within programme budgets or explicitly tied to programme activities through structured collaborations and partnerships.  
Action #5: More training opportunities for implementation research need to be made available to programme personnel or implementers. Implementation research should also be built into training programmes such as Masters in Public Health so that it is recognized as a core function of public health practice, a cycle in which knowledge is generated through research and used to inform programme implementation.

Action #6: Provide more guidance and opportunities for mentorship to researchers and implementers in LMICs who want to undertake implementation research.  No one is better placed to provide the kind of context-specific research needed to support implementation research in LMICs, and thus researchers in those countries represent a huge untapped resource. 

Action #7: Incentives for researchers should be linked to engaging in making changes in policies and programmes, in addition to incentives related to academic publication and teaching. So some reorientation of researchers is needed, with opportunities for researchers to experience and understand field work and programmes. This is possibly something that donors can support.

This Guide is an attempt to begin to redress the deficit in understanding of implementation research and it is the writers' hope that readers will pursue the subject in greater depth in the source material cited in these pages. In particular, it is hoped that programme personnel and implementers will take a greater interest in the subject, recognizing that implementation research is in fact a programme issue, that is to say an integral part of programme planning and execution, rather than something that happens once the programme is up and running, something that is practiced in the research 'bubble', and conducted largely for the benefit of other researchers. For their part implementation researchers can do much more to engage with implementers and programme personnel in research process. For things to change, it is essential that programme personnel and implementers who understand the importance of context and researchers who understand methods and science of inquiry come together. Only in this way can we hope to advance our understanding of implementation issues. 

## References

1.	 Fenner, F., Smallpox and its eradication. 1988, Geneva, 
Switzerland: World Health Organization.
2.	 Foege, W.H., J.D. Millar, and J.M. Lane, Selective epidemiologic control in smallpox eradication. American Journal of Epidemiology, 1971. 94(4): p. 311-315.
3.	 NORAD, Accelerating progress in saving the lives of women 
and children, in The Global Campaign for the Health Millenium Development Goals. 2013, Ministry of Foreign Affairs, Norway: Oslo, Norway.
4.	 Peters, D.H., et al., Improving Health Services in Developing 
Countries: From Evidence to Action. 2009, Washington, DC, USA: World Bank.
5.	 Mandala, J., et al., Prevention of mother-to-child transmission of HIV in Zambia: implementing efficacious ARV regimens in primary health centers. BMC Public Health, 2009. 9(314).
6.	 de Savigny, D., et al., Introducing vouchers for malaria 
prevention in Ghana and Tanzania: context and adoption of innovation in health systems. Health Policy and Planning, 2012. 27(4): p. iv32-iv43.
7.	 Waddington, C.J., A price to pay, part 2: The impact of user 
charges in the Volta region of Ghana. International Journal of Health Planning and Management, 1990. 5(4): p. 287- 312.
8.	 Agyepong, I.A. and S. Adjer, Public social policy development and implementation: a case study of the Ghana National Health Insurance scheme. Health Policy and Planning, 2007. 23(2): p. 150-160.
9.	 Victoria, C.G., et al., Context matters: Interpreting impact 
findings in child survival evaluations. Healthy Policy & Planning, 2005. 20(1): p. i18-i31.
10.	Thaddeus, S. and D. Maine, Too far to walk: maternal 
mortality in context. Social Science Medicine, 1994. 38(8): p. 1091-110.
11.	de Savigny, D. and T. Adam, Systems thinking for health 
systems strengthening. 2009, Alliance for Health Policy and Systems Research. World Health Organization. 
12.	Sankar, B., Micro-planning in Andhra Pradesh: Ensuring 
Quality HIV Care for Individuals and Communities, in AIDSTAR-One. 2013, USAID: Arlington, VA, USA.
13.	Peters, D.H., et al., A balanced scorecard for health services 
in Afghanistan. Bulletin of the World Health Organization, 2007. 85: p. 146-151.
14.	Hansen, P.M., et al., Measuring and managing progress in 
the establishment of basic health services: the Afghanistan health sector balanced scorecard. International Journal of 
Health Planning and Management, 2008: p. 107-117.
15.	Edward, A., et al., Configuring balanced scorecards for 
measuring health system performance: evidence from 5 years evaluation in Afghanistan. PLoS Med, 2011: p. e1001066.
16.	WHO and ExpandNET, Nine steps for developing a scalingup strategy. 2010, Department of Reproductive Health and Research, WHO: Geneva, Switzerland.
17.	Agurto, I., et al., Improving cervical cancer prevention in a 
developing country. International Journal for Quality in Health Care, 2006: p. 81-86.
18.	Faden, R.R., et al., An ethics framework for a learning 
health care system: A departure from traditional research ethics and clinical ethics, in Hastings Centre Report. 2013. p. S16-27.
19.	Smith, M., et al., Best Care at Lower Cost: The Path to 
Continuously Learning Health Care in America. 2013, Insitute of Medicine of the National Academies: Washington, DC, USA.
20.	IOM, Crossing the Quality Chasm: A New Health System for 
the 21st Century. 2001, Institute of Medicine: Washington, DC, USA.
21.	Lam, A., Tacit knowledge, organizational learning and 
societal institutions: An integrated framework. Organization Studies, 2000. 21(3): p. 487-513.
22.	Ciliska, D., et al., Diffusion and dissemination of evidencebased dietary srategies for the prevention of cancer. Nutrition Journal, 2005. 4(13).
23.	Remme, J.H.F., et al., Defining Research to Improve Health 
Systems. PLoS Medicine, 2010. 7(11).

## References

24.	Curran, G.M., et al., Effectiveness-implementation hybrid 
designs: Combining elements of clinical effectiveness and implementation research to enhance public health impact. Medical Care, 2012. 50(3): p. 217-226.
25.	Proctor, E., et al., Outcomes for implementation research: 
conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health, 2011. 38(2): p. 65-76.
26.	Brownson, R.C., G.A. Colditz, and E.K. Proctor, Dissemination and Implementation Research in Health: Translating Science to Practice. 2012: OUP USA.
27.	Peters, D.H., L. Paina, and S. Bennett, Expecting the unexpected: Applying the Develop-Distort Dilemma to maximize positive market impacts in health. Health Policy Plan, 2012. 27(4): p. iv44-iv53.
28.	Cornwall, A. and R. Jewkes, What is participatory research? 
Social Science & Medicine, 1995. 41(12): p. 1667-1676.
29.	Subramanian, S., et al., Do we have the right models for 
scaling up health services to achieve the Millennium Development Goals? BMC Health Services Research, 2011. 11.
30.	WHO. Bloomberg initiative to reduce tobacco use. Tobacco 
Free Initiative (TFI) 2013  [cited 2013; Available from: http://www.who.int/tobacco/about/partners/bloomberg/ en/.
31.	Peden, M., G. diPietro, and A. Hyder, Two years into the 
road safety in 10 countries project: How are countries doing? Injury Prevention, 2012. 18(4): p. 279.
32.	WHO, Changing Mindsets: Strategy on Health Policy and 
Systems Research. 2012, Alliance for Health Policy & Systems Research
33.	Hanney, S. and M. Gonzalez-Block, Evidence-informed 
health policy: Are we beginning to get there at last? Health Research and Policy Systems, 2009. 7(30).
34.	Habicht, J.P., C.G. Victora, and J.P. Vaughan, Evaluation 
designs for adequacy, plausibility and probability of public health programme performance and impact. International Journal of Epidemiology, 1999. 28: p. 10-18. 
35.	Tran, N., et al., Engaging policy makers in road safety 
research in Malaysia: A theoretical and contextual analysis. Health Policy, 2009. 90(1): p. 58-65.
36.	Consortium, P. PERFORM: Improving Health Workforce 
Performance. 2011  [cited 2013; Available from: http:// www.performconsortium.com/.
37.	Newman, I., et al., A typology of research purposes and its 
relationship to mixed methods, in Handbook of mixed methods in social and behavioral research, A. Tashakkori 
and C. Teddlie, Editors. 2003, Sage: Thousand Oaks, CA. p. 167-188.
38.	American Educational Research, A., Standards for reporting 
on empirical social science research in AERA publications. Educational Researcher, 2006. 35: p. 33-40.
39.	Zwarenstein, M., et al., Improving the reporting of pragmatic trials: An extension of the CONSORT statement. BMJ: British Medical Journal, 2008.
40.	Fairall, L., et al., Task shifting of antiretroviral treatment 
from doctors to primary-care nurses in South Africa (STRETCH): A pragmatic, parallel, cluster-randomised trial. Lancet, 2012: p. 889-898.
41.	Baqui, A.H., et al., Effect of community-based newborncare intervention package implemented through two service-delivery strategies in Sylhet district, Bangladesh: A cluster-randomised controlled trial. Lancet, 2008: p. 
1936-1944.
42.	Baqui, A.H., et al., Community-based validation of assessment of newborn illnesses by trained community health workers in Sylhet district of Bangladesh. Tropical Medicine & International Health, 2009. 14(12): p. 1448-1456.
43.	Choi, Y., et al., Can mothers recognize neonatal illness correctly? Comparison of maternal report and assessment by community health workers in rural Bangladesh. Tropical Medicine & International Health, 2010. 15(6): p. 743-753.
44.	Shah, R., et al., Community-based health workers achieve 
high coverage in neonatal intervention trials: A case study from Sylhet, Bangladesh. Journal of Health, Population, & Nutrition, 2010. 28: p. 610. 

## References

45.	Davidoff, F., et al., Publication guidelines for quality improvement in health care: Evolution of the SQUIRE project. Quality & Safety in Health Care, 2008. 17: p. i3-i9.
46.	Brassard, M., The Memory Jogger II: A Pocket Guide of 
Tools for Continuous Improvement and Effective Planning. Vol. First Edition. 1994, Methuen, MA: Goal/QPC.
47.	Campbell, D.T., J.C. Stanley, and N.L. Gage, Experimental 
and quasi-experimental designs for research. 1963, Boston, 
USA: Houghton Mifflin.
48.	Speroff, T. and G.T. O'Connor, Study designs for PDSA quality improvement research. Quality Management in Healthcare, 2004. 13: p. 17-32.
49.	Lindelow, M. and A. Wagstaff, Assessment of Health Facility 
Performance: An Introduction to Data and Measurement Issues, in Are You Being Served? New Tools for Measuring Service Delivery, S. Amin, J. Das, and M. Goldsteim, Editors. 2008, The World Bank: Washington, DC, USA. p. 19-66.
50.	Mergler, D., Worker participation in occupational health 
research: theory and practice. International Journal of Health Services, 1987. 17: p. 151-167.
51.	Chambers, R., Revolutions in development inquiry. 2008, 
UK: Earthscan.
52.	Smith, L., L. Rosenzweig, and M. Schmidt, Best Practices in 
the Reporting of Participatory Action Research: Embracing Both the Forest and the Trees The Counseling Psychologist, 
2010. 38: p. 1115-1138.
53.	Israel, B.A., et al., Critical issues in developing and following CBPR principles. Community-based participatory research for health: From process to outcomes, 2008: p. 47-66.
54.	Roy, S.S., et al., Improved neonatal survival after participatory learning and action with women's groups: A prospective study in rural eastern India. Bulletin of the World Health Organization, 2013. 91(6): p. 426-433B.
55.	Onwuegbuzie, A.J. and K.M.T. Collins, A typology of mixed 
methods sampling designs in social science research. The Qualitative Report, 2007. 12: p. 281-316. 
 
56.	Pawson, R., et al., Realist review: "A new method of 
systematic review designed for complex policy interventions. Journal of Health Services Research & Policy, 2005. 10: p. 21-34.
57.	Dieleman, M., et al., Realist Review and Synthesis of Retention Studies for Helath Workers in Rural and Remote Areas, in Increasing access to health workers in rural and remote areas. 2011, World Health Organization: Geneva, Switzerland.
58.	Collins, K.M.T., A.J. Onwuegbuzie, and I.L. Sutton, A model 
incorporating the rationale and purpose for conducting mixed methods research in special education and beyond. Learning Disabilities: A Contemporary Journal, 2006. 4: p. 67-100.
59.	Leech, N.L. and A.J. Onwuegbuzie, Guidelines for Conducting and Reporting Mixed Research in the Field of Counseling and Beyond. Journal of Counseling & Development, 2010. 88: p. 61-69.
60.	Creswell, J.W., et al., Best practices for mixed methods 
research in the health sciences. Maryland, USA. 2011: National Institutes of Health.
61.	Tashakkori, A. and C. Teddlie, Mixed Methodology: Combining Qualitative and Quantitative Approaches. 1998: SAGE Publications.
62.	Creswell, J.W., Research design: Qualitative, quantitative, 
and mixed methods approaches. 2008: Sage Publications, Incorporated.
63.	Creswell, J.W. and V.L.P. Clark, Designing and Conducting 
Mixed Methods Research. 2010: SAGE Publications.
64.	O'Cathain, A., E. Murphy, and J. Nicholl, The quality of 
mixed methods studies in health services research. Journal of Health Services Research & Policy, 2008. 13: p. 92-98.
65.	Glasgow, R.E., T.M. Vogt, and S.M. Boles, Evaluating the 
public health impact of health promotion interventions: The RE-AIM framework. American Journal of Public Health, 1999. 89(9): p. 1322-1327.
66.	Rogers, E.M., Diffusion of Innovations, 5th Edition. 2003: 
Free Press. 

## References

67.	Damschroder, L.J., et al., Fostering implementation of 
health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science, 2009. 4(50): p. 50.
68.	Epstein, J., Generative Social Science: Studies in Agent-
Based Computational Modeling. 2006, Princeton, NJ, USA: Princeton University Press.
69.	Humphreys, G., Are antibiotics still "automatic" in France? 
Bulletin of the World Health Organization, 2011. 89(1): p. 8-9.
70.	Agyepong, I.A., et al., When solutions of yesterday become 
problems of today: Crisis-ridden decision making in a complex adaptive system (CAS) - the Additional Duty Hours Allowance in Ghana. Health Policy and Planning, 2012. 27: p. iv20-iv31.
71.	Cooksey, D., A review of UK health research funding. 2006, 
London: HM Treasury.
72.	Sanders, D. and A. Haines, Implementation research is 
needed to achieve international health goals. PLoS Med, 2006. 3: p. 719-722.

## Implementation Research In Health A Practical Guide

We spend billions on health innovations, but very little on how best to apply them in real-world settings. Without implementation research, we are at best committing valuable resources in the hope that things will work out. This guide is an attempt to redress the deficit in understanding of implementation research and to encourage programme personnel and implementers to take a greater interest in the subject, recognizing that implementation research is in fact an integral part of programme planning and execution, rather than something that happens once programmes are up and running.  Intended for newcomers to the field, those already conducting implementation research, and those with responsibility for implementing programmes, the guide provides an introduction to basic implementation research concepts and language, briefly outlines what it involves, and describes the many exciting opportunities that it presents. 

World Health Organization Avenue Appia 20 CH-1211 Geneva 27 Switzerland Tel: +41 22 791 29 73 Fax: +41 22 791 41 69 alliancehpsr@who.int http://www.who.int/alliance-hpsr http://www.implementationresearchplatform.org ISBN 978 92 4 150621 2