Georgia Institute of Technology
From the SelectedWorks of Diana Hicks

2022

Social science research making an impact on
public decision-making
Kimberley R Isett
Diana Hicks

Available at: https://works.bepress.com/diana_hicks/55/

Social Science Research Making an Impact on Public Decision Making

Kimberley R. Isett* & Diana Hicks***Biden School of Public Policy and Administration
University of Delaware
Newark DE**School of Public Policy
Georgia Institute of Technology
Atlanta GA
kri@udel.edu
dhicks@gatech.edu

Acknowledgments:
We would like to thank Stephanie Noble for her work on creating an annotatedbibliography for this project. Her work was indispensable to the synthesis in this paper.
A seed grant from the Executive Vice President for Research at Georgia Tech and the
Quick Wins program funded early work. The authors were able to more fully developthese concepts and do the empirical work through the generosity of NSF award
51366FE.

1

Abstract
Many social science researchers seek to and are increasingly evaluated on their ability to
influence public decision makers. However, the literature suggests that making an
impact on decision makers is difficult. Yet, that same literature provides guidance on
getting research used. In this paper, we take a critical look at one part of this genre –policy use of research in health services. We find that the literature emphasizes
characteristics of knowledge that are under the control of researchers: content salience,
effective communication, quality and attending to parameters of use. However,theories of agenda setting and decision making suggest that many aspects of use of
research are beyond the control of researchers. Therefore, evaluation of researchers’
success in achieving policy use of their results can only rest on effort expended and not
results achieved.
Keywords
Agenda setting, decision making, policy, quality, communication, salience

Introduction
The quandary of getting research used in practice is an age-old one. For example, in thehealth sciences, it takes anywhere between 15 and 20 years before well-evidenced
practices are in widespread use in doctors’ offices (Isett & Phillips, 2010; Morris,
Wooding, & Grant, 2011; Panzano & Roth, 2006). This delay is in a field where there has
been a concerted evidence dissemination project for decades. What does that mean forthe social sciences where there has been less focus on translation to policy which is amore diffuse and more difficult realm to track? In this paper, we ask how social science
research and the evidence generated by that research comes to be used by publicdecision makers in the hopes of providing guidance for social scientists who will be
evaluated in part on their ability to change the course of public policymaking.
2

Policy use of research-based evidence is of broad concern, and several leading
institutions have developed frameworks to guide the analysis of research impact. In
Figure 1, the Centers for Disease Control and Prevention’s (CDC) impact framework aidentifies five “domains of influence” that drive health outcomes: disseminating science,creating awareness, catalyzing action, effecting change, and shaping the future (seefigure 1). The graphic is circular with arrows connecting every action, emphasizing
looping back and lack of chronological order; therefore, the degree of impact is not a
progression. The CDC has identified at least seven indicators for each domain, providing
a starting point for those seeking to make the case that their work has had policy
impact.
In another example, the Institute of Medicine (IOM) introduced the Degrees of Impact
Thermometer (Fineberg, 2013) (Figure 2). Reminiscent of Knott & Wildavsky’s (1980)
seven stages of knowledge use, the IOM’s five degrees of impact are: spreading themessage, receiving recognition, informing the field, inspiring action, and effectingchange. The actions are ordered from bottom to top on a graphic of a traditional
mercury thermometer, the implication being that while spreading the message is a start,
effecting change is a higher level of achievement. Two to five indicators are juxtaposedwith each action, from published articles in journals through improved health outcomes.
Both frameworks portray activities, only some of which are the responsibility of
researchers. Both suggest data that can be collected to indicate that the activity has
been undertaken. Both include policy change among their indicators. In the CDC’s
framework, we find Congressional hearings (in catalyzing action), legal/policy changes(in effecting change), and implementation of public health programs/initiatives (inshaping the future). On the IOM thermometer, we see subject of Congressional
hearing/investigation (by informing the field), legislation introduced (by inspiring
action), and policy change, including organizational policy (by effecting change).
Though the CDC and IOM figures articulate actions required to achieve impact andindicators of those actions having taken place, they are silent on what knowledge shouldlook like to enable impact. For insights to guide the work of knowledge production, welook to the well developed literature in health services that examines the translation ofresearch into practice in public decision making. We summarize the myriad of publishedarticles on how knowledge should be configured to be useful to decision makers andpresent our interpretation of the most common recommendations.

Configuring knowledge to enable use
The literature on health-related evidence translation in public decision making is one of
the best developed --more so than that in many other policy related areas. Thus, wea Found here: http://www.cdc.gov/od/science/impact/framework.html, August 26, 20203

focus primarily on that body of work. We began with two influential articles in the field:
Greenhalgh et al. (2004) and Jewell and Bero (2008). We then used a snowball
approach to collect review and framework articles cited by or citing these two articles
using both PubMed and Google Scholar. Articles were screened for a focus specificallyon evidence translation into public decision making. In all, 32 articles are included inthis review. We read each article looking for convergence and created a synthesis that
presents the most common elements across each of the 32 articles. Where feasible, we
leverage cross citations from other domains to illustrate commensurability with work inother substantive areas. The synthesis is organized into the following categories:
content salience, effective communication, quality, and parameters of use.

Content Salience

The concept of evidence is contested. What constitutes evidence is not at all agreed
upon by decision makers or even scientists and scholars – let alone the general public(Drake et al., 2001). In the social sciences, there are different kinds of evidence(qualitative and quantitative, empirical, and personal lived experience) as well as
differing levels of objectivity or rigor. So the question becomes, what evidence is “goodenough” for a policy decision (Atkins et al., 2005; Isett et al., 2016). The existing
literature does not have a definitive answer, but it does contain pointers.
Effective public evidence needs both qualitative and quantitative components (Jewell &Bero, 2008). On the qualitative side, stories provide an emotional hook and an intuitive
appeal (Brownson, Chriqui, & Stamatakis, 2009; Troy & Kietzman, 2016). Stamatakis
and colleagues (2010) point out that “protagonists [of a story] should reflect animportant constituency that could benefit from the proposed policy change.” Inparticular, stories are vital when the benefits to a group are difficult to monetize, suchas safety or quality of life (Brownson et al., 2009; Stamatakis et al., 2010).
Arguably, scholarship is not needed to generate stories for policy arguments –individuals are best at relaying their personal stories directly. The strength ofscholarship lies in providing quantitative evidence and arguments that can be difficult togenerate from other sources. Social scientists are skilled in providing cost-benefit
analyses, descriptive statistics, and impact analyses. They may, however, be less awareof or interested in other types of numbers influential with public decision makers. There
are three categories of numbers highlighted in the current literature. First, descriptionsof a problem that highlight disparities in the population (Stamatakis et al., 2010). How
are distinct communities differentially benefitting from or disadvantaged by the statusquo? Second, projections are essential. In particular, the number of cases that could
be prevented by a given intervention (Hanney et al., 2003), the cost of policy inaction
and how these costs are distributed (Stone, 1989), and the distribution of the benefits of
the intervention (Hanney et al., 2003; Jewell & Bero, 2008; Stamatakis et al., 2010) areof interest to public decision makers. Finally, for policy areas where outcomes are longterm, Fielding and Briss (2006) suggest including intermediate measures of benefits toprovide a shorter time to payoff for policymakers forced to work on short policy cycles.
4Geography matters for relevance to a policymaker as well. Statements about a generalneed or a pervasive phenomenon do not distinguish themselves from the informationnoise that surrounds decision makers (Hanney et al., 2003). Instead, information needs
to be specific to policymakers’ local jurisdiction and based on data about the people
they serve (Brownson, et al., 2016; Fielding & Frieden, 2004; Hanney et al., 2003;
Laugesen & Isett, 2013; Murthy et al., 2012; van de Goor et al., 2017). Policymakers also
want to see how they compare to their peers (Stamatakis et al., 2010; Stone, 1989)–
those jurisdictions that they think are similar or competitors to themselves can compel
decision makers to act, not wanting to be the “low man.” The connection between
causal ideas and responsibility to act (Isett, Laugesen, & Cloud, 2015; Stone, 1989), anda moral obligation to fix an identified problem (Gamble & Stone, 2006) for theirconstituents is crucial, both objectively and comparatively.

Effective communication

The literature is clear on the need for high-quality communication. This is an area
many scholars shy away from, feeling the data should speak for itself (Pisano, 2016).
Busy decision makers have limited time and cognitive resources to filter the vast
amount of material needed to make decisions. They must rely on both heuristics (aboutsource and content) and the summaries provided to them by others (Cyert & March,
1963; Dagenais, Laurendeau, & Briand-Lamarche, 2015; Ostrom, 1998). Whileacademics often point to the systematic review as a gold standard, these kinds of
documents present barriers to uptake (Tricco et al., 2016). So concise, well writtensummaries are crucial.
The ability to summarize and distill information transparently and credibly is key togetting decision makers’ attention without contributing to overload (Burris et al., 2010;
Cyert & March, 1963; Hanney et al., 2003; Murthy et al., 2012). Information presented
straightforwardly (without jargon) that is easy to understand can be absorbed more
quickly – and possibly used (Burris et al., 2010; Coffman, Hong, Aubry, Luft, & Yelin,
2009; Gamble & Stone, 2006). Targeted information, with explicit scope and relevanceto the decision at hand, shaven of all secondary and tangential information, needs to beinserted into the process at the right time (Brownson, Fielding, et al., 2009; Burris et al.,2010; Coffman et al., 2009; Hanney et al., 2003; Lavis, Oxman, Moynihan, & Paulsen,
2008). Further, the benefits of policy adoption should be visible and unambiguously
presented (Atkins et al., 2005; Gamble & Stone, 2006). “Well designed translational
materials and strategies that reflect an understanding of the constraints thatpolicymakers face certainly have a greater chance of carrying the day than do research
reports that are physically and cognitively inaccessible to policymakers” (Burris et al.,2010, p. 197).
Cognitive inaccessibility is a critical issue and one that social scientists do not necessarily
understand. Studies find that quantitative data is inaccessible to most public decision
makers (Brownson et al., 2009) who are trained to do other things than sort and5interpret data. At least one leading study has shown that public decision makers
confuse cause and correlation (Jewell & Bero, 2008). Moreover, while showing decision
makers and their staffs evidence or teaching them how to access it seems like it might
address capacity deficits (Redman et al., 2015; VanLandingham & Silloway, 2016), there
is limited evidence that this works (Murthy et al., 2012).
Thus, the craft of messaging empirical research is crucial to getting research used.
Above all, messages must be tweaked for multiple audiences (Troy & Kietzman, 2016;van de Goor et al., 2017). The details given to a policymaker differ from those produced
for agencies, advocacy, or the public and take into consideration their different foci,authority, and scope of operations (Hanney et al., 2003; Lavis et al., 2003; Oliver et al.,2014; Sabatier & Jenkins-Smith, 1993). While multiple messaging artifacts should beinternally consistent, the language, highlights, and modes of communication differ.
Because the requirements of multiplex messaging are somewhat at odds with therequirements of scholarly incentives, and because any single study is rarely definitive
enough to guide policy by itself, intermediaries play a significant role in facilitating theuse of research in policy (Lavis et al., 2003; Meagher & Lyall, 2013). Intermediaries can
produce systematic overviews incorporating research-based insights crafted in clear
language, targeted to a policy domain. Lemay and Sá (2014) found that aggregatedsynthesizes produced by the World Health Organization and other public healthagencies were most commonly used by those in a Canadian public health agency tolearn about new scientific evidence. Users know the organizations in their domains that
produce broad syntheses and so can quickly find relevant reviews. The bottom line isthat social science findings do not stand alone on their merits. Instead, they must be
interpreted for use and comprehension by decision makers and the general public(Stamatakis et al., 2010) so that there is a consensus about the cause and solution to agiven issue.

Quality

While social scientists like to think that they know good evidence when they see it, thecriteria that public servants use to assess evidence are somewhat different and morenuanced. In scientific circles, the randomized controlled trial and a low p-value are thegold standards indicating quality results. However, public decision makers want toknow how good the evidence is that something will affect outcomes in a meaningfulway (Atkins et al., 2005) --that is effect size, not statistical significance.
Credibility is the second component of quality evidence. Strictly speaking, credibility“involves the scientific adequacy of the technical evidence and arguments” (Cash et al.,2003, p.8086). However, policy audiences without the requisite specialist expertise to
make a technical judgment, instead tend to assess credibility through face validity of themessenger, or the research team (Brownson, Chriqui, et al., 2009; Lavis et al., 2008;
Lavis et al., 2003). Professional or known government organizations are often

6

considered credible sources (Lavis et al., 2003), as are researchers from respected
universities.

Parameters of Use

It is a mistake to think that just because evidence points to a problem (or a solution)that something can be done to address it. A public system must have the capacity to dosomething. Hanney and colleagues (2003) point out that sometimes a system justcannot absorb the findings. Either the system has little to no resources to address theconcern, or no technical solution exists.
Given this constraint, the extant literature provides insight into how evidence is used,
when it is used. First, public decision makers are more likely to use evidence if it
comports with their operating realities. Recommendations need to be feasible andconsistent with decision makers’ jurisdictions, resources, authority, and scope of the
problem (Brownson, Fielding, et al., 2009; Lavis et al., 2003). Solutions that fall outside
established authorities cannot be implemented and might be illegal. Reversibility is also
crucial for public servants. Decision makers might require room to back away frompublicly stated positions when implementation or context changes (Jewell & Bero, 2008;
Zaltman, Duncan, & Holbek, 1984). Path dependency stemming from a potential
decision could lead to risk aversion and maintaining the status quo rather than engaging
in needed change.
Getting evidence into the hands of and used by public decision makers can be facilitatedby interaction. Interaction, including feedback from the decision makers, is a way toengage and co-opt the decision-making body to take up the evidence from a given
investigation. These feedback mechanisms can be as simple as a conversation to inquire
about perceived shortcomings - simple knowledge transfer methods are as effective as
more complex ones (Dobbins et al., 2009). And active engagement with decision
makers is more effective than passive modes of knowledge transmission (Lavis et al.,2003). Cash et al. (2003) established that ongoing interaction is central to providing
salient, credible, legitimate information. Projects that involve interaction only at the
beginning risk providing solutions to yesterday’s problems (not salient) or outdated
knowledge (not credible), and excluded parties often question the legitimacy of
information resulting from ensuing conversations.
The literature also addresses the purpose of evidence. Over and over, the literaturesuggests that evidence is not used to determine when to intervene (Murthy et al.,2012). Instead, it is used to understand how to fix problems (Fielding & Briss, 2006;
Lavis et al., 2003; Liverani, Hawkins, & Parkhurst, 2013). Using evidence to illustrate a“new” problem exists is likely ineffective. Identifying a new problem means that a newprogram needs to be developed. New programs have high upfront costs and long term
benefits, yielding a low powered incentive (Fielding & Briss, 2006; Frant, 1996). Despite
reluctance to initiate a new program, evidence is more likely to be used to start a newprogram rather than replacing an existing one (Fielding & Briss, 2006) – adding to thestatus quo rather than changing it (Cyert & March, 1963; Downs, 1967).
7

Configuring the knowledge is not enoughAttending to content salience, effective communication, high quality information andthe use context is demanding, but still may not be enough. Complexities of the policy
world are outside the control of researchers and also determine whether research isused by decision makers.
The discussion above surfaced many strong points, its focus on the quality ofevidence is perhaps the most potent aspect. Well executed analyses that illustrated theextent of a problem generated by a credible messenger are central. The messengers arelikely a mix of the scholars themselves and intermediaries that aggregate and interpretthe findings for policymakers. This mix is consistent with the policy process wherescholars do not necessarily have to participate directly in the policy process to havetheir work utilized.
Another strength is the discussion of the breadth of quantitative approaches that areused and valued by policymakers. A range of quantitative methods will make a case, but
descriptive quantification seems to be more critical than sophisticated modeling.
Description is more easily understood and helps with a basic understanding of the
problem and an imperative to act. However, the literature reviewed does not assist inunderstanding which kinds of numbers matter when and whether certain types ofnumbers have more impact under certain circumstances.
It is unreasonable to expect policymakers to invest in understanding the complicatedformulas unique to every study. Intermediaries know this and strive to offer a story of
numbers that is uncomplicated, clear, and stripped of unnecessary content except forthe credibility of the author of the number. This is why white papers and policy
documents contain references. The number is never truly alone. It derives a halo ofcredibility from its referenced source. Thus staged, numbers can captivate thediscussion and motivate change. Despite the power of captivating numbers, policy
stories are narratives. Documents that include numbers incorporate them in syntheses
deploying a wide range of data.
A final strength lies in the emphasis on a strong narrative. As one author has said, it is
the ideas that sell a policy, not the data behind it (Lavis et al., 2003). The ability to
weave a compelling story about needs, obligations, and solutions is crucial to thesuccess of the policy. Numbers contribute to narratives supporting the story. A strong
narrative is essential; often, that narrative is constructed by an intermediary, not thescholar.
There is a clear division of labor in the policy process that harkens back to Oakerson’s(1999) division of the production (scholarly output of research) and provision (useful,targeted synthesis of research) of public goods –the good here being policy evidence.
This is likely necessary, as studies have illustrated that researchers may not be well

8

suited to the task (Fiske & Dupree, 2014). Scholars do not feel comfortable lobbying, as
being an advocate would reduce their credibility as information providers in the eyes ofdecision makers. Intermediaries serve as the linchpin between scholarly output andinputs to the policy process. They work to bundle related studies that bear on the topic
of interest, contextualize, and interpret the information for salience to and easy
processing by the decision making body (c.f. Dutton, 1997). Intermediaries can also
resolve the tensions scholars face when interacting with policy – lack of time, clarity
versus complexity in communication, timely input versus in-depth, high-quality
assessment (Sarkki et al., 2014). Known intermediaries can be “honest brokers”
producing credible syntheses that are useful and unbiased. This aspect of the policy
process has been underemphasized in the current literature and ought to beinvestigated more thoroughly.
Other areas of the discussion seem less relevant. In particular, interaction with
policymakers and proposing a solution are not aspects always present. Often
scholarship contributes to the agenda-setting phase of policy. This leads to the questionof which elements are necessary and which are sufficient to move scholarship into
policy use. We need to understand better which characteristics are required, and if thecontext matters for when the elements are relevant.
Analytically, the difference between making marginal changes or undertaking asignificant overhaul of a program is unclear. Which is within the boundaries of
proposing a new program or out of its scope? These differences are essential forunderstanding the analytical distinctions between “new” vs. “old” programs. Borrowing
from classic treatments of agency behavior (Downs, 1967), it is plausible that new
programs or staff lines in a programmatic expansion would be welcomed by agency staff
as an extension of their turf. How does this comport with the framework and theresistance to new programs asserted in it?
A more general concern could be that this is an overly rational and sanguine way toperceive the policy process (e.g., Cairney et al., 2016) that does not attend to policy
areas that necessarily have to lurch due to rapid knowledge development or fast-moving
policy windows (Kingdon, 1995). However, there are stable policy areas that change
slowly over time because of the nature of the substantive domain. On a deeper level,
the use of evidence in policy requires a system committed to openness, transparency,
and accountability in decision-making processes, characteristics that should not betaken for granted (Donadelli, 2020).
Another way the discussion is inconsistent with existing theory is in the role ofambiguity in the policy process. The idea that the benefits to society must be presented
in unambiguous terms conflicts with classic political science findings that ambiguity is
conducive to policy being passed (e.g., Stone, 2002). This may be one area where thetangibility of the policy domain makes a difference. When may policy domain

9ambiguities permit ambiguous language about the benefits and harms of policy
proposals and solutions?
Nothing here, on face value, is specific to health, so it is reasonable to assert
generalizability. However, further development could determine better scopeconditions. Can these lessons be applied only within domains with stable policy arenas,
and unambiguous knowledge? At this point, scoping conditions are not clear.
The analysis above presents evidence use in public decision making as a linear, technicalissue. These ideas were by and large developed as grounded theory to conceptualize
what happened when evidence was used in public decision making. Sometimes, what
did happen was combined with what ought to happen – as a normative assessment ofwhat leaders believe. These perspectives overemphasize the technical portions of theprocess (good science that combines stories and numbers) and underemphasize thesocial and contextual portions of the process.
Within the current paradigm, we know that the existence of high-quality research andproviding access to that research does not significantly increase the use of evidence inpolicymaking (Dobbins et al., 2009; Gamble & Stone, 2006; Hanney et al., 2003;
Stamatakis, McBride, & Brownson, 2010). We also know that public decision makers
agree that evidence should be used in decision making (Brownson, Fielding, & Maylahn,
2009; Newman et al., 2016). However, what public decision makers and researchersconsider to be evidence may differ.
Empirical evidence is just one input into the socio-political process of making publicdecisions. Public decision makers necessarily combine values and evidence when
making decisions (Atkins, Siegel, & Slutsky, 2005; Cairney et al., 2016) –that is their job.
If public decisions were purely technical affairs, there would be less debate on the“best” course of action (Hanney et al., 2003; Jewell & Bero, 2008; Lavis et al., 2003).
Evidence does not eliminate debate but is simply one input into that debate (Atkins etal., 2005; Troy & Kietzman, 2016). Any conception of the public decision-making
process that posits a purely technical approach to evidence’s influence is naïve andmisguided.
Thus, in Table 1 we reconfigure the elements discussed above and combine them with atheoretical lens to produce a blended framework emphasizing four elements: technical,
jurisdictional, social, and scientific context.

10

Table 1. Characteristics of knowledge and policy context affecting policy use
Technical Jurisdictional
SocialScientific Context
Analysis
Constituents affected
Problem recognition
Synthesis
Solutions Administrative feasibility Solution acceptability
Gaps
Authority

Government intervention

Best available
solution

Technical. Here we see the empirical prowess of the research community. The
entirety of the quantitative and qualitative elements of content salience is contained
here. Problems are given shape through numbers and words but are establishedempirically. Solutions, also modeled empirically, fit here. These are the methodologicalfindings produced from systematic investigation and are devoid of the constraints ofoperating realities and social parameters of public decision making. Technical elements
can determine the possible – although those solutions can sometimes be improbable.
Jurisdictional. In this component are all the elements that address the internal
constraints and pressures public decision makers face. The extent to which the
problems exist or will soon exist in their jurisdiction is vital here. Public decision makers
are not very likely to allocate scarce resources to issues, no matter how real, that are
not affecting significant or widespread constituencies they serve. Additionally, publicorganizations must be able to implement policies and programs that solve issues. This isrelated to both the administrative feasibility and whether there is the authority to
implement such programs.
Social. The fact that constituents help to determine what is a problem in theircommunities, and which issues deserve public attention is almost absent from theexisting characterizations. Social movements can, and do, provide powerful momentum
for public attention allocation. Further, while those in the field know that a lack ofsocial acceptability can derail even the most important (Gaines 2014 b) and impactful(Piorkowski 2017 c) public initiatives, this aspect has been neglected. Finally, part ofacceptability is the broader question of whether the government is the appropriate
entity to solve problems that are agreed to exist. While at times, these social aspects
have been acknowledged, current frameworks do not incorporate them. We not only
see this component as critical, consistent with Heikkila and Isett’s (2004) external
institutions, but we believe it should carry more weight than the technical andjurisdictional components in any evidence uptake framework.
Scientific Context. Like the social component, our model places a heavier
weighting and emphasis on the scientific context of evidence use in public decisionb http://www.macon.com/news/local/article30150015.html

chttp://healthpolicy.unm.edu/sites/default/files/Navajo%20Nation%20Soda%20Tax%20E

xpansion%20Brief%20-%20Final.pdf

11

making. How science is packaged and placed into its knowledge context can be crucial
to its chances of being used. There are three elements to this: evidence ought to besynthesized and contextualized, gaps identified and explicitly stated to suggest thelimitations of knowledge, and a conclusion drawn that provides the best available
solution given the evidence. The last part – a conclusion drawn is a piece that makes
most researchers uncomfortable.
The discomfort with drawing conclusions and making recommendations combines withinformation synthesizers’ role to underscore information intermediaries’ importance.
There are different kinds of information intermediaries in the knowledge ecosystem.
Here we limit our discussion to third party organizations that are not lobbyists.
Intermediaries play a crucial role in brokering the massive corpus of empirical researchin a digestible and targeted way for a policy problem. These organizations have highcredibility, and their long-term engagement in an area leads to visibility, thus garnering
attention from public decision makers. We posit that intermediaries play a central rolein the uptake of evidence in public decision making, a role outsized to existing
characterizations of them.

Implications for Research Assessment
This chapter examines what is required for social science to have a policy impact. We
identified some elements researchers could control, for example, content salience,
effective communication, quality, and attending to parameters of use. However, within
the broader classical theory of agenda setting and public decision making, we see muchthat is relevant to the use of knowledge in policy is beyond the control of social science
researchers. Assessment of contributions to policy making made by social science
research must engage with the full complexity discussed above, see Table 1.
Researchers can only be accountable for their effort because the results achieved
depend on complex policy contextual factors.
Researchers need to remember that overall, quantitative work wrapped in a compelling
narrative wins the day. Dazzle them with ideas, not the technical details of the data
behind it. Tell a story that is bolstered by empiricism, but tell the story. Most of all, help
individuals understand the salience of the solution to them and how it comports with
their daily lives or how change will make daily lives better. Policy solutions are nottechnical. They are human.

References

Atkins, D., Siegel, J., & Slutsky, J. (2005). Making policy when the evidence is in dispute.
Health Aff (Millwood), 24(1), 102-113. https://doi.org/10.1377/hlthaff.24.1.102

12

Brownson, R. C., Chriqui, J. F., & Stamatakis, K. A. (2009). Understanding evidence-based
public health policy. Am J Public Health, 99(9), 1576-1583.
https://doi.org/10.2105/AJPH.2008.156224
Brownson, R. C., Dodson, E. A., Kerner, J. F., & Moreland-Russell, S. (2016). Framing
research for state policymakers who place a priority on cancer. Cancer Causes &Control, 27(8), 1035-1041. https://doi.org/10.1007/s10552-016-0771-0
Brownson, R. C., Fielding, J. E., & Maylahn, C. M. (2009). Evidence-based public health: afundamental concept for public health practice. Annu Rev Public Health, 30, 175201. https://doi.org/10.1146/annurev.publhealth.031308.100134
Burris, S., Wagenaar, A. C., Swanson, J., Ibrahim, J. K., Wood, J., & Mello, M. M. (2010).
Making the case for laws that improve health: a framework for public health law
research. Milbank Q, 88(2), 169-210. https://doi.org/10.1111/j.14680009.2010.00595.x
Cairney, P., Oliver, K., & Wellstead, A. (2016). To bridge the divide between evidence
and policy: reduce ambiguity as much as uncertainty. Public Administration
Review, 76(3), 399-402.
Case, D. (2012). Looking for information: a survey of research on information seeking,needs and behavior. Emerald.
Cash, D. W., Clark, W. C., Alcock, F., Dickson, N. M., Eckley, N., Guston, D. H., . . .
Mitchell, R. B. (2003). Knowledge systems for sustainable development.
Proceedings of the National Academy of Sciences, 100(14), 8086-8091.
Coffman, J. M., Hong, M. K., Aubry, W. M., Luft, H. S., & Yelin, E. (2009). Translating
medical effectiveness research into policy: lessons from the California Health
Benefits Review Program. Milbank Q, 87(4), 863-902.
https://doi.org/10.1111/j.1468-0009.2009.00582.x
Cyert, R. M., & March, J. G. (1963). A behavioral theory of the firm. Blackwell.
Dagenais, C., Laurendeau, M. C., & Briand-Lamarche, M. (2015). Knowledge brokering inpublic health: A critical analysis of the results of a qualitative evaluation.
Evaluation and Program Planning, 53, 10-17.
https://doi.org/10.1016/j.evalprogplan.2015.07.003
Dobbins, M., Hanna, S. E., Ciliska, D., Manske, S., Cameron, R., Mercer, S. L., . . .
Robeson, P. (2009). A randomized controlled trial evaluating the impact ofknowledge translation and exchange strategies. Implement Sci, 4, 61.
https://doi.org/10.1186/1748-5908-4-61
Dodson, E. A., Geary, N. A., & Brownson, R. C. (2015). State legislators’ sources and useof information: bridging the gap between research and policy. Health Education
Research, 30(6), 840-848. https://doi.org/10.1093/her/cyv044
Donadelli, F. (2020). When evidence does not matter: The barriers to learning fromscience in two cases of environmental policy change in Brazil. Science and Public
Policy, 47(3), 313-321. https://doi.org/10.1093/scipol/scaa006
Downs, A. (1967). Inside bureaucracy. Little Brown.
Drake, R. E., Goldman, H. H., Leff, H. S., Lehman, A. F., Dixon, L., Mueser, K. T., & Torrey,
W. C. (2001). Implementing evidence-based practices in routine mental healthservice settings. Psychiatric Services, 52(2), 179-182.
13

Dutton, J. E. (1997). Strategic agenda building in organizations. In Z. Shapira (Ed.),
Organizational decision making (pp. 81-107). Cambridge University Press.
Fielding, J. E., & Briss, P. A. (2006). Promoting evidence-based public health policy: canwe have better evidence and more action? Health Aff (Millwood), 25(4), 969978. https://doi.org/10.1377/hlthaff.25.4.969
Fielding, J. E., & Frieden, T. R. (2004). Local knowledge to enable local action. American
Journal of Preventive Medicine, 27(2), 183-184.
doi.org/10.1016/j.amepre.2004.04.010
Fineberg, H. V. (2013). Presidential Address: The institute of medicine: what makes it
great? Institute of Medicine Annual Meeting.
Fiske, S. T., & Dupree, C. (2014). Gaining trust as well as respect in communicating tomotivated audiences about science topics. Proceedings of the National Academyof Sciences of the United States of America, 111, 13593-13597.
https://doi.org/10.1073/pnas.1317505111
Frant, H. (1996). High-powered and low-powered incentives in the public sector. Journalof Public Administration Research and Theory, 6(3), 365-381.
Gamble, V. N., & Stone, D. (2006). U.S. policy on health inequities: the interplay ofpolitics and research. J Health Polit Policy Law, 31(1), 93-126.
https://doi.org/10.1215/03616878-31-1-93
Greenhalgh, T., Robert, G., Macfarlane, F., Bate, P., & Kyriakidou, O. (2004). Diffusions ofinnovations in service organizations: systematic review and recommendations.
Milbank Quarterly, 82(4), 581-629.
Hanney, S. R., Gonzalez-Block, M. A., Buxton, M. J., & Kogan, M. (2003). The utilisation ofhealth research in policymaking: concepts, examples and methods ofassessment. Health Res Policy Syst, 1(1), 2.
Heikkila, T., & Isett, K. R. (2004). Modeling operational decision making in publicorganizations - An integration of two institutional theories. American Review of
Public Administration, 34(1), 3-19. https://doi.org/10.1177/0275074003260911
Isett, K. R., Head, B., & VanLandingham, G. (2016). Caveat Emptor: What do we know
about public administration evidence and how do we know it? Public
Administration Review, 76(1). https://doi.org/10.1111/puar.12467.
Isett, K. R., Laugesen, M. J., & Cloud, D. H. (2015). Learning from New York City: a casestudy of public health policy practice in the Bloomberg administration. Journal of
public health management and practice, 21(4), 313-322.
Isett, K. R., & Phillips, S. D. (2010). Improving practice-research connections throughtechnology transfer networks. Journal of Behavioral Health Services Research,
37(1), 111-123. https://doi.org/10.1007/s11414-009-9183-1
Jewell, C. J., & Bero, L. A. (2008). Developing good taste in evidence: facilitators of andhindrances to evidence-informed health policymaking in state government.
Milbank Q, 86(2), 177-208. https://doi.org/10.1111/j.1468-0009.2008.00519.x
Kingdon, J. W. (1995). Agendas, alternatives, and public policies (2nd ed.). HarperCollins.
Knott, J., & Wildavsky, A. (1980). If dissemination is the solution, what is the problem?.
Knowledge, 1(4), 537-578.

14

Laugesen, M. J., & Isett, K. R. (2013). Policymakers’ use of evidence to change publichealth in New York City. Frontiers in Public Health Services and Systems Research,2(7).
Lavis, J. N., Oxman, A. D., Moynihan, R., & Paulsen, E. J. (2008). Evidence-informed
health policy 1 - synthesis of findings from a multi-method study of organizations
that support the use of research evidence. Implement Sci, 3, 53.
https://doi.org/10.1186/1748-5908-3-53
Lavis, J. N., Robertson, D., Woodside, J. M., McLeod, C. B., Abelson, J., & Knowledge
Transfer Study, G. (2003). How can research organizations more effectivelytransfer research knowledge to decision makers? Milbank Q, 81(2), 221-248,
171-222.
Lemay, M. A., & Sá, C. (2014). The use of academic research in public health policy andpractice. Research Evaluation, 23(1), 79-88.
Liverani, M., Hawkins, B., & Parkhurst, J. O. (2013). Political and institutional influenceson the use of evidence in public health policy. A systematic review. PLoS One,
8(10), e77404. https://doi.org/10.1371/journal.pone.0077404
Meagher, L., & Lyall, C. (2013). The invisible made visible: using impact evaluations toilluminate and inform the role of knowledge intermediaries. Evidence & Policy,
9(3), 409-418. https://doi.org/10.1332/174426413x14818994998468
Morris, Z. S., Wooding, S., & Grant, J. (2011). The answer is 17 years, what is thequestion: understanding time lags in translational research. J R Soc Med,
104(12), 510-520. https://doi.org/10.1258/jrsm.2011.110180
Murthy, L., Shepperd, S., Clarke, M. J., Garner, S. E., Lavis, J. N., Perrier, L., . . . Straus, S.
E. (2012). Interventions to improve the use of systematic reviews in decisionmaking by health system managers, policy makers and clinicians. Cochrane
Database Syst Rev, 9, CD009401.
https://doi.org/10.1002/14651858.CD009401.pub2
Newman, J., Cherney, A., & Head, B. W. (2016). Do policy makers use academic
research? reexamining the “two communities” theory of research utilization.
Public Administration Review, 76(1), 24-32. https://doi.org/10.1111/puar.12464
Oakerson, R. J. (1999). Governing local public economies: Creating the civic metropolis.
ICS Press.
Oliver, K., Innvar, S., Lorenc, T., Woodman, J., & Thomas, J. (2014). A systematic review
of barriers to and facilitators of the use of evidence by policymakers. BMC HealthServ Res, 14(2). https://doi.org/10.1186/1472-6963-14-2
Ostrom, E. (1998). A behavioral approach to the rational choice theory of collective
action: Presidential Address, American Political Science Association. American
Political Science Review, 92(1), 1-22.
Panzano, P. C., & Roth, D. (2006). The decision to adopt evidence-based and other
innovative mental health practices: risky business? Psychiatr Serv, 57(8), 11531161.
Pisano, M. (2016). How research can drive policy: econometrics and the future ofcalifornia’s infrastructure. Public Administration Review, 76(4), 538-539.

15

Redman, S., Turner, T., Davies, H., Williamson, A., Haynes, A., Brennan, S., . . . Green, S.
(2015). The SPIRIT action framework: A structured approach to selecting andtesting strategies to increase the use of research in policy. Social Science &Medicine, 136, 147-155. https://doi.org/10.1016/j.socscimed.2015.05.009
Sabatier, P. A., & Jenkins-Smith, H. C. (1993). Policy change and learning : an advocacy
coalition approach. Westview Press.
Sarkki, S., Niemela, J., Tinch, R., van den Hove, S., Watt, A., & Young, J. (2014). Balancing
credibility, relevance and legitimacy: A critical assessment of trade-offs inscience-policy interfaces. Science and Public Policy, 41(2), 194-206.
https://doi.org/10.1093/scipol/sct046
Stamatakis, K. A., McBride, T. D., & Brownson, R. C. (2010). Communicating prevention
messages to policy makers: the role of stories in promoting physical activity. J
Phys Act Health, 7 Suppl 1, S99-107.
Stone, D. (1989). Causal stories and the formation of policy agendas. Political Science
Quarterly, 104(2), 281-300.
Stone, D. (2002). Policy paradox: The art of political decision making (revised edition).
Norton.
Tricco, A. C., Cardoso, R., Thomas, S. M., Motiwala, S., Sullivan, S., Kealey, M. R., . . .
Straus, S. E. (2016). Barriers and facilitators to uptake of systematic reviews by
policy makers and health care managers: a scoping review. Implementation
Science, 11(1), 4. https://doi.org/ARTN 410.1186/s13012-016-0370-1
Troy, L. M., & Kietzman, K. G. (2016). Enhancing Evidence-Based Public Health Policy
Developing and Using Policy Narratives. Journal of Gerontological Nursing, 42(6),11-17. https://doi.org/10.3928/00989134-20160516-04
van de Goor, I., Hämäläinen, R. M., Syed, A., Lau, C. J., Sandu, P., Spitters, H., ... & Aro, A.
(2017). Determinants of evidence use in public health policy making: Resultsfrom a study across six EU countries. Health Policy, 121(3), 273-281.
https://doi.org/10.1016/j.healthpol.2017.01.003
VanLandingham, G., & Silloway, T. (2016). Bridging the gap between evidence and policy
makers: A case study of the pew-macarthur results first initiative. Public
Administration Review, 76(4), 542-546. https://doi.org/10.1111/puar.12603
Zaltman, G., Duncan, R., & Holbek, J. (1984). Innovations & Organizations. Kreiger.

16Figure 1 Five domains of CDC influence

Figure 2 Institute of Medicine Degrees of Impact

17