Comparative Logic Modeling for policy analysis: the case of HIV testing policy change at the Department of Veterans Affairs.

Health care decision makers have used research evidence to justify adoption of Centers for Disease Control and Prevention (CDC) human immunodeficiency virus (HIV) testing guidelines into a variety of health care settings. In 2006, the CDC released revised HIV testing guidelines for adults and adolescents (Branson et al. 2006). Within the U.S. Department of Veterans Affairs (VA), reports emerged indicating that HIV testing rates were low (Owens et al. 2007), prompting reevaluation of VA HIV testing practices and policies. As a result, in 2009, the VA adopted CDC recommendations by eliminating written informed consent requirements for HIV testing and making testing a routine part of Veterans' health services (Department of Veterans Affairs 2009). Translation of specific clinical recommendations and evidence into policies and practice within health care systems is a major challenge. If the structural logic and rationale of the CDC recommendations are not clear, initiatives and resource allocations necessary for a change in testing policy could be difficult to implement. James and Jorgensen (2009) have suggested that research utilization theory offers a robust conceptual framework for assessing the policy process. There is a pressing need to use effective evaluation tools to reveal the evidence-based resources, inputs, and outputs of policy in a systematic, logical, and transparent manner as health care settings implement the CDC guidelines. Logic Modeling is a technique that offers a way to analyze and quantify how policy makers use research—how their “research utilization” informs the process and outcomes of translating specific research-based knowledge into evidence-based practice (Rich 1997). To understand research utilization, we propose a novel application of the logic model. Logic Modeling provides a visual representation of input, throughput, and output components that are brought together to produce intended change. While Logic Modeling can be used after completion of primary activities as a way of evaluating how well policy was able to meet intended outcomes, it may also be used earlier in institutional change processes to plan for and guide future evaluation. Logic Modeling as an evaluation tool has been used widely to examine the resources, inputs, outputs, and outcomes of programs in a clear and systematic fashion. Therefore, we applied this to better understand the evolution of VA HIV testing and assess how evidence was brought to bear on the policy design. Over the past 40 years, logic models have largely been applied to evaluations of specific social programs (Wholey 1987), but Logic Modeling has also been used in both community-based and systems-level initiatives (Julian 1997; Moyer, Verhovsek, and Wilson 1997; Kaplan and Garrett 2005), in planning (Dwyer 1996; Macaskill et al. 2000), management (Millar, Simone, and Carnevale 2001), and preevaluation to develop indicators and document outcomes (Kellogg Foundation 2004; Innovation Network 2008). Logic Modeling can stimulate reflection (Moyer, Verhovsek, and Wilson 1997), enable communication, and promote continued learning toward health service objectives (Kellogg Foundation 2004; Innovation Network 2008). Policy documents such as the VA HIV directive and CDC HIV recommendations represent endpoints within the decision making process and are particularly relevant to successful replication of policies within other contexts. Used as primary sources for analysis, policy documents can counter the problematic “fuzziness” of health policy by providing insights into the formal rules and intentions of policy measures (Kroneman and van der Zee 1997). James and Jorgensen (2009) have suggested that by working backward from final policy statements, it is possible to reconstruct policy decision making to explore knowledge utilization, a term which encompasses both scientific and nonscientifically generated information sources apparent in the policy making process (Trostle, Bronfman, and Langer 1999; Dobrow, Goel, and Upshur 2004). In this formulation, “research utilization” is a sub-type of knowledge utilization that can describe the development of evidence-based practices. Use of research data in policy making has previously been characterized as instrumental (i.e., direct, actionable), conceptual (i.e., diffuse, gradually enlightening), or symbolic (i.e., strategic, tactical) (Pelz 1978; Weiss 1979; Trostle, Bronfman, and Langer 1999; Hanney et al. 2003; Almeida and Bascolo 2006). We explore the symbolic use of research evidence. This use can occur to attain political legitimacy, support a position, give confidence, reduce uncertainty, and raise financial resources for policy decisions (Hanney et al. 2003). Despite the personal, economic, and public health benefits of HIV screening, the revised 2006 CDC guidelines have not been universally adopted. Lack of consensus in how testing policy should be applied may stem in part from lack of transparency in how evidence-based resources, inputs, and outputs have been synthesized, translated, and adopted in different settings. We propose that when informed by research utilization theory, policy Logic Modeling can generate this transparency. Modeling policy alongside causal assumptions allows us to compare across policy settings in order to better understand the rationales of specific policy choices. Where research-based justifications are entirely lacking in the model, there may exist gaps in CDC or VA policy logic. We propose that by generating insights into the policy making process, comparative Logic Modeling is thus an effective tool for health policy analysis.