Implementation strategies for high impact nephrology trials: the end of the trial is just the beginning
Authors: Donna M. Reidlinger, David W. Johnson, Jonathan C. Craig, Stephen P. McDonald, Alistair R. Mallard, Laura E. Robison, Germaine Wong, Carmel M. Hawley
Journal: Kidney International (2022)

There remains wide variation in treatment patterns and outcomes for kidney patients, and nephrology continues to lag behind other medical specialties in the number and quality of randomized controlled trials (RCTs).1Bello A.K. Levin A. Lunney M. et al.Status of care for end stage kidney disease in countries and regions worldwide: international cross sectional survey.BMJ. 2019; 367: l5873Crossref PubMed Scopus (121) Google Scholar As currently conducted, RCTs require substantial investment in time and resources with enormous effort poured into trial design, funding, partnerships, data acquisition, analysis, and publications. Less attention is paid to activities to maximize practice and policy impact, and evaluation of evidence uptake commonly occurs only through the narrow lens of output metrics in peer-reviewed literature. Implementation (or research utilization, knowledge transfer, knowledge exchange, and knowledge translation) is the process of integrating new practices within a setting, and de-implementation (or de-commission, disinvest, de-adopt, and discontinue) is the removal from practice of low-value, wasteful or harmful treatments. Substantial and avoidable research waste, whereby promising research results fail to transform health care and improve patient outcomes or healthcare system efficiency, occurs as a result of low uptake of trial evidence.2Grimshaw J.M. Eccles M.P. Lavis J.N. et al.Knowledge translation of research findings.Implement Sci. 2012; 7: 50Crossref PubMed Scopus (1452) Google Scholar Publication of trial results does not signal the "end" of a trial; publication signals the beginning of activities to ensure implementation of high-value care, or de-implementation of low-value care, into healthcare policy and practice (Figure 1). The current mechanism for reporting new research involves several knowledge exchange pathways including journal publications, presentations, commentaries, opinion pieces, regulatory reviews, and evidence synthesis into clinical practice guidelines. In addition, pharmaceutical companies invest considerably in marketing new products, including "me-too" drugs (drugs within the same therapeutic class) that increase clinical treatment options. Ongoing global harm reduction and healthcare quality campaigns (e.g., Choosing Wisely; Too Much Medicine) may also support clinicians' point of care decisions. However, ample evidence shows that wide variation in evidence-based practice remains a problem,3Van Der Veer S.N. Tomson C.R.V. Jager K.J. Van Biesen W. Bridging the gap between what is known and what we do in renal medicine: improving implementability of the European Renal Best Practice guidelines.Nephrol Dial Transplant. 2014; 29: 951-957Crossref PubMed Scopus (21) Google Scholar implying key drivers in clinical decision-making are not adequately addressed by current knowledge-transfer mechanisms. Moreover, although multiple theories and frameworks have been designed to identify key drivers and inform strategies to bridge the knowledge–healthcare practice divide, practical application of these frameworks by researchers is at best modest, and reliable empirical data on their efficacy are scant.4Waltz T.J. Powell B.J. Chinman M.J. et al.Expert recommendations for implementing change (ERIC): protocol for a mixed methods study.Implement Sci. 2014; 9: 39Crossref PubMed Scopus (84) Google Scholar Although evidence is limited, it seems few kidney trials, particularly end-user focused, investigator-initiated trials, preplan for implementation or de-implementation. Two examples of kidney-related trials are discussed because they demonstrate how uptake of new knowledge into practice can be highly variable. The chosen examples meet commonly applied indicators of high impact research: (i) publication in a journal with a high journal impact factor, (ii) highly cited in the literature by peers, and (iii) sustained long-term relevance to and impact on clinical practice.S1 The Initiation of Dialysis Early And Late (IDEAL) trial was a landmark study conducted in Australia and New Zealand and published in the New England Journal of Medicine in 2010: The primary aim was to determine whether initiation of dialysis at an estimated glomerular filtration rate of 10–14 ml/min per 1.73 m2 compared with 5–7 ml/min per 1.73 m2 reduced all-cause mortality.S2 The trial showed that earlier dialysis initiation was not associated with improved outcomes or survival. In Canada, an immediate change in dialysis initiation timing was observed after publication of the IDEAL trial, with associated lower dialysis costs and no change in quality of life.S3 Similarly, in Australia and New Zealand, a rapid reduction in the trend for dialysis commencement at higher estimated glomerular filtration rate values was reported post-IDEAL, with accompanying substantial healthcare system economic benefits.S4 In the United States, uptake has been less marked; the percentage of patients commencing dialysis at higher estimated glomerular filtration rates increased from 1996 until 2010 but remained stable or has been slightly decreasing since 2010.S5 The international Trial to Reduce Cardiovascular Events with Aranesp Therapy (TREAT) study was published in the New England Journal of Medicine in 2009.S6 TREAT investigated the use of an erythropoiesis-stimulating agent (ESA), darbepoietin alfa to treat anemia in patients with type 2 diabetes mellitus and chronic kidney disease (CKD), and examined the benefit of aiming for a high hemoglobin (Hb) target, >13 g/dl in this cohort compared with a placebo group, whose hemoglobin levels were allowed to fall to around 9 g/dl before a prespecified rescue therapy could be invoked. The trial found no benefit compared with placebo and a harm signal. Globally, prescription of ESA therapy had already been in decline after safety concerns raised by earlier trials (Correction of Anemia with Epoetin Alfa in Chronic Kidney Disease [CHOIR]S7 and Cardiovascular risk Reduction by Early Anemia Treatment with Epoetin Beta [CREATE]S8); however, this decline accelerated in the post-TREAT period. In the United States, despite changes to regulatory and clinical guidelines, post-TREAT ESA use only significantly declined in 2011, after the introduction of Medicare's new payment system for dialysis providers that replaced a lucrative fee-for-service ESA reimbursement model. Although patient characteristics remained unchanged, Medicare's new payment model triggered an immediate alignment of dialysis unit practices with ESA treatment guidelines,S9 signaling a potential role for healthcare payment reform in de-implementing low-value or harmful practices. Within this complex environment, implementation science theory, a potential tool for enhancing knowledge exchange, remains underused by clinical researchers. Inconsistent nomenclature, alongside an extensive and growing array of conceptual implementation frameworks, many of which are yet to be empirically tested, may deter researchers from applying a theoretical framework to their knowledge-transfer activities.4Waltz T.J. Powell B.J. Chinman M.J. et al.Expert recommendations for implementing change (ERIC): protocol for a mixed methods study.Implement Sci. 2014; 9: 39Crossref PubMed Scopus (84) Google Scholar Thus, the translation of evidence into clinical practice does not always occur or is often significantly delayed. Grimshaw et al.2Grimshaw J.M. Eccles M.P. Lavis J.N. et al.Knowledge translation of research findings.Implement Sci. 2012; 7: 50Crossref PubMed Scopus (1452) Google Scholar suggested that implementation activities targeting healthcare professionals and consumers are more likely to be effective if informed by 5 key knowledge-transfer questions:(i)What should be transferred?(ii)To whom should it be transferred?(iii)By whom should it be transferred?(iv)How should it be transferred?(v)With what effect should it be transferred? Additionally, for kidney research, methodical consideration of structural or situational barriers and enablers will explain "why" knowledge from landmark kidney trials might fail to be translated, and these should inform early knowledge-transfer planning activities. The Reach, Effectiveness, Adoption, Implementation, Maintenance (RE-AIM) framework is a frequently applied public health research framework that provides (i) practical information to enhance knowledge translation activities of evidence-based interventions into practice and (ii) enables evaluation of implementation reach and effectiveness for sustained practice change. However, RE-AIM does not explain the structural or situational conditions that may impact knowledge-transfer outcomes. The Consolidated Framework for Implementation Research (CFIR), a widely used conceptual framework, addresses the systemic contexts not currently considered by RE-AIM. Therefore, although RE-AIM promotes understanding of the what, to whom, by whom, how, and with what outcome questions of evidence-based knowledge transfer, the CFIR uses 5 domains (intervention characteristics, outer and inner setting, individual characteristics, and process) to address why an implementation strategy may (or may not) be likely to succeed.5King D.K. Shoup J.A. Raebel M.A. et al.Planning for implementation success using RE-AIM and CFIR frameworks: a qualitative study.Front Public Health. 2020; 8: 59Crossref PubMed Scopus (75) Google Scholar Successful implementation of evidence-based treatments depends on a 2-pronged top-down and bottom-up approach. Used in isolation, the top-down approach, which involves rigid compliance with scientifically robust protocols to produce an evidence base, may conflict with a bottom-up approach, which focuses on utility of the intervention to end-users. The integrated RE-AIM and CFIR framework approach has previously been successfully used in other health settings5King D.K. Shoup J.A. Raebel M.A. et al.Planning for implementation success using RE-AIM and CFIR frameworks: a qualitative study.Front Public Health. 2020; 8: 59Crossref PubMed Scopus (75) Google Scholar,6Holtrop J.S. Estabrooks P.A. Gaglio B. et al.Understanding and applying the RE-AIM framework: clarifications and resources.J Clin Translat Sci. 2021; 5: e126Crossref PubMed Scopus (66) Google Scholar and has recently been identified as the planned approach for a multistage implementation and evaluation protocol of an electronic health self-management tool for kidney patients.7Donald M. Beanlands H. Straus S. et al.A research protocol for implementation and evaluation of a patient-focused ehealth intervention for chronic kidney disease.Global Implement Res Appl. 2022; 2: 85-94Crossref PubMed Google Scholar For RCTs in kidney disease, the integrated use of the 2 frameworks may allow a complementary top-down (RE-AIM), bottom-up (CFIR) approach to implementation that would enable researchers to strategically plan for implementation and impact measurement (Figure 2). Perhaps the most important enabler of implementation is ensuring that a trial is implementable in the first place. Implementability refers to the generation of new evidence underpinned by prior consideration of practical components related to the validity, relevance, and usability of results that may influence evidence uptake.8Cumpston M.S. Webb S.A. Middleton P. et al.Understanding implementability in clinical trials: a pragmatic review and concept map.Trials. 2021; 22: 232Crossref PubMed Scopus (8) Google Scholar Trial implementability can therefore be enhanced by paying proper attention to appropriate design, conduct, and reporting to facilitate uptake of research results by diverse kidney care centers and for patient groups in many settings (metropolitan vs. regional and remote, academic vs. nonacademic, low vs. high resource, private vs. public, and so on). A recent literature review conducted for the Australian Clinical Trials Alliance drew on a broad group of disciplines to produce a concept map for use by trialists to enhance the implementability of their research.8Cumpston M.S. Webb S.A. Middleton P. et al.Understanding implementability in clinical trials: a pragmatic review and concept map.Trials. 2021; 22: 232Crossref PubMed Scopus (8) Google Scholar Concepts such as co-design and applying standardized statistical and analytical methods to assist end-user interpretation were among 38 items presented in the tool. Importantly, several activities require little or no additional resources that research teams can readily apply to their trials (Figure 3). In many developed countries, agencies such as the U.K.'s National Institute for Health and Care Excellence (NICE) and the U.S.'s Institute for Clinical and Economic Review (ICER) use an economic and ethical lens to review evidence, including from RCTs, to conduct independent health technology assessments on new healthcare therapies. Although independent agencies such as NICE and ICER may act as "watchdogs" to ensure equitable access to treatments at a fair price, health technology assessments are developed for population-level decisions and are not intended as a tool for individual patient healthcare choices. Furthermore, health technology assessments draw heavily on comparative clinical effectiveness evidence from RCTs, considered the most reliable for assessing treatment effectiveness. However, many RCTs lack external validity or generalizability, making extrapolation of treatment efficacy to treatment effectiveness in clinical practice difficult. Medical societies are powerful vehicles for supporting practice change and promoting evidence uptake. In the absence of a prespecified systematic implementation plan for the results of the IDEAL trial, the trial's rapid uptake and consequent practice change in Australia and New Zealand and Canada were likely supported through the extensive knowledge-translation activities facilitated by local nephrology societies. The interest generated from these events was followed soon after by subsequent synthesis of IDEAL's evidence into updated versions of the Canadian Society of Nephrology, Kidney Disease: Improving Global Outcomes (KDIGO), and European Renal Association–European Dialysis and Transplant Association (ERA-EDTA) guidelines. In contrast, in the United States where no immediate significant early dialysis decrease occurred, the 2015 Kidney Disease Outcomes Quality Initiative (KDOQI) clinical practice guidelines maintained a more conservative approach and stopped short of recommending deferral, thus potentially tempering IDEAL's global impact. Although there are complexities around the influence of industry partners on guideline development, a link between industry payments and favorable nephrology guideline recommendations was reported and discussed extensively after the 2006 update of the KDOQI anemia guidelines.S10 Recently, major departures by KDIGO from the national standards for managing financial conflicts of interests, alongside the (not necessarily related) conclusion that many KDIGO guidelines are predicated on the weak evidence of expert opinion, has renewed interest in the relationship between industry partners and guideline producers.9Chengappa M. Herrmann S. Poonacha T. Self-reported financial conflict of interest in nephrology clinical practice guidelines.Kidney Int Rep. 2021; 6: 768-774Abstract Full Text Full Text PDF PubMed Scopus (3) Google Scholar Clearly, this issue is not unique to nephrology clinical practice guidelines; however, compliance with a minimum set of standards for managing financial conflicts of interests, enforced by journal editors, may enhance synthesis of RCT evidence into guideline recommendations. For kidney research around dialysis and transplantation, measurement of evidence uptake, policy change, and impact on patient outcomes can be derived from registries, such as the Scientific Registry of Transplant Recipients, Australia and New Zealand Dialysis and Transplant Registry, and the ERA-EDTA Registry. Although more work is required in this area, a scoping search of the literature revealed that impact measurement using registry data was rarely systematically performed for kidney trials. Furthermore, there are few large-scale, clinical, quality CKD registries or existing longitudinal observational datasets with sufficient data granularity for research. These are valuable hypothesis-generating resources and, where an RCT may not be feasible, can aid discovery of clinical care practice patterns and relationships to inform future research. Additionally, conducting individual patient data meta-analyses using "raw" observational data rather than aggregated data from published RCTs enables researchers to examine person-level heterogeneity of treatment effects and assess generalizability of RCT evidence. However, considerable challenges exist, possibly related to disparate approaches to registry data collection and exacerbated by the extensive resource use (technical and human) required for data access, an obstacle in many typically under-resourced research settings. Therefore, the challenges of outcome measurement in the CKD population often rests on the independent capture of current CKD management practices by individual research groups using qualitative research methods and self-reported survey instruments. As can be seen, there are several leverage points along the translation pathway to enhance evidence uptake, but the difficulty lies in how individual components might be synergized to create an appropriate, unbiased structure to drive evidence translation. For the present, critical appraisal of implementation barriers and enablers using the CFIR methodology, in conjunction with alignment of the RE-AIM framework to the trial design during the development stages of kidney trials, may assist research teams to address some of the current issues. Resourcing for implementation and impact measurement remains a challenge. A 2008 study found most funders consider knowledge translation to be the shared responsibility of the funding agency and the researcher.S11 However, beyond the grant itself, which typically ends with the main results publication, funding agencies and institutions rarely explicitly provide financial support for knowledge-transfer activities. Furthermore, many trial research groups consider it a conflict of interest to perform knowledge-translation activities for their own research. Although funders and researchers have a role in translating research knowledge, other stakeholders may be better positioned to ensure trial outcomes inform treatment practices.2Grimshaw J.M. Eccles M.P. Lavis J.N. et al.Knowledge translation of research findings.Implement Sci. 2012; 7: 50Crossref PubMed Scopus (1452) Google Scholar The type of trial and characteristics of its reported outcomes, such as any identified safety or cost issues, could act to identify the group with the leading responsibility for implementation of results (or de-implementation of disproven therapies). This deserves wider debate, however, and Figure 4 provides a suggested approach for defining implementation responsibility. For example, implementation of results from kidney trials reporting a safety concern, such as those investigating ESAs to find the optimal degree of anemia correction in patients with CKD, might be expected to fall under the auspices of regulatory agencies, such as the U.S. Food and Drug Administration (FDA). A recent example in the context of the current coronavirus disease 2019 (COVID-19) pandemic highlights the critical role regulatory agencies can play in fast-tracking the implementation of promising therapies. Based on data from a single (ongoing) phase III trial (PROVENT), the FDA issued an emergency use authorization for the unapproved product Evusheld (AstraZeneca), a combination of 2 monoclonal antibodies tixagevimab and cilgavimab. Monoclonal antibodies provide an alternative method of protection for immunocompromised populations who are likely to continue to have greater vulnerability to COVID-19 illness, even after multiple doses of vaccine. Although the FDA's evidence standard for an emergency use authorization is lower than the standard for product approvals, given the severity of COVID-19 infection in solid organ transplant recipients and the urgency to find effective alternative strategies to COVID-19 vaccination to protect high-risk patients, Evusheld was authorized for emergency use as a pre-exposure prophylaxis for prevention of COVID-19 in certain immunocompromised adults and pediatric patients.S12 For studies reporting outcomes from repurposed established therapies or processes of care and for which implementation will provide no commercial advantage (or may even provide a possible disadvantage) for pharmaceutical companies, responsibility would likely fall to clinicians (via their professional collaborations and medical societies) to ensure appropriate uptake or de-implementation. Defining these actors has important implications for reducing research waste, maximizing efficient use of scarce health resources, minimizing patient harm, and improving patient quality of life. For many trials, evidence uptake after publication has been insufficient to confer a broader benefit to the global kidney care community. Publication of results is not the end of a trial; rather, publication signals commencement of activities to enhance evidence uptake by stakeholders. More research is required to inform and evaluate strategies for these activities. Nevertheless, the CFIR and RE-AIM frameworks are promising avenues of enquiry for kidney researchers. However, the greatest challenge for the kidney research community is how we might synergize our needs with the needs of influential, key knowledge translation stakeholders (pharmaceutical companies, health translation agencies, medical societies, and guideline producers) to create an unbiased translation pathway for evidence implementation.