Priority setting for Canadian Take-Home Naloxone best practice guideline development: an adapted online Delphi method

Background: Take-Home Naloxone (THN) is a core intervention aimed at addressing the toxic illicit opioid drug supply crisis. Although THN programs are available in all provinces and territories throughout Canada, there are currently no standardized guidelines for THN programs. The Delphi method is a tool for consensus building often used in policy development that allows for engagement of stakeholders.Methods:We used an adapted anonymous online Delphi method to elicit priorities for a Canadian guideline on THN as a means of facilitating meaningful stakeholder engagement. A guideline development group generated a series of key questions that were then brought to a 15-member voting panel. The voting panel was comprised of people with lived and living experience of substance use, academics specializing in harm reduction, and clinicians and public health professionals from across Canada. Two rounds of voting were undertaken to score questions on importance for inclusion in the guideline.Results:Nine questions that were identified as most important include what equipment should be in THN kits, whether there are important differences between intramuscular and intranasal naloxone administration, how stigma impacts access to distribution programs, how effective THN programs are at saving lives, what distribution models are most effective and equitable, storage considerations for naloxone in a community setting, the role of CPR and rescue breathing in overdose response, client preference of naloxone distribution program type, and what aftercare should be provided for people who respond to overdoses.Conclusions:The Delphi method is an equitable consensus building process that generated priorities to guide guideline development.
with minimal adverse effects [bib_ref] Modelling the combined impact of interventions in averting deaths during a synthetic-opioid..., Irvine [/bib_ref] [bib_ref] Reducing the harm of opioid overdose with the safe use of naloxone:..., Kim [/bib_ref]. In Canada, publiclyfunded provincial and territorial Take-Home Naloxone (THN) programs were launched between 2012 and 2017 to provide people who use drugs and their families and friends with naloxone and training on naloxone administration [bib_ref] Lessons learned from ramping up a Canadian Take Home Naloxone programme during..., Young [/bib_ref]. The efficacy of THN programs has been demonstrated globally through studies in Canada, Germany, the UK, and the USA [bib_ref] Naloxone effectiveness: a systematic review, Chimbar [/bib_ref] [bib_ref] Exploring the life-saving potential of naloxone: a systematic review and descriptive meta-analysis..., Mcauley [/bib_ref]. In British Columbia (BC), modeling studies estimated that a total of 298 deaths were prevented by the BC THN programme between January 2012 and October 2016, and 1580 death events were prevented by the BC THN programme between April 2016 and December 2017 [bib_ref] Modelling the combined impact of interventions in averting deaths during a synthetic-opioid..., Irvine [/bib_ref] [bib_ref] Distribution of take-home opioid antagonist kits during a synthetic opioid epidemic in..., Irvine [/bib_ref].
The Canadian THN Guideline development project was funded through the Canadian Research Initiative in Substance Misuse Emerging Health Threat Implementation Science program in response to escalating mortality associated with opioid toxicity. THN programs are widely recognized as an important response to the to toxic drug supply yet there are discrepancies within THN programs in Canada, such as whether intranasal THN kits are available and the number of ampoules/vials of naloxone included in THN kits.
Guidelines are based on current evidence and used to optimize consistent service provision and administration [bib_ref] Clinical practice guideline development manual, third edition: a quality-driven approach for translating..., Rosenfeld [/bib_ref]. There are currently no Canadian guidelines on THNs. The World Health Organization and American Society of Addiction Medicine released guidelines that discussed THN, and the European Monitoring Centre for Drugs and Drug Addiction published a report on current evidence on THN in 2014, 2015, 2016, respectively [bib_ref] American Society of Addiction Medicine (ASAM) national practice guideline for the use..., Kampman [/bib_ref]. Further, Tsuyuki et al. released Canadian guidelines for naloxone prescribing by pharmacists [bib_ref] Canadian national consensus guidelines for naloxone prescribing by pharmacists, Tsuyuki [/bib_ref].
While past THN guidelines offer valuable information, we identified a gap in guidelines developed specifically for community THN distribution programs, which have different informational needs and challenges compared to professional groups. The Canadian harm reduction context may differ from other countries as there is regional variation regarding the scope and quantity of harm reduction policies [bib_ref] Canadian harm reduction policies: a comparative content analysis of provincial and territorial..., Cameron [/bib_ref] , available harm reduction services [bib_ref] Principles, practice, and policy vacuums: policy actor views on provincial/territorial harm reduction..., Hyshka [/bib_ref] , and THN distribution programswithin Canada. Additionally, harm reduction science continues to develop. In this study, we aim to integrate new evidence and engage stakeholders from across the country to generate new guideline recommendations [bib_ref] Clinical practice guideline development manual, third edition: a quality-driven approach for translating..., Rosenfeld [/bib_ref]. A guideline with updated evidence review will help address the current surge in deaths due to illicit opioid toxicity in the Canadian context.

# The delphi method
The Delphi method is a consensus building approach used to inform policy decisions [bib_ref] How to use the nominal group and Delphi techniques, Mcmillan [/bib_ref] [bib_ref] Consensus development methods: a review of best practice in creating clinical guidelines, Black [/bib_ref]. Through gathering diverse expert perspectives, the Delphi method leverages a range of direct knowledge and experience while stimulating discussion between members. This leads to stronger, context-relevant decision-making; an essential outcome for guideline development where the end product must be applicable over multiple jurisdictions [bib_ref] Consensus development methods: a review of best practice in creating clinical guidelines, Black [/bib_ref]. Anonymous decisions are elicited, individual views are aggregated and presented back to the group and the process is repeated a recommended two times; this allows for an exchange of ideas while ensuring anonymity and equitable space to voice alternate views [bib_ref] Consensus development methods: a review of best practice in creating clinical guidelines, Black [/bib_ref].
Guideline development groups should be multi-disciplinary and represent diverse views, involving a variety of methodological experts and clinicians, as well as any population affected by the guidelines. The Delphi method facilitates equitable participation by ensuring that everyone's input and perspectives will be included, and reducing the potential for uneven power dynamics or dominant personalities to unduly influence the process [bib_ref] How to use the nominal group and Delphi techniques, Mcmillan [/bib_ref]. This is important in clinical and academic projects related to harm reduction, where the voices of people with lived or living experience of substance use (PWLLE), while necessary for guideline validity, are often absent or disrespected. Previous studies have used the Delphi approach in the field of harm reduction [bib_ref] Elements of community-based models for treating hepatitis C virus in supporting HCV..., Pourmarzi [/bib_ref] [bib_ref] Effectiveness and implementability of state-level naloxone access policies: expert consensus from an..., Smart [/bib_ref].

# Methods

## Study aim
The aim of the adapted online Delphi method was to investigate which research questions stakeholders believed should be prioritized in a Canadian THN guideline.

## Study setting
All meetings and data collection were conducted online in order to get perspectives of stakeholders from across Canada and to respect physical distancing measures during the COVID-19 pandemic. Author MF facilitated all subcommittee meetings with support from author JN. Authors AM, CS, KR, TDR sat on the Guideline Steering Committee.

## Guideline development group formation
Guideline Development Group recruitment was an iterative process aiming for pan-Canadian membership and diverse perspectives. Four Guideline Development Group subcommittees participated in the Delphi method.
Recruitment occurred between November 2020 and January 2021. The study principal investigators and the affiliated Canadian Research Initiative in Substance Misuse (CRISM) working group members nominated members for the Guideline Development Group. These initial members nominated other potential stakeholders to expand and diversify recruitment using a snowball sampling approach.
PWLLE were explicitly invited via the snowball sampling process. PWLLE who were connected or unconnected to advocacy, research, or frontline harm reduction organizations were eligible for participation. We recruited from existing communities of PWLLE such as the Canadian Association of People who Use Drugs, the People with Lived and Living Expertise of Drug Use National Working Group, and Professionals for Ethical Engagement of Peers, from our professional networks, and sought feedback from identified PWLLE to ensure diversity of experiences and cross-country representation.
We invited participants from all subcommittees to introduce themselves in whatever manner made them feel comfortable, including lived and living experience, but did not explicitly ask all group members whether they identify as people who use drugs. We recruited sufficient self-identified PWLLE to ensure representation in all subcommittees. PWLLE in the Guideline Development Group who were not being compensated for participation through their workplace were provided $25/hr honoraria.
Effort was made to find representation across Canadian provinces and territories through the local and national networks of existing members. Indigenous, Black, and People of Color and 2S/LGBTQ group members were explicitly invited in the snowball sampling process.
Subcommittees of the Guideline Development Group included:
1. The Guideline Steering Committee (composed of eight members) steered discussions, encouraged constructive debate, summarized main points and key decisions, and gave oversight to the guideline development process. Author AM, a member of the Guideline Steering Committee, introduced meeting facilitation strategies drawing on experiences facilitating with community groups and varied stakeholders on harm reduction topics in British Columbia [bib_ref] Putting indigenous harm reduction to work: developing and evaluating "not just Naloxone, Levine [/bib_ref]. Meetings began with territory acknowledgements, and facilitation techniques such as introductions, icebreakers and co-creating group agreements were introduced into the Guideline Development Group meetings to foster safer space for group members to share perspectives and to address potential power dynamics [bib_ref] Putting indigenous harm reduction to work: developing and evaluating "not just Naloxone, Levine [/bib_ref]. This collaborative process acknowledged and honored the varied experiences, cultures, and strengths of committee members and provided the foundation for safer engagement [bib_ref] Putting indigenous harm reduction to work: developing and evaluating "not just Naloxone, Levine [/bib_ref] and promoted a more relational process of knowledge generation. We invited participants to participate in whatever manner made them most comfortable whether that included simply speaking up, using "raise hand" function in ZOOM video conferencing software, or typing in ZOOM chat.

## Key question development
We identified initial key questions based on a recent scoping review of the literature on the use of naloxone for reversal of opioid overdose in a community setting [bib_ref] Take-home naloxone programs for suspected opioid overdose in community settings: a scoping..., Moustaqim-Barrette [/bib_ref]. Themes identified in the scoping review included general opioid overdose response strategies, naloxone administration methods, naloxone distribution program strategy and implementation, cost-effectiveness of program implementation, and acceptability of naloxone distribution programs [bib_ref] Take-home naloxone programs for suspected opioid overdose in community settings: a scoping..., Moustaqim-Barrette [/bib_ref].
Three subcommittees (Guideline Steering Committee, Affected Community Committee, and Clinical Expertise Committee) reviewed these initial questions and suggested new questions before the Guideline Development Panel voted on the importance of each question. The Guideline Development Group discussed whether to include questions on cost-effectiveness due to concern about the potentially stigmatizing effect of comparing lives lost to public health programming cost. However, some Guideline Development Group members described challenges getting funding for services in the absence of evidence on cost-effectiveness. Questions on cost remained in the voting process but were flagged for careful consideration for presentation of evidence.
The Guideline Steering Committee, Affected Community Committee, and Clinical Expert Committee recommended that plain language questions rather than academic research questions or statements be developed to guide the process [bib_ref] Clinical practice guideline development manual, third edition: a quality-driven approach for translating..., Rosenfeld [/bib_ref]. The Guideline Development Group found it easier to evaluate the importance of the questions if they were in plain language format.

# Delphi method
A total of 15 Guideline Development Panel members participated in voting on which key questions should be addressed in the guideline. The Guideline Development Group was composed of people that had overlapping lived or living/clinical/academic experience. A table with descriptions of the panel members' professional and advocacy roles is included as a Additional file 1.
Methodology for the Delphi method was drawn from McMillan et al. [bib_ref] How to use the nominal group and Delphi techniques, Mcmillan [/bib_ref]. Data collection tools were developed and piloted by research team members not involved in tool development and administered online via REDCap (Research Electronic Data Capture) software. The Guideline Development Panel was presented with 35 key questions and asked: "Do you agree that this question is important to discuss in a best practice guideline?" (see [fig_ref] Figure 1: reviewed for content, completeness, and readability by Guideline Steering Committee, Affected Community... [/fig_ref]. Response options were presented in a 5-point Likert scale (strongly agree/agree/ neutral/disagree/strongly disagree where strongly agree = 1 and strongly disagree = 5) and respondents were then asked: "Can you please explain how you came to the rating for the previous question?" Consensus was operationalized a priori as a mean score below 2.0 [bib_ref] How to use the nominal group and Delphi techniques, Mcmillan [/bib_ref]. Key questions which scored 2.0 or above were excluded from the second round of voting. In the second round, panel members were asked to consider the reasons their peers agreed/disagreed with the importance of the remaining key questions and rate these questions once more. The mean scores for each question were then calculated, using R, version 4.0.5. See [fig_ref] Figure 1: reviewed for content, completeness, and readability by Guideline Steering Committee, Affected Community... [/fig_ref] for an overview of Delphi method.
The method was adapted from early literature on the Delphi by moving data collection online to accommodate for the national scope of the project and to avoid limitations related to the ability to meet in-person during the COVID-19 pandemic [bib_ref] How to use the nominal group and Delphi techniques, Mcmillan [/bib_ref].
All Guideline Development Panel members participated in the first round of voting (15/15) while 80% of members (12/15) participated in the second round of voting.
The research team used the CREDES tool to inform reporting of the Delphi method in this manuscript [bib_ref] Guidance on Conducting and REporting DElphi Studies (CREDES) in palliative care: recommendations..., Junger [/bib_ref].

# Results
A total of 17 of the 35 original research questions scored below 2.0 in the first round of voting and so were retained for the second round. Nine of the 17 key questions were rated below 2.0 in the second round of voting and qualified for inclusion in the THN guideline. The mean scores for both rounds are presented in [fig_ref] Table 1: Key questions scored independently by Guideline Development Panel [/fig_ref]. The questions with the lowest score in the second round of voting (and therefore the highest perceived importance) were on what should be in naloxone kits and the effectiveness of different methods of naloxone administration.

# Discussion
The Delphi method was conducted to identify what questions community stakeholders thought important to prioritize in a Canadian guideline on THN. The following key questions were identified as important to address: What equipment should be in THN kits, whether there are important differences between intramuscular and intranasal naloxone administration, how stigma impacts access to distribution programs, how effective are THN programs at saving lives, what distribution models are most effective and equitable, storage considerations for naloxone in a community setting, the role of CPR and rescue breathing in overdose response, client preference of naloxone distribution program type, and what aftercare should be provided for people who assist with overdoses.
Group heterogeneity among those making decisions can lead to more thorough consideration of all relevant aspects of a topic including applicability; by including a diversity of stakeholders, we were able to thoroughly weigh the pros and cons of key questions from different perspectives [bib_ref] Consensus development methods: a review of best practice in creating clinical guidelines, Black [/bib_ref]. The Delphi method is especially important for ensuring meaningful inclusion of PWLLE within clinical and academic contexts and has been applied elsewhere in harm reduction research [bib_ref] Elements of community-based models for treating hepatitis C virus in supporting HCV..., Pourmarzi [/bib_ref] [bib_ref] Effectiveness and implementability of state-level naloxone access policies: expert consensus from an..., Smart [/bib_ref]. PWLLE who provide harm reduction services including overdose response are uniquely knowledgeable about harm reduction practices and services, engaging communities of people who use drugs (PWUD), and creating comfortable and safe service environments [bib_ref] Peer worker involvement in low-threshold supervised consumption facilities in the context of..., Kennedy [/bib_ref]. As we have seen in a recent analysis of BC Take-Home Naloxone kit distribution data, PWUD are the group most likely to receive a kit because their previous kit was used to reverse an overdose; thus PWUD are key stakeholders, both as the people who will benefit from THN programs and policies and who save lives using THN. Other committee members offered valuable insights from clinical (with representation from harm reduction, nursing, pharmacy, and medicine), public health, and academic perspectives from different provinces and territories within Canada. Prior Delphi studies in harm reduction report including people from advocacy groups [bib_ref] Elements of community-based models for treating hepatitis C virus in supporting HCV..., Pourmarzi [/bib_ref] or advocates and people with lived and living experience [bib_ref] Effectiveness and implementability of state-level naloxone access policies: expert consensus from an..., Smart [/bib_ref]. The expertise of PWUD is invaluable in creating meaningful and impactful service delivery and they should have a central voice in programs and policies that affect them.
We found the online Delphi method to be especially beneficial in the context of a pan-Canadian project occurring during dual public health emergencies of drug toxicity and the COVID-19 pandemic. Public health and community health organizations reported feeling increasingly burdened trying to respond to the pandemic while concurrently seeing increases in overdose-related mortality [bib_ref] Changes in substance supply and use characteristics among people who use drugs..., Ali [/bib_ref] [bib_ref] Measuring the burden of opioid-related mortality in Ontario, Canada, during the COVID-19..., Gomes [/bib_ref]. The online Delphi method eliminated travel or time-consuming videoconferencing. Engagement of the committee members was flexible and safe during the COVID-19 pandemic and did not incur travel and meeting costs. Additionally, it did not require participants to leave their home community which can be difficult for those who use substances.

## Next steps
The Delphi method is the first step in community engagement for a Canadian THN guideline and will inform the next stage of the guideline development project. Three of the highly ranked key questions will be translated to research questions for systematic reviews and the evidence will be evaluated using the GRADE approach. Evidence from the systematic reviews will be used to inform recommendations on THN in Canada. These recommendations will be brought back to the Guideline Development Group for input based on their content expertise and knowledge of the Canadian harm reduction context.

# Limitations
Despite targeted recruitment, Newfoundland and Labrador, the Northwest Territories, Nunavut, and Prince Edward Island were not represented in the Guideline Development Group. Some of the nominees from these provinces and territories reported a higher than usual workload due to the COVID-19 pandemic as a barrier to participation. While two of the three territories were not represented, perspectives from the Yukon were included. Conclusions drawn through this Delphi method may not be generalizable to the provinces and territories not represented.
A key limitation is reliance on online meetings. While use of online meetings and data collection allowed us to engage people from across Canada, it could present barriers to engagement especially for those with limited access to technology or limited technological knowledge including PWLLE. Additionally, our project relied on participants' comfort with sharing ideas. While data collection was conducted using survey software, other meetings occurred over ZOOM video conferencing software. Prior qualitative research shows that while participants may feel able to express ideas during online focus groups, they also felt less engaged with other participants and less able to have a conversation that flowed compared to face-to-face discussions [bib_ref] Using online meeting software to facilitate geographically dispersed focus groups for health..., Matthews [/bib_ref]. We observed that using ice breakers and spending time on introductions helped group members share ideas more readily.
The research team drafted the guideline questions using Population/Problem, Intervention/Exposure, Comparison, Outcome/Findings (PICO) language, then reworded them into accessible language for the Delphi method. However, this created uncertainty with the research team around the interpretation of some of the prioritized questions. The team was unable to ascertain how individuals interpreted the common language questions, which led to challenges in creating the final PICO questions used in the literature searches in the absence of more extensive pilot testing. Capturing comments on the questions or asking the voters to explain their understanding of the questions may have ensured that voters all had similar interpretations. Additionally, key questions included multiple outcomes, but the research team needed to narrow down questions for feasibility of executing the systematic reviews that will generate evidence for the THN guidelines.

# Conclusion
The Delphi method allowed Guideline Development Group members from across disciplines, life experiences, and geographical space to share insights and help establish priorities for a Canadian guideline. A guideline on

## General opioid overdose response strategy
What is the effectiveness of Take-Home Naloxone programs? 1.6* 1.6* Are there different rates of mortality and morbidity for persons experiencing overdose in community setting associated with:
Rescue breathing in addition to naloxone administration Conventional CPR including rescue breathing in addition to naloxone administration Compression-only CPR in addition to naloxone administration Naloxone administration alone?

## Cost-effectiveness of program implementation
What is the cost-effectiveness of the intervention? 2.0 -What evidence exists of social and economic benefits of program implementation and overdose prevention and what are economic evaluations of costs and cost-benefit ratios from health care and societal perspectives?

## -
What is the relative cost-effectiveness between administration methods (nasal vs intramuscular)? 2.6 -What is the cost-effectiveness of funding of naloxone kits, including individual doses, assembly of kits, pharmacy dispensing fees, and training fees?

## -
What is the cost for implementation and running the naloxone distribution program? 2.4 -What are the costs for data collection and program monitoring, as well as researcher/agency capacity to evaluate distribution programs?

## -
Take-Home Naloxone informed by key stakeholders will provide evidence to support consistent, effective, and equitable service delivery across Canada. The evidence on the nine questions identified through this Delphi method study will be explored through systematic review.
Abbreviations PWLLE: People with lived and living experience of substance use; PWUD: People who use drugs; PICO: Population/problem, intervention/exposure, comparison, outcome/findings; THN: Take-Home Naloxone.

## Supplementary information
The online version contains supplementary material available at https:// doi. org/ 10. 1186/ s12954-022-00650-4.
Additional file 1.

## Acceptability of naloxone distribution program
What kind of naloxone distribution program do people impacted want? Is this impacted by PWUD demographics? 1.9* 1.8* What is the demand for and acceptability of naloxone distribution programs among people who use drugs, people who may witness an overdose, health professionals in community settings, the general public, government, and policy authorities?

## -
How is the need for naloxone distribution programs communicated to the general public and how effective is that communication?
[fig] 2: The Affected Community Committee (composed of seven members) provided guidance and recommendations based on the values and preferences of people with lived experience related to naloxone distribution and use in opioid overdose. 3. The Clinical Expert Committee (composed of 13 members) provided guidance and recommendations on the use of naloxone in opioid overdose from a clinical perspective. 4. The Guideline Development Panel (composed of 15 members) voted on the key questions to be addressed by the guideline as part of a Delphi method. Members of the Guideline Steering Committee, Affected Community Committee, and Clinical Expert Committee were invited to participate in voting as part of the Guideline Development Panel. [/fig]
[fig] Figure 1: reviewed for content, completeness, and readability by Guideline Steering Committee, Affected Community Committee, and Clinical Expert Committee The Guideline Development Panel voted on the importance of 35 key questions and provided insights on their rationale for each questions score The Guideline Development Panel were shown the other panelists' anonymized rationale on 17 of the key questions voted as most important in the first round of voting and asked to vote again in light of others' opinions A final score for each key question was generated Delphi method [/fig]
[fig] *: Scores under 2.0 which the guideline development panel identified as important to prioritize in the THN guideline a Where 1 = strongly agree and 5 = strongly disagree that the question is important b People who use drugs [/fig]
[table] Table 1: Key questions scored independently by Guideline Development Panel [/table]