JMIR FORMATIVE RESEARCH

Eldridge et al

Original Paper

Implementation Documentation and Process Assessment of thePharmNet Intervention: Observational Report
Lori Ann Eldridge1, PhD; Beth E Meyerson2,3, PhD; Jon Agley4, MPH, PhD
1

Department of Health Education and Promotion, College of Health and Human Performance, East Carolina University, Greenville, NC, United States

2

Harm Reduction Research Lab, Family and Community Medicine, College of Medicine-Tucson, University of Arizona, Tucson, AZ, United States

3

Comprehensive Center for Pain and Addiction, University of Arizona Health Sciences, University of Arizona, Tucson, AZ, United States

4

Prevention Insights, Department of Applied Health Science, School of Public Health Bloomington, Indiana University Bloomington, Bloomington,
IN, United States

Corresponding Author:
Jon Agley, MPH, PhD
Prevention Insights, Department of Applied Health Science
School of Public Health Bloomington
Indiana University Bloomington
809 E 9th Street
Bloomington, IN, 47405
United States
Phone: 1 812 855 3123
Email: jagley@indiana.edu
Related Article:
This is a corrected version. See correction statement in: https://formative.jmir.org/2024/1/e59427

Abstract
Background: The number of overdose deaths in the United States involving opioids continues to exceed 100,000 per year. Thishas precipitated ongoing declarations of a public health emergency. Harm reduction approaches, such as promoting awarenessof, ensuring access to, and fostering willingness to use naloxone to reverse opioid overdose, are a key component of a larger
national strategy to address the crisis. In addition, overdose reversal with naloxone directly and immediately saves lives. Becauseof pharmacies’ ubiquity and pharmacists’ extensive clinical training, community pharmacies are well-positioned, in principle, tofacilitate naloxone access and education.
Objective: In 2022, a single-site pilot study of PharmNet, a community pharmacy intervention incorporating naloxone distribution,awareness building, and referral, showed promising outcomes for both naloxone and resource distribution in the community. Asa next step, this study was intended to be a pilot randomized controlled trial of PharmNet in 7 pharmacies. However, due tocircumstances outside of the study team’s control, data collection was unable to be fully completed as planned. In keeping with
open research standards, we transparently report all available data from the study and discuss trial barriers and processes. We doso both to provide insights that may inform similar studies and to avoid the “file-drawer” (publication bias) problem, which canskew the aggregated scholarly literature through nonpublication of registered trial results or selective publication of findings
affirming authors’ hypotheses.
Methods: This paper reports an in-depth implementation study assessment, provides the available observational data, anddiscusses implementation considerations for similar studies in independent (eg, nonchain) community pharmacies.
Results: Retrospective assessment of study outcomes and fidelity data provided for robust discussion around how resource
differences in independent community pharmacies (vs well-resourced chain pharmacies), as well as high demands on staff, canaffect intervention implementation, even when leadership is highly supportive.
Conclusions: Community pharmacies, particularly independent community pharmacies, may require more support than anticipatedto be successful when implementing a new intervention into practice, even if it might affect estimates of real-world effectiveness.
Further implementation science research is needed specific to independent community pharmacies. All study elements are outlinedin the International Registered Report Identifier (IRRID) PRR1-10.2196/42373. Although this paper reports results associatedwith that registration, results and conclusions should not be given the weight assigned to findings from a preregistered study.
International Registered Report Identifier (IRRID): RR2-10.2196/42373
https://formative.jmir.org/2024/1/e54077

XSL• FO
RenderX

JMIR Form Res 2024 | vol. 8 | e54077 | p. 1
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Eldridge et al(JMIR Form Res 2024;8:e54077) doi: 10.2196/54077
KEYWORDS
naloxone; Narcan; pharmacy; harm reduction; PharmNet; overdose; opioids; implementation; pragmatic trial

Introduction
BackgroundIn the last decade, hundreds of thousands of lives have been
lost in the United States due to fatal opioid-involved overdoses[1], and annual overdose death rates (computed as 12-monthyear-over-year incidence) have remained over 100,000 in recent
reports from the Centers for Disease Control and Prevention[2]. Additionally, overdose deaths represent over one-third of
all accidental deaths in the United States [3], with the majority
involving illicitly manufactured fentanyl, a synthetic opioid [4].
In the state of Indiana, where this study was conducted,
preliminary data indicate 2250 overdose deaths in 2022 [5],around 71% of which involved fentanyl or other synthetic
opioids [5].
Naloxone, an opioid antagonist, works by blocking the effects
of opioids and can allow persons who have overdosed to resume
normal breathing [6]. It is a safe and effective way to rapidly
reverse the effects of opioid overdose and can prevent death[6,7]. Research has shown that overdose deaths decline when
overdose education and naloxone distribution (OEND) efforts
are undertaken in communities [8], and that OEND programs
are both an effective [9,10] and cost-effective [11] strategy toaddress overdose fatality.
In 2018, the US Surgeon General issued an official
recommendation that all US citizens carry naloxone [12]. By
2021, most US states, including Indiana, had begun to address
the overdose epidemic using combinations of policies thatgenerally included “Good Samaritan” laws and standing orders
for naloxone at pharmacies [13]. More recently, the declarationof an opioid public health emergency was renewed in September
2023 [14], approximately a year after the release of the US
National Drug Control Strategy, which outlines goals that
include the expansion of access to evidence-based harm
reduction practices, including naloxone [15].
Beyond the impact in terms of lives saved, individuals who arerevived with naloxone and who also have opioid use disorder(OUD) have an opportunity to engage with an “Opioid Cascadeof Care” [16], such as through treatment initiation withmedications for opioid use disorder (MOUD) in the emergency
department [17]. For these benefits to accrue, however, naloxone
must be easily accessible by those who can use it to reverse an
opioid overdose. In the United States, nasal formulations of
naloxone were recently approved for over-the-counter sale [18],following a unanimous vote by Food and Drug Administration
advisors [19]. While this change might in principle increase thenumber of people carrying naloxone, data from Australia suggestthat it may be a helpful but not sufficient step to increase access[20].
Even as additional means of obtaining naloxone are developed(eg, naloxone vending machines [21]), community pharmacies
will likely remain an important component of supporting
https://formative.jmir.org/2024/1/e54077

XSL• FO
RenderX

naloxone access and education. In some populations, patients
interact with their pharmacists more often than with theirprimary care physicians [22], and pharmacists have been
identified as some of the most accessible health care providers
in the United States [23,24]. Studies show that community
pharmacists generally acknowledge their role in assisting inharm reduction [25-27], and an Arizona study found that
community pharmacists were typically comfortable providing
consultation around and dispensing naloxone, even when they
were uncomfortable with other harm reduction approaches suchas syringe dispensing without a prescription [28]. As such,
community pharmacists are theoretically well-positioned to
improve harm reduction education and accessibility of related
supplies. At the same time, recent studies also continue tosuggest that naloxone remains unavailable for purchase in somecommunity pharmacies and may be less likely to be available
in independent pharmacies compared to chain pharmacies [29].

The PharmNet Intervention
To better understand the dynamics and potential of community
pharmacy harm reduction practice, we completed multiple
preliminary studies [26,28,30-35]. Our work examined naloxone
access, pharmacist comfort with harm reduction,
pharmacy-based research approaches, and the “fit” of thepharmacy environment for harm reduction–related services.
The studies were conducted from 2016 through 2022, and manyof them engaged community and academic pharmacists as partof the research team.
By the end of the process, we had determined that our initialconceptualization of a “short” or “minimally demanding”
intervention (eg, the first draft of PharmNet, a 10- to 15-minutescreening and brief intervention-style intervention) would likely
be perceived as useful but not necessarily feasible without asubstantive added support system (eg, financial and personnel).
Further, there was variability in the a priori intervention
components that would be available at different pharmacy types.
For example, a systematic review [36] in addition to our own
earlier research in Indiana [33] and a recent study in North
Carolina [29] identified independent pharmacies as being
significantly less likely to stock or dispense naloxone compared
to chain pharmacies (eg, ability to dispense naloxone, even atcost, could not be assumed). While independent pharmacies
are, in theory, more likely to be able to participate in a smallintervention implementation program (eg, they do not need toseek approval from a large bureaucracy associated with anational or international chain), they also lack the financial andsystemic resources that are available to chain pharmacies, which
can have implications for stocking and dispensing of products,
as well as pharmacists’ availability to undertake additional work.
Thus, the PharmNet intervention was modified with the goal
“to study procedures that have as minimal an impact as possible
on pharmacy costs and operational functioning while maximallyfacilitating harm reduction from opioid overdose—in other
JMIR Form Res 2024 | vol. 8 | e54077 | p. 2
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH
words, to find an optimal intersection point of those concerns”[37]. The overall goals of the finalized PharmNet intervention
are outlined in a previous paper, which reported the results of
our initial single-site pilot study [37]. For clarity, we excerpt
them here verbatim:
1.

2.

3.

4.

5.

Building awareness of naloxone availability at the site
among patients (eg, the use of yard signs and scrolling
messages on television screens).
Supporting awareness among pharmacists and pharmacy
technicians about proactively offering naloxone (eg,customized post-it notes).
Facilitating service provision by conducting a priori
negotiations and establishing written agreements with local
nonprofits to facilitate a pipeline of no-cost naloxone forthe pharmacy.
Emphasizing bidirectional naloxone provision (eg,pharmacist- and pharmacy technician–initiated offers in
addition to patient-initiated requests), andFacilitating referral by providing a physical, durable, andcurated list of community resources that can be used bypharmacists, handed to patients, and placed into
pharmaceutical bags.

Interestingly, these goals are generally aligned with theintervention approaches described in a scoping review of opioid
counseling and naloxone services published after our study
concluded [38]. The review found that “all studies incorporated
naloxone recommendations into their pharmacy-based opioid
counseling and naloxone services,” and that most programs
focused mostly on OEND [38]. Further, most of our awarenessand communication strategies were similar to those in thereview, though only one study in the review used outdoor yard
signs, and none appear to have established pathways forpharmacies to receive no-cost naloxone [38].
Our first implementation of PharmNet was in a single
independent pharmacy in the Midwestern United States. In that
3-month single-site pilot study, naloxone dispensing by anymeans (paid or no cost) nearly doubled, increasing by 96.48%(+9.33 doses per month); this included both distribution of
no-cost doses (+6 doses per month) and an increase in themonthly rate of naloxone sold (+3.33 doses per month)compared to the 3 months before the intervention [37]. Anaverage of 2.85 referral and resource cards (1/4 page in size)
customized to the local area were distributed to patients eachday. The initial pilot study also allowed us to further adjust theprocedures based on feedback from community pharmacists[37]. The most substantive change was a drop from 3 sets ofdifferent post-it reminders to one, because only one type wasregularly used by the pharmacy staff members in practice.
To better understand the results, and to begin assessing theintervention’s scalability, we then developed and published astudy protocol for a pilot randomized controlled trial (RCT) of
PharmNet [39]. Our stated goal was “to facilitate an improved
understanding of whether it is reasonable to believe that thePharmNet intervention causes increased dispensing of naloxone...” and we proposed to test the hypothesis that, “Monthly
naloxone dispensing (combined sales and no-cost distribution)

https://formative.jmir.org/2024/1/e54077

XSL• FO
RenderX

Eldridge et al
will be significantly increased in the pharmacies implementing
PharmNet compared to those in the control arm” [39].

This Report
This paper was originally planned to describe the results froma small RCT of PharmNet based on our protocol paper that also
served as a registered report (International Registered Report
Identifier: PRR1-10.2196/42373) [39]. However, several eventsand circumstances beyond the control of the study team (eg, a
COVID-19 outbreak among pharmacists) resulted in substantial
deviations from the proposed approach and outcome measures.
The conceptual magnitude of these differences and the resultant
effects on data and procedures mean that it would beinappropriate to attempt to assert that the information reported
in this paper serves as answers to our original research questions.
At the same time, it is important for us to provide a complete
description of the study and any resultant observations andlessons learned. In doing so, we avoid contributing to the “filedrawer” (publication bias) problem, wherein studies that areperceived to be unsuccessful or that have nonsignificant results
are never published to the detriment of the scientific enterpriseand those affected by it [40-42]. Publication bias has been
described as being conceptually similar to the practice of
p-hacking [43]; it can skew the perception of a field of study ina particular direction through suppression (either deliberate or
inadvertent) of results that do not align with the scientists’ goals.
This can happen in at least 2 ways. First, results from registered
trials may never be published at all (a recent study of RCTs in
New Zealand found that 27% of results had not been publishedmore than a decade after the study registration date) [44]. Thiscan produce the mistaken impression that not much is known
about a particular approach. Second, a “failed” study may berepeated with some minor modifications, and the resultant datareported as though it was the originally registered trial. Thiscan lead to overestimation of the intervention’s likely outcomesas well as reducing the likelihood that procedural andimplementation considerations—some of which may be
important—are fully and transparently addressed.
Therefore, we share the results of this study in the form of anobservational report that includes (1) the findings that we wereable to collect, (2) the methods to collect that information, as
well as (3) specific information related to deviations fromplanned data collection and analysis. Then, we provide a detailed
written record of fidelity or process checks and the results of
those checks. Finally, the implications of the external issues
leading to these events, including discourse around the role that
implementation fidelity checks may play in the conduct ofintervention studies, and of the unique nature of independent
pharmacies, are discussed.

Methods
Study Design
We planned to conduct a pilot cluster RCT with 7 independentcommunity pharmacies consisting of 2 parallel groups. In total,
4 pharmacies were allocated to the intervention arm, and 3
pharmacies were allocated to the control arm. The original
project flow diagram is provided as Figure 1 [39,45], along withJMIR Form Res 2024 | vol. 8 | e54077 | p. 3
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Eldridge et alcolored marks added after the study to indicate the degree ofcompletion for that step.

this timeframe are provided subsequently). Some preintervention
staff data are provided and discussed.

Detailed implementation notes were collected from the 4
pharmacies randomly assigned to the intervention arm. We also
obtained data on naloxone sales and no-cost distribution for the
3 months prior to the study period as well as for a period of 3
or more months after the intervention began (more details on

No data were collected from control pharmacies, no posttest
data were able to be collected from pharmacy staff members,
and no counts of referral cards were able to be obtained fromany pharmacy.

Figure 1. Original study protocol diagram for the PharmNet pilot trial, modified with indicators of completion status (adapted from Eldridge et al [39],which is published under the terms of the Creative Commons Attribution License 4.0 [45]).

•

Sources of the Data
There were 2 types of participants in this study, pharmacies
and pharmacy staff members.
•

The 7 participating pharmacies were part of a single
independent pharmacy organization. They were allocatedsimultaneously to a study arm using a web-based random
sequence generator with a 4:3 ratio.
•
These pharmacies all operate in the Midwestern UnitedStates and have a shared leadership structure (eg, there
are individuals who oversee operations at all 7
pharmacies). In addition, each pharmacy has amanaging pharmacist (which we call “lead pharmacists”
in this paper to avoid conflating their management rolewith those of pharmacists overseeing the wholeorganization).

https://formative.jmir.org/2024/1/e54077

XSL• FO
RenderX•

Lead pharmacists manage day-to-day pharmacy
operations at single sites.

Pharmacists and pharmacy staff members at each of the 7sites (both the intervention and control arms) were asked
to participate in the study by providing individual-level
data at pretest. Posttest data collection was planned but notcompleted (as described in the Results section).

Intervention Description
The development of this intervention was described in detail inprior papers [37,39], and a structured table outlining thesequential steps of PharmNet was included in the publishedprotocol paper [39]. The specific steps involved in theintervention are included in Tables 1 and 2, which describe eachplanned step (excerpted verbatim or paraphrased from theprotocol paper, as appropriate, for clarity) as well as the actual
outcome.

JMIR Form Res 2024 | vol. 8 | e54077 | p. 4
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Eldridge et al

Table 1. Planned implementation steps and observed outcomes prior to the PharmNet study start date.
Intervention stage and planned steps

Outcomes

Introduction to the study (prior to the start date)
1. Digital video (.mp4) describing the intervention is distributed tothe pharmacy manager.

An email was sent to the pharmacy organization management team, andthey responded with an indication that they had received the video.

2. Pharmacy manager disseminates the video.

The managing pharmacist reported they would show it to all 4 intervention
pharmacies prior to the start date at their monthly employee meetings.

3. All staff members view the video.

As far as we can determine, staff members viewed the video (based onphone conversations).

4. Lead pharmacists are asked by the managing pharmacist to include We were unable to verify whether this occurred.
the video in the orientation package for new pharmacists and pharmacy
technicians who missed the initial meeting.
Early implementation (prior to the start date)
1. Confirm with the managing pharmacist that all intervention pharmacies have viewed the video at the “all hands” staff meeting.

LAE spoke with the managing pharmacist on the phone on the start dateto confirm this.

2. Deliver the following supplies for each participating pharmacy:

JA verified all materials and personally delivered them to organizational
management approximately 1 month prior to the start date. On the startdate, LAE verified with the managing pharmacist that all 4 intervention
pharmacies had received the supplies and that the intervention pharmacies
were “live” with the intervention as of November 1, 2022.

•
•
•
•

A set of 30 doses of naloxone (individually packaged Narcan
nasal spray) to each pharmacy in numerically marked bags imprinted with “Not for Sale.”
Deliver a set of 500 consecutively numbered harm reduction referral slips to each pharmacy (customized for each pharmacy
location).
Deliver sets of reminder post-it notes to pharmacies.
Deliver the requested number of yard signs to each pharmacy.

3. Ensure the pharmacy’s in-store television has access to the scrolling In principle, this was made available. As we note below, we cannot verify
reminder text.
that it was used.

https://formative.jmir.org/2024/1/e54077

XSL• FO
RenderX

JMIR Form Res 2024 | vol. 8 | e54077 | p. 5
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Eldridge et al

Table 2. Planned implementation steps and observed outcomes concurrent with the PharmNet study start date.
Intervention stage and planned steps

Outcomes

Launch (on the start date)
1. Verify the intervention has launched.
•
•
•
•

•

Reminder post-its are placed in visible areas on the interior ofthe dispensing location (eg, staff side rather than patient side). •Harm reduction referral cards are placed at each checkout foreasy access.
Yard signs are placed outside the building in visible locations•
where traffic (vehicle or foot) is common.
Scrolling television banner will be enabled to run continuously
through the intervention, reading “76.7% of overdoses happen
at home. Bystanders who witness an overdose can be effective
in reducing overdose mortality. Ask us about how you can savea life with naloxone today!”••

2. Obtain additional details about the launch.

•

•

Data
The primary data points that we report include
preimplementation naloxone sale data from intervention
pharmacies (August 1, 2022, through October 31, 2022),
postimplementation naloxone sale data from the same
pharmacies (November 1, 2022, through January 31, 2023), anddoses of no-cost naloxone distributed between the receipt ofthe doses at the end of May 2023. These data were obtained
through objective pharmacy record extraction and physical(manual) counts of remaining naloxone doses by the leadpharmacist at each intervention pharmacy.
Data from individual staff members were collected via a survey
administered prior to the implementation start date for
PharmNet. The questionnaire was built in Qualtrics (Qualtrics),
a secure web-based survey and data collection tool. The
organizational pharmacy manager was provided with a link to
distribute to pharmacists and pharmacy technicians at all
independent community pharmacies in the study (both theintervention and control pharmacies). Measures included
sociodemographic information, questions about pharmacy
practice, comfort with harm reduction practices, and belief-based
measures about people who use drugs. The wording for eachhttps://formative.jmir.org/2024/1/e54077

XSL• FO
RenderX

Launch verification calls were made by LAE to the intervention
pharmacies on November 2, 2022 (the day after launch).
Pharmacy 1: The pharmacist who was working that day was a floating
pharmacist and transferred LAE to the managing pharmacist. The
managing pharmacist reported that the pharmacy “has the supplies
and is ready to go.”Pharmacy 2: The pharmacist who was working that day said they
had just returned from leave and did not know where the PharmNet
intervention materials were in the pharmacy. They reported theywould contact the managing pharmacist to find out where the materials were. The pharmacist stated they had watched the training video
and were aware of the PharmNet intervention.
Pharmacy 3: The pharmacist who was working that day reported they
were not aware of receiving the PharmNet materials. They put LAE
on hold to check if the supplies were in the pharmacy. They reported
to LAE that they could not locate any supplies.
Pharmacy 4: The pharmacist who was working that day reported thatthey had watched the training video and were aware of the PharmNet
intervention. They reported that the pharmacy had not received any
PharmNet supplies.
On day 4 of the intervention, LAE called the organizational management team. The primary purpose of the call was to request that they
contact intervention pharmacies 2-4 to inform the pharmacists where
the PharmNet materials were. LAE asked the managing pharmacist
to inform the lead pharmacists at the intervention pharmacies to begin
PharmNet if they had not already done so.
The managing pharmacist informed the researcher that they would
contact the pharmacists over the next few days. Unfortunately, soonafter that call, we learned that the managing pharmacist had COVID19 and was on sick leave. In addition, at least 1 additional leadpharmacist was also on sick leave for COVID-19, and pharmacists
from other sites, and (when possible) the managing pharmacist, were
often required to staff those roles when no additional staff members
were available to do so.

item is provided in Multimedia Appendix 1. Completion of thesurvey was specifically not required for any individual
pharmacist or staff member, but they were encouraged toparticipate. A copy of the full survey instrument is available as
an appendix to the published protocol [39].
Finally, we also originally intended to have a single in-person
fidelity check for each intervention pharmacy around 1 month
into the project’s launch. For the reasons we describe in the
Results section, multiple additional checks were added, and themodality was switched to telephonic and email correspondence
to facilitate the increased frequency.

Ethical Considerations
This study received ethics reviews prior to being initiated.
Participation by pharmacies was designated as not humansubjects research by the Indiana University institutional review
board (document 12339). Participating pharmacy sites in theintervention arm were each offered an institutional incentive of
US $1000 for their participation. Pharmacy employees
completed a study information sheet prior to participating in
the individual-level data collection and were compensated witha US $5 digital gift card for completing the pretest survey.
Individual-level procedures were reviewed separately by the
JMIR Form Res 2024 | vol. 8 | e54077 | p. 6
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Eldridge et al

Indiana University institutional review board and received adesignation of “exempt” (document 13956). Due to the small
number of individual participants, data are reported only inaggregate and without cross-sectional tabulation.

Results
Pharmacy-Level Outcomes
On February 1, 2023, the intervention was concluded. From
February through the first week of April 2023, we attempted toobtain pharmacy-level outcome data via phone calls, voicemails,
in-person visits, and physical notes. However, these processes
were unsuccessful. Our subjective observation is that a
combination of illness and staffing complexity made it extremely

difficult to collect the outcome data as originally planned in the
protocol.
Although they were not originally involved in data collection,
we were able to collaborate with the lead pharmacists atintervention pharmacies, who had participated in fidelity
checking processes and were familiar with our team, to obtainthe limited data set described in the Data section in May 2023.
Importantly, we do not know the exact date when each pharmacy
began offering no-cost naloxone doses, and the numbers ofdoses of no-cost naloxone dispensed were obtained several
months after the intervention technically ended. As a result, wecannot compare naloxone distribution prior to and during theintervention period. However, we can still provide raw numbers(Table 3).

Table 3. Pharmacy-level naloxone sale and dispensing data, August 2022 through May 2023.
Pharmacy

Preimplementation naloxone sale data
(August 1, 2022-October 31, 2022)

Implementation naloxone sale data
(November 1, 2022-January 31, 2023)

Doses of no-cost naloxone dispensed (of 30) as of May 2023

Pharmacy 1

19

23

12

Pharmacy 2

45

19

30

Pharmacy 3

9

3

28

Pharmacy 4

11

10

24

Pharmacy Staff Preintervention Data
A total of 8 staff members at least partly completed the study
pretest during the period prior to the intervention, including 3
pharmacists and 5 pharmacy technicians. Pharmacy 4 waswell-represented with 5 respondents, and there were 2 frompharmacy 3 and 1 from pharmacy 1. No staff members frompharmacy 2 or the control pharmacies participated.
All but 1 respondent who provided demographics (n=6) werefemale, and all were White, non-Hispanic, and heterosexual.
Staff ages ranged from 32 to 52 years (n=5). Staff members
who responded were comfortable with most harm reduction
approaches, but fewer were comfortable with consulting withpatients about pre-exposure prophylaxis for HIV prevention ordispensing syringes for nonprescription drug use. No participants
indicated they were uncomfortable dispensing naloxone foroverdose reversal or making referrals to community services.

https://formative.jmir.org/2024/1/e54077

XSL• FO
RenderX

Further, staff member attitudes toward patients who use drugs
or have OUD were not typically stigmatizing, though they
expected that others (“most people”) might blame persons with
OUD or believe that they are dangerous. A table of these data
is provided in Multimedia Appendix 1.

Fidelity Data
We also collected a full written accounting of the 9 distinct
fidelity checkpoints during the intervention period. This
information helped to contextualize some of the implementation
issues around which we provide interpretation and discussion
in the next section. Although not all study processes were inplace at the time of the final fidelity check at week 10 of the
12-week intervention, by the time we collected outcome data,
the intervention was apparently functioning at all sites (eventhough the technical “intervention period” was over). All
documentation is available in Tables 4 and 5.

JMIR Form Res 2024 | vol. 8 | e54077 | p. 7
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Eldridge et al

Table 4. Implementation fidelity checkpoint documentation for the PharmNet study through day 22.
Timeframe

OutcomeDay 10—Calls to all inter- •vention pharmacies•••

Pharmacy 1: The pharmacist who answered the phone reported that they were not the usual pharmacist and only work
1 day a week, but that the pharmacy did have the yard signs and the no-cost naloxone. However, they reported thatthe yard signs were not placed outside the store. They also indicated that the managing pharmacist was on leave andthat they recommend calling when they returned the following week to follow-up about the yard signs.
Pharmacy 2: The pharmacist reported they were not the usual pharmacist and only work at the pharmacy 2 times amonth, and that the regular pharmacist who works at that store would be back the following week. They did not know
about PharmNet nor about the no-cost naloxone, yard signs, or harm reduction referral sheets.
Pharmacy 3: LAE spoke with the same pharmacist as previously (the startup verification). The pharmacist reported
that they did not think they had received PharmNet supplies, but that they were ready to start the intervention as soon
as they received the supplies.
Pharmacy 4: We spoke with the same pharmacist as previously (the startup verification). The pharmacist reported
that they did not think they had received PharmNet supplies.

Day 14—Attempt to coor- •dinate with the pharmacies

LAE attempted to reach the managing pharmacist at 2 separate pharmacies. A pharmacy technician called an additional 5 pharmacies but was unable to locate them. We left a voicemail at their office. We later learned that they wereon leave during this period.

Days 16, 17, and•
21—Follow-up calls, after which we learned
about illnesses or leaves
among the pharmacists

We called the original pilot pharmacy attempting to contact the leadership team. LAE left a voicemail and sent theman email on each of the 3 days.

Day 22—Calls to all inter- •vention pharmacies

LAE learned that the managing pharmacist had returned to the office, but they were in a meeting when we called. A
pharmacy technician took a message. Calls were also made to each of the pharmacies in the intervention arm.

•

Pharmacy 1: When LAE called and requested to speak to a pharmacist, the pharmacy technician transferred her tothe original pilot pharmacy location (which was not part of the study). LAE spoke with a pharmacist at this location,and this pharmacist reported they were unaware of the status of PharmNet at pharmacy 1 as they never worked atthat site. They transferred the researcher back to pharmacy 1, where the pharmacy technician forwarded the researcher
to voicemail. LAE left a message.
Pharmacy 2: We spoke with a pharmacist who confirmed that the supplies had been received and that the intervention
had begun.
Pharmacy 3: The pharmacist reported that they still were not aware of having received the PharmNet supplies.
Pharmacy 4: The pharmacist reported that they still had not received the PharmNet supplies. The pharmacist reported
they had reached out to their leadership about the supplies and were informed that their supplies were located at
pharmacy 1 and would be delivered after Thanksgiving.

•••https://formative.jmir.org/2024/1/e54077

XSL• FO
RenderX

JMIR Form Res 2024 | vol. 8 | e54077 | p. 8
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Eldridge et al

Table 5. Implementation fidelity checkpoint documentation for the PharmNet study, week 4 through week 10.
Timeframe

OutcomeRemainder of week•
4—Additional verification calls to intervention
pharmacies to determine
status and if any addition- •al supplies were needed•••
Week 6—Additional•verification calls to intervention pharmacies todetermine status and if•
any additional supplies
were needed•••

Week 10—Additional•verification calls to intervention pharmacies todetermine status and if•
any additional supplies
were needed•••

LAE made calls to each of the pharmacies in the intervention arm (1) to determine if pharmacy 1 and 2 needed supplies
and were following PharmNet protocols and (2) to assess if supplies were received and that PharmNet started at
pharmacy 3 and 4.
Pharmacy 1: We spoke to an on-call pharmacist who confirmed that they were following PharmNet’s protocol andthat they had enough supplies (eg, harm reduction referral cards and no-cost naloxone).
Pharmacy 2: The pharmacist confirmed that they were following PharmNet’s protocol and that they had enoughsupplies (eg, harm reduction referral cards and naloxone).
Pharmacy 3: The pharmacist reported that they still were not aware of having received the PharmNet supplies.
Pharmacy 4: The pharmacist reported that they still were not aware of having received the PharmNet supplies.
LAE made calls to each of the pharmacies in the intervention arm (1) to determine if pharmacy 1 and 2 needed supplies
and were following PharmNet protocols and (2) to assess if supplies were received and that PharmNet started at
pharmacy 3 and 4.
Pharmacy 1: The pharmacist confirmed that they were following PharmNet’s protocol and that they had enoughsupplies (eg, harm reduction referral cards and naloxone).
Pharmacy 2: The pharmacist confirmed that they were following PharmNet’s protocol and that they had enoughsupplies (eg, harm reduction referral cards and naloxone).
Pharmacy 3: The pharmacist reported that they still were not aware of having received the PharmNet supplies.
Pharmacy 4: The pharmacist reported that they had not received all of the PharmNet supplies, but they had received
the referral cards in the mail.
LAE made calls to each of the pharmacies in the intervention arm (1) to determine if pharmacy 1 and 2 needed supplies
and were following PharmNet protocols, (2) to assess if supplies were received and if PharmNet had started at pharmacy 3, and (3) to check if pharmacy 4 received the remaining supplies and started the intervention.
Pharmacy 1: The pharmacist confirmed that they were following PharmNet’s protocol and that they had enoughsupplies (eg, harm reduction referral cards and naloxone).
Pharmacy 2: The pharmacist confirmed that they were following PharmNet’s and that they had enough supplies (eg,harm reduction referral cards and naloxone).
Pharmacy 3: There was no pharmacist available. We left a message requesting a callback, but it was never received.
Pharmacy 4: The pharmacist reported that they had not received all of the PharmNet supplies, but that they werecurrently using the referral cards.

Discussion
Context of the Report
PharmNet was developed with independent community
pharmacies in mind as a core implementation venue. This waspartly because of trends around naloxone stocking anddispensing, as described in the Background section, whichsuggested that independent pharmacies might especially benefit
from intervention support. In addition, there is a large segmentof the US population, typically older, rural, and with lower
income levels, for whom independent pharmacies are theprimary point of pharmacy access (recent estimates place thenumber of individuals at over 15 million) [46]. Thus, it isparticularly important to know whether interventions can operate
effectively in independent pharmacy venues.
As we have previously indicated, our single-site pilot, which
preceded this study, was successful [37], whereas this study
encountered multiple barriers, even though they were conductedwithin the same small, independent pharmacy group. Our
subjective qualitative assessment is that these weremeta-structural barriers—perhaps common to independent
pharmacies or related to the implementation decisions—and
not a function of the individuals in the independent pharmacy
group. Further, many of the barriers we encountered did notappear to be especially related to the program’s focal area (ie,naloxone distribution).
https://formative.jmir.org/2024/1/e54077

XSL• FO
RenderX

Barriers Specific to Pharmacy Naloxone Distribution
The research literature has identified several barriers that canoccur when implementing pharmacy-based OEND programs,
but interestingly, this study of the PharmNet intervention neverencountered most of these barriers (though it encounteredothers). In some cases, this may have been due to theintervention’s design, as we spent considerable time in thedevelopment phase determining how we might obviatefrequently discussed implementation issues. For example,
multiple publications have identified that reimbursement for
products and services can be an implementation barrier (eg, forthe costs of naloxone, pharmacist time, or other program
features) [38,47]. In certain program structures (eg, where freeor reduced cost naloxone doses are unavailable), patients’
out-of-pocket costs to purchase naloxone can also be a concern[48]. However, PharmNet included a planning step where theresearch team negotiated with nonprofit organizations to procuresources of no-cost naloxone. This meant that pharmacies could
continue selling naloxone where appropriate, but that they also
had the ability to provide free doses when indicated (eg, topatients who could not afford it) without needing to find a sourceof reimbursement.
Similarly, while pharmacy-based counseling is sometimes
included in pharmacy OEND interventions, we were concerned
about its feasibility in low-resource pharmacies, especially after
COVID-19. Thus, for PharmNet, we opted to provide referral
JMIR Form Res 2024 | vol. 8 | e54077 | p. 9
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH
and resource cards in prescription bags that were tailored toeach unique geographic location. In doing so, we attempted to
avoid common barriers reported by similar studies such as lackof time or space for counseling [48]. Another larger study in 2
pharmacy chains also reported adaptations resulting fromCOVID-19, such as “streamlined counseling and drive-thru
provision adaptations ...” [49].
Further, while a study of opioid overdose prevention programsin pharmacies identified barriers related to pharmacy leadershipsupport [48], our perception is that the leadership team andpharmacy management in place at this independent pharmacy
group were incredibly motivated, competent, and willing tosupport this intervention to the best of their ability—and it
appeared that they were primarily limited by factors of raw
capacity. In addition, though our pharmacist and pharmacy
technician data were not collected from all intervention
pharmacies, the limited data set that we procured did not identify
any substantive concerns around comfort with dispensing
naloxone.

Barriers Related to Independent Pharmacies as
Scalable Intervention Sites
We intentionally designed this pilot trial to be pragmatic,
meaning that the implementation environment would mirror
real-world practice [39]. Our primary concern was that theresources provided as part of an intensive trial support system
would introduce a kind of heterogeneity of treatment effect [50],where pharmacies working to conduct the intervention outsideof the context of the RCT would be systematically different byvirtue of not having the same startup support. This would
potentially inflate the effect size of the intervention within the
study relative to its true effectiveness. Therefore, for individuals
working in single pharmacies (eg, lead pharmacists, pharmacy
technicians, and other staff members), we did not plan any direct
face-to-face interaction with our team. Instead, the intervention
materials themselves—introductory videos, instructions to theoverall leadership of the organization, and materials, along witha planned fidelity check—were the primary means ofdissemination.
However, the pharmacy practice environment is also
experiencing a variety of substantive external pressures. For
example, a 2022 survey from the National Community
Pharmacists Association found nearly one-quarter of responding
pharmacy owners or managers had difficulty filling pharmacist
roles, and 88.8% had similar difficulty filling pharmacy
technician positions [51]. A 2019 survey of full-time pharmacists
found that even though independent pharmacists had higher job
satisfaction, organizational commitment, and perceived control
in the work environment than chain, supermarket, or
mass-merchandiser pharmacies, those numbers had still declined
over the past 5 years [52]. Strains on the broader pharmacy
working environment have recently appeared in media reports
as well [53]. Understanding these pressures and accounting forthem with pragmatic intervention design will likely be importantto sustainable implementation success.
We suspect that these or similar external stressors played ameaningful role in the outcomes of this trial. Lessons fromremote decentralized clinical trials, for example, highlight how
https://formative.jmir.org/2024/1/e54077

XSL• FO
RenderX

Eldridge et al
reducing the research burden on some elements of a team orpartnership may inadvertently shift the burden onto others
working in the field [54]. In other words, eventually, someone
must do the work, and when staffing levels are low andexogenous stressors (eg, a COVID-19 outbreak) occur, an
unreasonably large volume of responsibility may accrue to asmall number of people. This is a particular concern due to theprevalence of pharmacist burnout, which a systematic review
recently estimated to exceed 50% [55], though a slightly older
review reported a lower mean percentage [56] (both reviews
indicated heterogeneity among included studies). In this case,while we do not have (nor are intending to imply) any direct
evidence of burnout, per se, we noted that there was not a “deepbench” of staff and that when illnesses or similar issues occurredin these independent pharmacies, the work was often coveredby others who already had full-time (or greater) workloads.
In contrast, in one well-conducted, harm reduction–focused,pragmatic RCT of 175 community chain pharmacies, theresearchers encountered both COVID-19 and wildfires [57].
Their study team conducted the majority of their fidelity checks
face-to-face and had regular phone contact with the pharmacies.
Their published protocol states the research team maintained a“close collaboration and regular meetings with the pharmacy
corporate leadership.” Unexpected circumstances that arose
were sometimes able to be mitigated by pharmacy corporate
leadership (eg, the provision of additional pharmacists to relieve
pharmacists from their regular work duties so that the onsite“academic detailing” could be conducted). These types ofresources and supports may not be accessible to mostindependent pharmacies.

Do Fidelity Checks Contribute to Intervention
Effectiveness?
The major differences between our single-site pilot study (which
was completed successfully and which produced positive
outcome data) [37] and this follow-up study remain of interest
to our team. One of our working theories relates to the natureof implementation fidelity checking that occurred.
Implementation fidelity checking has long been understood as
an important component of implementation science [58], thoughcommunity pharmacy intervention studies do not always report
implementation fidelity tools or processes [59].
Our single-site pilot study incorporated face-to-face fidelity
checks on a regular basis [37]. This involved a study investigator
regularly visiting the pharmacy in person, speaking with staff
members and pharmacists, obtaining interim information, andidentifying necessary points of action. In designing thisfollow-up study, we did not consider the effects that regularin-person fidelity checking might have had on intervention
effectiveness in our single-site pilot. In other words, we do notknow how much of the implementation effectiveness in theoriginal study was a direct or indirect result of the routine
physical presence of a researcher in the pharmacy. It is notnecessarily the case that a single-site pilot is an insufficient firststep to prepare for a larger-scale study, but rather, it seems that
researchers might benefit from carefully examining the ways
in which intensive fidelity procedures may actively contribute

JMIR Form Res 2024 | vol. 8 | e54077 | p. 10(page number not for citation purposes)

JMIR FORMATIVE RESEARCH
to successful implementation rather than merely documenting
what happened.

Limitations
As noted throughout this paper, there were numerous limitations
that resulted in our reporting of fidelity encounter data andlimited outcome measures. Even though this study waspreregistered, due to the differences in the data collected fromwhat was planned, we could not test the planned hypotheses.
This study should therefore be considered exploratory. In
addition, while we carefully consider the context of the study
in our Discussion section, the ideas presented therein are
hypotheses only and should not be interpreted as definitive
statements of fact.

Conclusions
Independent pharmacies, especially in rural areas, will likely
continue to be key parts of US health care teams. For example,
in April 2023, the Express Scripts business of Evernorth, asubsidiary of The Cigna Group, announced that they areextending their efforts to expand health care in ruralcommunities via independent pharmacies [60]. As theprevalence of fatal opioid overdose remains high, multiple
avenues to prevent fatality are warranted. Even though nasal
formulations of naloxone are now available over the counter,

Eldridge et alpharmacies have a role in creating awareness and fostering
accessibility for patients for whom the cost is prohibitive. As
one peer reviewer (Steven H Linder MD) aptly noted, “theoptimal means of naloxone community distribution remains to
be determined.”
In terms of broader program implementation considerations forpharmacies, it may be the case—especially for independent
pharmacies—that the start of a new initiative needs to beaccompanied by more substantive support in the current
pharmacy practice environment. If a trial is conducted over alonger period, support could in theory be withdrawn, andeffectiveness measured longitudinally. However, we could not,with the data obtained from this study, disentangle issues aroundfidelity checking and externalities from those related to theoverall pharmacist workload and burden and the intervention
mechanics.
The strength of this paper is its detailed process description andtransparent disclosure of the issues that affected the intervention.
Given the likely importance of independent pharmacies to thenational overdose response effort, we encourage further
implementation science research that is specific to independentcommunity pharmacies, which are often collapsed with othertypes of pharmacies but that have unique facilitators and barriers
to intervention uptake.

Acknowledgments
This paper was funded by the Indiana University Grand Challenge: Responding to the Addictions Crisis. The funder did not havea role in the preparation of this paper, and the opinions and information expressed herein do not necessarily reflect the views or
opinions of the funder.

Data Availability
All data generated or analyzed during this study are included in this published paper (Multimedia Appendix 1), except for the
raw data for the small sample (N=8) of staff members who completed the pretest, which are reported in aggregate but not at theindividual level due to privacy concerns.

Authors' Contributions
All authors conceptualized the study, conducted the investigation, worked on the methodology, administered the project, contributed
resources toward the project, and validated the information presented herein. LAE and JA provided data curation and wrote theoriginal draft. BEM and JA obtained funding for the study. JA managed visualization. All authors reviewed the final paper.

Conflicts of Interest
The authors have no conflicts of interest related to the content of this paper but note that they participate in ongoing research (ascited herein) related to this project and similar projects through grants awarded to their institutions and have served as grant peer
reviewers on harm reduction funding streams for federal agencies and foundations.

Multimedia Appendix 1
Staff-level preintervention data.
[DOCX File , 16 KB-Multimedia Appendix 1]

References
1.
2.

Drug overdose death rates. National Institute on Drug Abuse. 2023. URL: https://nida.nih.gov/research-topics/trends-statistics/
overdose-death-rates [accessed 2023-05-23]
Provisional drug overdose death counts. Centers for Disease Control and Prevention. 2023. URL: https://www.cdc.gov/
nchs/nvss/vsrr/drug-overdose-data.htm [accessed 2023-05-23]

https://formative.jmir.org/2024/1/e54077

XSL• FO
RenderX

JMIR Form Res 2024 | vol. 8 | e54077 | p. 11(page number not for citation purposes)

JMIR FORMATIVE RESEARCH
3.
4.
5.
6.
7.
8.

9.
10.
11.

12.

13.
14.
15.

16.

17.

18.
19.
20.

21.

22.

23.

24.
25.
26.

New report confirms U.S. life expectancy has declined to lowest level since 1996. National Center for Health Statistics.
2022. URL: https://www.cdc.gov/nchs/pressroom/nchs_press_releases/2022/20221222.htm [accessed 2023-05-25]
SUDORS dashboard: fatal overdose data. Centers for Disease Control and Prevention. 2022. URL: https://www.cdc.gov/
drugoverdose/fatal/dashboard/index.html [accessed 2023-05-23]
Indiana drug overdose dashboard. Indiana Department of Health. 2023. URL: https://www.in.gov/health/overdose-prevention/
data/indiana/ [accessed 2023-05-23]
Naloxone DrugFacts. National Institute on Drug Abuse. 2022. URL: https://nida.nih.gov/publications/drugfacts/naloxone[accessed 2022-05-03]
Naloxone for opioid overdose: life-saving science. National Institute on Drug Abuse. 2017. URL: https://www.drugabuse.gov/
publications/naloxone-opioid-overdose-life-saving-science [accessed 2021-12-15]
Walley AY, Xuan Z, Hackman HH, Quinn E, Doe-Simkins M, Sorensen-Alawad A, et al. Opioid overdose rates andimplementation of overdose education and nasal naloxone distribution in Massachusetts: interrupted time series analysis.
BMJ. 2013;346:f174. [FREE Full text] [doi: 10.1136/bmj.f174] [Medline: 23372174]
Clark AK, Wilder CM, Winstanley EL. A systematic review of community opioid overdose prevention and naloxone
distribution programs. J Addict Med. 2014;8(3):153-163. [doi: 10.1097/ADM.0000000000000034] [Medline: 24874759]
McDonald R, Strang J. Are take-home naloxone programmes effective? Systematic review utilizing application of theBradford Hill criteria. Addiction. 2016;111(7):1177-1187. [FREE Full text] [doi: 10.1111/add.13326] [Medline: 27028542]
Townsend T, Blostein F, Doan T, Madson-Olson S, Galecki P, Hutton DW. Cost-effectiveness analysis of alternative
naloxone distribution strategies: first responder and lay distribution in the United States. Int J Drug Policy. 2020;75:102536.
[doi: 10.1016/j.drugpo.2019.07.031] [Medline: 31439388]
Adams J. U.S. Surgeon General's advisory on naloxone and opioid overdose. U.S. Department of Health and Human
Services. 2018. URL: https://www.hhs.gov/surgeongeneral/priorities/opioids-and-addiction/naloxone-advisory/index.html[accessed 2021-07-03]
Drug misuse: most states have good samaritan laws and research indicates they may have positive effects. United States
Government Accountability Office. 2021. URL: https://www.gao.gov/assets/gao-21-248.pdf [accessed 2022-06-03]
Renewal of determination that a public health emergency exists. Administration for Strategic Preparedness & Response.
2023. URL: https://aspr.hhs.gov/legal/PHE/Pages/Opioid-26Sept2023.aspx [accessed 2023-10-22]
National drug control strategy. The White House, Executive Office of the President, Office of National Drug Control Policy.
2022. URL: https://www.whitehouse.gov/wp-content/uploads/2022/04/National-Drug-Control-2022Strategy.pdf [accessed2023-05-05]
Williams AR, Nunes EV, Bisaga A, Levin FR, Olfson M. Development of a Cascade of Care for responding to the opioid
epidemic. Am J Drug Alcohol Abuse. 2019;45(1):1-10. [FREE Full text] [doi: 10.1080/00952990.2018.1546862] [Medline:
30675818]
D'Onofrio G, O'Connor PG, Pantalon MV, Chawarski MC, Busch SH, Owens PH, et al. Emergency department-initiatedbuprenorphine/naloxone treatment for opioid dependence: a randomized clinical trial. JAMA. 2015;313(16):1636-1644.
[FREE Full text] [doi: 10.1001/jama.2015.3474] [Medline: 25919527]
Harris E. FDA Green-lights first over-the-counter naloxone spray. JAMA. 2023;329(16):1341. [doi: 10.1001/jama.2023.5626][Medline: 37018007]
Facher L. FDA advisers recommend approval of over-the-counter naloxone to fight opioid overdose. StatNews. 2023. URL:
https://www.statnews.com/2023/02/15/naloxone-otc-opioisa-fda-panel-recommends/ [accessed 2023-02-21]
Tse WC, Sanfilippo P, Lam T, Dietze P, Nielsen S. Community pharmacy naloxone supply, before and after reschedulingas an over-the-counter drug: sales and prescriptions data, 2014-2018. Med J Aust. 2020;212(7):314-320. [FREE Full text][doi: 10.5694/mja2.50524] [Medline: 32124984]
Allen ST, O'Rourke A, Johnson JA, Cheatom C, Zhang Y, Delise B, et al. Evaluating the impact of naloxone dispensation
at public health vending machines in Clark County, Nevada. Ann Med. 2022;54(1):2692-2700. [FREE Full text] [doi:10.1080/07853890.2022.2121418] [Medline: 36168975]
Berenbrok LA, Gabriel N, Coley KC, Hernandez I. Evaluation of frequency of encounters with primary care physicians vsvisits to community pharmacies among medicare beneficiaries. JAMA Netw Open. 2020;3(7):e209132. [FREE Full text][doi: 10.1001/jamanetworkopen.2020.9132] [Medline: 32667653]
Berenbrok LA, Tang S, Gabriel N, Guo J, Sharareh N, Patel N, et al. Access to community pharmacies: a nationwide
geographic information systems cross-sectional analysis. J Am Pharm Assoc (2003). 2022;62(6):1816-1822.e2. [FREEFull text] [doi: 10.1016/j.japh.2022.07.003] [Medline: 35965233]
Pharmacies—the face of neighborhood healthcare since well before the pandemic. National Association of Community
Drug Stores. 2021. URL: https://www.nacds.org/pdfs/about/rximpact-leavebehind.pdf [accessed 2022-08-18]
Bratberg JP, Smothers ZPW, Collins K, Erstad B, Veve JR, Muzyk AJ. Pharmacists and the opioid crisis: a narrative reviewof pharmacists' practice roles. J Am Coll Clin Pharm. 2019;3(2):478-484. [doi: 10.1002/jac5.1171]
Vadiei N, Eldridge LA, Meyerson BE, Agley J. "The gatekeepers in prevention": community pharmacist perceptions of
their role in the opioid epidemic. Subst Abus. 2022;43(1):319-327. [doi: 10.1080/08897077.2021.1941516] [Medline:
34214407]

https://formative.jmir.org/2024/1/e54077

XSL• FO
RenderX

Eldridge et al

JMIR Form Res 2024 | vol. 8 | e54077 | p. 12(page number not for citation purposes)

JMIR FORMATIVE RESEARCH
27.
28.

29.

30.

31.

32.

33.

34.

35.

36.

37.
38.

39.

40.
41.
42.
43.
44.

45.
46.

47.

48.

Kelling SE, Rondon-Begazo A, Mager NAD, Murphy BL, Bright DR. Provision of clinical preventive services by community
pharmacists. Prev Chronic Dis. 2016;13:E149. [FREE Full text] [doi: 10.5888/pcd13.160232] [Medline: 27788064]
Agley J, Meyerson BE, Eldridge LA, Crosby RA, Bentele KG, Jun M, et al. Exploration of pharmacist comfort with harm
reduction behaviors: cross-sectional latent class analysis. J Am Pharm Assoc (2003). 2022;62(2):432-440. [doi:10.1016/j.japh.2021.10.015] [Medline: 34742654]
Marley G, Annis IE, Ostrach B, Egan K, Delamater PL, Bell R, et al. Naloxone accessibility by standing order in North
Carolina community pharmacies. J Am Pharm Assoc (2003). 2024.:102021. [FREE Full text] [doi:10.1016/j.japh.2024.01.017] [Medline: 38307248]
Agley J, Meyerson B, Eldridge L, Smith C, Arora P, Richardson C, et al. Just the fax, please: updating electronic/hybrid
methods for surveying pharmacists. Res Social Adm Pharm. 2019;15(2):226-227. [doi: 10.1016/j.sapharm.2018.10.028][Medline: 30416040]
Agley J, Meyerson BE, Shannon DJ, Ryder PT, Ritchie K, Gassman RA. Using the hybrid method to survey U.S. pharmacists:
applying lessons learned to leverage technology. Res Social Adm Pharm. 2017;13(1):250-252. [doi:10.1016/j.sapharm.2016.09.007] [Medline: 27743847]
Eldridge LA, Agley J, Meyerson BE. Naloxone availability and dispensing in Indiana pharmacies 2 years after the
implementation of a statewide standing order. J Am Pharm Assoc (2003). 2020;60(3):470-474. [doi:10.1016/j.japh.2019.11.024] [Medline: 31917248]
Meyerson BE, Agley JD, Davis A, Jayawardene W, Hoss A, Shannon DJ, et al. Predicting pharmacy naloxone stocking
and dispensing following a statewide standing order, Indiana 2016. Drug Alcohol Depend. 2018;188:187-192. [FREE Fulltext] [doi: 10.1016/j.drugalcdep.2018.03.032] [Medline: 29778772]
Meyerson BE, Agley JD, Jayawardene W, Eldridge LA, Arora P, Smith C, et al. Feasibility and acceptability of a proposed
pharmacy-based harm reduction intervention to reduce opioid overdose, HIV and hepatitis C. Res Social Adm Pharm.
2020;16(5):699-709. [FREE Full text] [doi: 10.1016/j.sapharm.2019.08.026] [Medline: 31611071]
Meyerson BE, Moehling TJ, Agley JD, Coles HB, Phillips J. Insufficient access: naloxone availability to laypeople in
Arizona and Indiana, 2018. J Health Care Poor Underserved. 2021;32(2):819-829. [doi: 10.1353/hpu.2021.0107] [Medline:
34120979]
Lai RK, Friedson KE, Reveles KR, Bhakta K, Gonzales G, Hill LG, et al. Naloxone accessibility without an outsideprescription from U.S. community pharmacies: a systematic review. J Am Pharm Assoc (2003). 2022;62(6):1725-1740.
[doi: 10.1016/j.japh.2022.07.008] [Medline: 35989151]
Eldridge LA, Meyerson BE, Agley J. Pilot implementation of the PharmNet naloxone program in an independent pharmacy.
J Am Pharm Assoc (2003). 2023;63(1):374-382.e12. [FREE Full text] [doi: 10.1016/j.japh.2022.09.004] [Medline: 36209035]
Hohmann L, Harris K, Zhao Y, Marlowe K, Phillippe H, Correia C, et al. Organizational readiness to implement community
pharmacy-based opioid counseling and naloxone services: a scoping review of current practice models and opportunities.
Pharmacy (Basel). 2023;11(3):99. [FREE Full text] [doi: 10.3390/pharmacy11030099] [Medline: 37368424]
Eldridge LA, Agley J, Meyerson BE, Golzarri-Arroyo L. The PharmNet harm reduction intervention for community
pharmacies: protocol for a pilot randomized controlled trial. JMIR Res Protoc. 2022;11(10):e42373. [FREE Full text] [doi:10.2196/42373] [Medline: 36279161]
Rosenthal R. The file drawer problem and tolerance for null results. Psychol Bull. 1979;86(3):638-641. [doi:10.1037/0033-2909.86.3.638]
Pautasso M. Worsening file-drawer problem in the abstracts of natural, medical and social science databases. Scientometrics.
2010;85(1):193-202. [doi: 10.1007/s11192-010-0233-5]
Ekmekci PE. An increasing problem in publication ethics: publication bias and editors' role in avoiding it. Med Health Care
Philos. 2017;20(2):171-178. [doi: 10.1007/s11019-017-9767-0] [Medline: 28342053]
Stefan AM, Schönbrodt FD. Big little lies: a compendium and simulation of p-hacking strategies. R Soc Open Sci.
2023;10(2):220346. [FREE Full text] [doi: 10.1098/rsos.220346] [Medline: 36778954]
Showell M, Buckman S, Berber S, Allah NA, Patterson B, Cole S, et al. Publication bias in trials registered in the Australian
New Zealand Clinical Trials Registry: is it a problem? A cross-sectional study. PLoS One. 2023;18(1):e0279926. [FREEFull text] [doi: 10.1371/journal.pone.0279926] [Medline: 36602999]
Attribution 4.0 International (CC BY 4.0 Deed). Creative Commons. URL: https://creativecommons.org/licenses/by/4.0/[accessed 2024-02-23]
Hernandez I, Tang S, Morales J, Gabriel N, Patel N, Mathis WS, et al. Role of independent versus chain pharmacies inproviding pharmacy access: a nationwide, individual-level geographic information systems analysis. Health Aff Sch.
2023;1(1):qxad003. [FREE Full text] [doi: 10.1093/haschl/qxad003] [Medline: 37750164]
Rawal S, Osae SP, Tackett RL, MacKinnon NJ, Soiro FD, Young HN. Community pharmacists' naloxone counseling: atheory-informed qualitative study. J Am Pharm Assoc (2003). 2023;63(6):1743-1752.e2. [doi: 10.1016/j.japh.2023.08.018][Medline: 37633453]
Tofighi B, Martino D, Lekas HM, Williams SZ, Blau C, Lewis CF. Scaling opioid overdose prevention and naloxone
dispensation among rural and small metro area pharmacists: findings from a qualitative study. J Subst Use.
2022;28(4):568-573. [doi: 10.1080/14659891.2022.2070877]

https://formative.jmir.org/2024/1/e54077

XSL• FO
RenderX

Eldridge et al

JMIR Form Res 2024 | vol. 8 | e54077 | p. 13(page number not for citation purposes)

JMIR FORMATIVE RESEARCH
49.

50.

51.

52.

53.
54.

55.
56.
57.

58.
59.
60.

Eldridge et al

Green TC, Silcox J, Bolivar D, Gray M, Floyd AS, Irwin AN, et al. Pharmacy staff-reported adaptations to naloxone
provision and over-the-counter (OTC) syringe sales during the COVID-19 pandemic: experiences across multiple states
and 2 pharmacy chains. J Am Pharm Assoc (2003). 2024;64(1):71-78. [FREE Full text] [doi: 10.1016/j.japh.2023.10.014][Medline: 37863398]
Averitt AJ, Weng C, Ryan P, Perotte A. Translating evidence into practice: eligibility criteria fail to eliminate clinicallysignificant differences between real-world and study populations. NPJ Digit Med. 2020;3:67. [FREE Full text] [doi:10.1038/s41746-020-0277-8] [Medline: 32411828]
Survey of community pharmacy economic pressures 2022 report. National Community Pharmacists Association. 2022.
URL: https://ncpa.org/sites/default/files/2022-08/Pharmacy%20economic%20health%20survey%20AUG%2022.pdf[accessed 2023-10-27]
Doucette WR, Mott DA, Kreling DH, Witry MJ, Gaither CA, Schommer JC. 2019 National pharmacist workforce study:
final report. American Association of Colleges of Pharmacy; Jan 10, 2020. URL: https://www.aacp.org/sites/default/files/
2020-03/2019_NPWS_Final_Report.pdf [accessed 2024-02-27]
Burakoff M. Walgreens pharmacy staffers stage walkouts over work conditions. Associated Press. 2023. URL: https://apnews.com/article/walgreens-pharmacist-walkouts-ed9ff39eba9745a823710a95e541c857 [accessed 2023-11-28]
Coyle J, Rogers A, Copland R, De Paoli G, MacDonald TM, Mackenzie IS, et al. Trials@Home Consortium. Learningfrom remote decentralised clinical trial experiences: a qualitative analysis of interviews with trial personnel, patient
representatives and other stakeholders. Br J Clin Pharmacol. 2022;88(3):1031-1042. [FREE Full text] [doi:10.1111/bcp.15003] [Medline: 34296777]
Dee J, Dhuhaibawi N, Hayden JC. A systematic review and pooled prevalence of burnout in pharmacists. Int J Clin Pharm.
2023;45(5):1027-1036. [FREE Full text] [doi: 10.1007/s11096-022-01520-6] [Medline: 36446993]
McQuade BM, Reed BN, DiDomenico RJ, Baker WL, Shipper AG, Jarrett JB. Feeling the burn? A systematic review ofburnout in pharmacists. J Am Coll Clin Pharm. 2020;3(3):663-675. [doi: 10.1002/jac5.1218]
Green TC, Bratberg J, Irwin AN, Boggis J, Gray M, Leichtling G, et al. Study protocol for the Respond to Prevent Study:
a multi-state randomized controlled trial to improve provision of naloxone, buprenorphine and nonprescription syringes incommunity pharmacies. Subst Abus. 2022;43(1):901-905. [FREE Full text] [doi: 10.1080/08897077.2021.2010162][Medline: 35213293]
Breitenstein SM, Gross D, Garvey CA, Hill C, Fogg L, Resnick B. Implementation fidelity in community-based interventions.
Res Nurs Health. 2010;33(2):164-173. [FREE Full text] [doi: 10.1002/nur.20373] [Medline: 20198637]
Rickles NM, Sharma M, Harrow S, Silverwatch J. A narrative review: pharmacy intervention fidelity. J Am Pharm Assoc(2003). 2023;63(2):491-499.e2. [FREE Full text] [doi: 10.1016/j.japh.2022.10.010] [Medline: 36585297]
Express Scripts launches new initiative to expand rural health care access through partnerships with independent pharmacies.
Evernorth Health Services. 2023. URL: https://www.evernorth.com/articles/
express-scripts-expands-rural-health-care-access-partnerships-independent-pharmacies [accessed 2023-05-25]

Abbreviations
MOUD: medications for opioid use disorder
OEND: opioid education and naloxone distribution
OUD: opioid use disorder
RCT: randomized controlled trial

Edited by A Mavragani; submitted 28.10.23; peer-reviewed by SH Linder, D Carpenter; comments to author 24.11.23; revised version
received 28.11.23; accepted 19.02.24; published 18.03.24
Please cite as:
Eldridge LA, Meyerson BE, Agley J
Implementation Documentation and Process Assessment of the PharmNet Intervention: Observational Report
JMIR Form Res 2024;8:e54077
URL: https://formative.jmir.org/2024/1/e54077
doi: 10.2196/54077
PMID: 38498037©Lori Ann Eldridge, Beth E Meyerson, Jon Agley. Originally published in JMIR Formative Research (https://formative.jmir.org),
18.03.2024. This is an open-access article distributed under the terms of the Creative Commons Attribution License(https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium,https://formative.jmir.org/2024/1/e54077

XSL• FO
RenderX

JMIR Form Res 2024 | vol. 8 | e54077 | p. 14(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Eldridge et alprovided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information,a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.

https://formative.jmir.org/2024/1/e54077

XSL• FO
RenderX

JMIR Form Res 2024 | vol. 8 | e54077 | p. 15(page number not for citation purposes)