Prescription of Benzodiazepines by General Practitioners: Characteristics of Prescribing Trend and Implementation of an Online Educational Program
Phase: NA
Conditions: Anxiety, Prescription Drug Abuse and Dependency
Sponsor: Universidade Nova de Lisboa

 
 
 
 
 
Official title: Prescription of benzodiazepines by Primary health care doctors: 
characteristics of prescribing trend and implementation of an online educational 
program 
 
 
 
NCT number: not yet assigned 
 
 
Document date: July 29,2016 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Abstract 
 
BZD excessive prescription has long been considered a serious mental health concern 
in many countries. A large number of interventions using different methodologies have 
been implemented to change BZD prescription pattern at primary health care settings, 
with limited positive results.  
We aimed to propose the implementation of an effectiveness-implementation hybrid 
type 1 intervention. In our study we developed an online platform, named 
ePrimaPrescribe, which was delivered using a Digital Behavior Change 
Intervention(DBCI). 
We included in our study all primary health care units from one region in Portugal 
which were randomly allocated to receive a Digital Behavior Change Intervention 
(DBCI) in the format of an online platform to reduce BZD prescription 
(ePrimaPrescribe) or an online platform concerning communication techniques 
(control).  
We primarily aimed to evaluate the effectiveness of our Digital Behavior Change 
Intervention (DBCI) using as outcome measure the frequency of BZD prescriptions 
issued per month. Secondarily we aimed to analyze the effect of ePrimaprescribe on 
antidepressant prescriptions, to study the effect of ePrimaprescribe on diagnosis 
definition associated with BZD and antidepressant prescription;  to perform a cost 
analysis considering the monthly National Health Service spending with BZD co-
payment. We finally aim to analyze the implementation process using quantitative and 
qualitative methods.   
 
Ethics and dissemination 
 
This trial was approved by the Ethics Commission for Health of the Regional 
Administration of Health for Alentejo Region (Portugal) (02/2016(CES) and the Nova 
Medical School, Nova University Lisbon, Portugal Ethics Commission (47/2016/CEFCM). 
The results will be submitted for publication in peer-reviewed journals. 
 
 
Introduction 
 
Benzodiazepines (BZD) are commonly used psychotropic drugs for treating insomnia 
and anxiety (1). Their long-term use is associated with considerable adverse effects (2), 
such as increased number of falls and bone fractures (2–4), a higher number of road 
accidents (2,5), and possible role in inducing the phenomenon of suicide (6–8). 
Numerous studies have shown that BZD are still overprescribed and commonly used 
long-term (9,10), despite the existence of clinical guidance advising the use of non-
pharmacological psychological treatments first-line, and to restricting BZD prescription 
to a maximum of 8-12-weeks.  
The consistently high prescription of BZD in Portugal has been the target of references 
in national and international publications for the last 20 years, with this fact being 
recognized as a sign of inadequate management of mental disorders and considered a 
serious public health concern (11).  It is also a documented fact that BZD in Portugal 

are mostly prescribed in primary health care (12), hence pointing to this setting as 
crucial to implementation of interventions aiming to change BZD prescription pattern.  
A large number of interventions using different methodologies have been 
implemented to change BZD prescription pattern at primary health care settings, such 
as minimal educational interventions (13–15), systematic discontinuation interventions 
(16,17), audit and feedback interventions (18), and policy interventions (19,20). The 
results are inconsistent; when positive, the effect is frequently lost after a short period 
and seems closely related to each country's primary healthcare setting's particular 
characteristics. 
An educational outreach intervention previously implemented in Portugal had a small 
effect on BZD prescribing trend and pointed as a primary limitation the lack of staff to 
implement face-to-face educational sessions to a larger number of General 
Practitioners (GPs) and with adequate frequency (21). Online interventions, namely 
targeting a behavioral change, with a low cost and high possibility of wide distribution, 
were considered a possible way to surpass time and limited resources in primary 
health care settings to implement an intervention of this nature. 
BZD prescription has been reported as a complex behavior influenced by personal 
values, beliefs, attitudes, experiences, patients’ characteristics and demands (22–24). 
Therefore, besides the relevance of adequately implementing an intervention aiming 
to change BZD prescription pattern, it is also crucial to gather information about 
barriers to this implementation, and consider influence factors such as acceptability, 
practicability, effectiveness, affordability and equity.  
This study aimed to evaluate the impact of Digital Behavior Change Intervention 
tailored online program, ePrimaPrescribe, on changing BZD prescription patterns and, 
at the same time, to test its implementation in a real-world situation, using an 
effectiveness-implementation hybrid type 1 design. BZDs are often wrongly chosen to 
treat anxiety disorder symptoms, at the expense of the adequate pharmacological 
treatment with antidepressants (25,26). Thus, this study also aimed to verify the effect 
of our effectiveness trial on antidepressant prescription trends and on diagnoses 
definition associated with BZD prescription. 
 
Objectives 
 
Main objective 
To evaluate the effectiveness of a Digital Behavior Change Intervention (DBCI) at a 
primary health care setting, in which GPs are given access to an online program, 
ePrimaPrescribe, aiming to reduce BZD prescription.  
 
Secondary objective 
To analyze the effect of ePrimaprescribe on antidepressant prescriptions.  
To study the effect of ePrimaprescribe on diagnoses registration associated with BZD 
and antidepressant prescription.  
To perform a cost analysis considering the monthly National Health Service spending 
with BZD co-payment. 
To analyze the implementation process using quantitative and qualitative methods.   
 
Methods and analysis 

 
Study design and setting 
 
We choose an effectiveness-implementation hybrid design taking a dual focus a priori 
in assessing clinical effectiveness and implementation (27).  
A hybrid type 1 study tests a clinical intervention, while gathering information on its 
delivery during the effectiveness trial, and its potential for implementation in a real-
world situation. 
We performed our hybrid type 1 effectiveness-implementation using as methodology 
for intervention implementation a two-arm cluster randomized clinical trial. The 
cluster design was selected because the allocation and intervention were implemented 
at the healthcare unit level. 
The setting for our intervention was primary health care units at a rural region in 
Portugal, an area of 7393 km², an estimated population of 166 706 inhabitants (2011), 
a population density of 22.5 inhab./km², and approximately 250 GPs working in 
primary health care units (of which 110 were included in our study). Portugal has a 
public accessible national health service, but mental health indicators are alarming 
(28). BZD prescription in Portugal is very high, as introduced in our background section, 
and the consumption of these drugs is particularly significant in the region where our 
intervention was implemented (11). 
 
Eligibility criteria and recruitment 
 
All primary healthcare units from the Portuguese Central Alentejo region were 
considered eligible. A researcher contacted each healthcare unit coordinator, 
explained the project's nature, and invited to participation. 
The primary inclusion criteria were healthcare units with acceptance of at least 90% of 
GPs to participate in the study. We chose a higher acceptance rate than usually 
reported in the literature since we expected, although accepting to participate in the 
study, a significant number of GPs in each cluster would not actively use the DBCI 
ePrimaPrescribe platform.  
 
Sample size 
 
The unit of observation for primary analysis was at the individual GP prescription level. 
Data collected from a previous research implemented in a similar setting  (29) allowed 
performing estimation for sample size (SS) calculation. We started by doing an SS 
calculation based on all observations' independence and an effect size of 20% 
reduction. We obtained a total of N=58 participants, N=29 participants per study arm. 
To adjust for the intra-cluster correlation coefficient (30), we then calculated the 
design effect (Deff) expressed as: 
Deff= 1+r(m-1) - where r denotes the ICC and m is the size for each cluster.  
Considering an average cluster of m=5 and an ICC of 0.02, Deff=1.08.  
The SS adjusted for ICC was then be given by: 
N*= N(1+r (m-1)), so N*=29(1+0.02(5-1)=31.3, hence approximately 32 per study arm. 
The number of clusters (k) was given by:  
k= N(1+r (m-1))/m, so k=6.4 7 

Considering an ICC of 0.02, a 1:1 ratio of allocation of controls per intervention unit, an 
alpha type error of 0.05, a cluster size of 5 doctors per unit, a minimal difference 20% 
reduction in the number of BZD prescriptions, the study would have to include 7 
clusters per study arm in order to have an 80% power. 
 
Random allocation 
 
Randomization was stratified according to the organizational type of primary health 
care unit (UCSP vs. USF), the number of GPs per unit, the average number of 
appointments, and the number of patients per unit per month. We further included in 
our randomization criteria primary health care unit's proximity, meaning that when 
distinct units functioned on the same location/building, we allocated them to the same 
study arm. 
In previous research, using similar baseline BZD prescription data from primary health 
care units, there was a large spread of monthly BZD prescription (29). Since this spread 
was similar throughout different units, no stratified randomization (or matching for 
similar prescribing patterns) was considered to be necessary (31).  
Healthcare units were included if 90% of the GPs accept participation. 
For concealment of allocation, after all eligible healthcare units accepted participation, 
we allocated them simultaneously to intervention or control group using a computer-
generated random number of tables. 
The distribution of participants after randomization is presented in the following 
figure. 
 
 
 
Figure 1: Distribution of participants after randomization 
ACES Alentejo Central
18 potential primary 
health care units
18 interested primary 
health care units
18 enroled primary health 
care units
250 GPs
intervention arm: 9 
primary health care units
130 GPs
4 USF
69 GPs
5 UCSP
61 GPs
control arm: 9 primary 
health care units
120 GPs
6 USF
70 GPs
3 UCSP
50 GPs

 
Blinding 
 
GPs could not be blinded due to the fact that they were asked to participate actively in 
a study seeking to change their prescription practice. Both GPs included in intervention 
and control arm were asked to access an online platform. However, they were blinded 
to the existence of alternative content to the platform presented in their primary 
health care unit.  
 
The authors were not blinded to primary health care unit allocation.  
 
The primary and secondary outcomes were assessed by analysis of the electronic 
prescription and diagnosis registration database for each GP. These data were 
extracted and anonymized (except to primary health care unit identification) by a data 
manager at the central Shared Services of the Portuguese Ministry of Health, who was 
not involved in the study and was blinded to participant allocation. 
 
Intervention 
 
Development of Digital Behavior Change Intervention (DBCI) online programs 
 
The online programs were designed based on Behavior Changing Wheel theory (32) 
theory, considering that behavior change is key to improving healthcare and health 
outcomes. We used a Digital Behavior Change Intervention (DBCI) mode of delivery.  
The ePrimaPrescribe program, designed to be implemented in the intervention group, 
was developed based on guidelines for anxiety and depression treatment and 
benzodiazepines withdrawal. Our primary sources of information were National 
Institute for Health and Care Excellence (NICE) guidelines (33), guidelines issued by the 
Portuguese National Health Directorate (DGS) (34,35), and other relevant literature 
specifically addressing deprescribing evidence-based practice guidelines (36,37). The 
program comprised three e-learning modules, each with approximately 30 minutes of 
duration and with the following subjects: pharmacological effect and clinical use of 
BZD; how to treat anxiety disorder avoiding the continuous use of BZD; how to manage 
BZD dependence and BZD withdrawal proposals.  
 
The primary health care units included in the control arm of our study were also 
offered an online platform, named ComunicaSaudeMental. This DBCI platform's 
content was developed based on literature concerning general communication 
techniques and more specific communication techniques for addressing light to 
moderate mental health disorders or patient's emotional management at primary 
health care settings. This program comprised (similarly to the one offered to the 
intervention arm) three e-learning modules, each with approximately 30 minutes. 
 
After the development phase, modules were uploaded to two moodle platforms (one 
for the intervention educational platform, ePrimaPrescribe; another for the control 
educational platform, ComunicaSaudeMental), and an individual coded access was 
created to each participant.  

Before the implementation phase, both programs were tested by three GPs and three 
psychiatrists.  Two of the GPs were members of the Primary Health Care Research 
Department at NOVA Medical School, and one was also a member of the Mental 
Health Research Department at NOVA Medical School. Two of the three psychiatrists 
were specialists in substance misuse. These experts were asked to comment on the 
accuracy and quality of the e-learning modules critically. Their suggestions were 
carefully considered, and when appropriate, were integrated into the module's final 
version.  
 
Intervention implementation 
 
We used as guidance to the following intervention implementation report, the 
Template for Intervention Description and Replication (TIDieR) checklist and guide 
(38).  
 
ePrimaPrescribe online platform was implemented in primary health care setting since 
this is where most BZD initial and renovation prescriptions are issued. A Digital 
Behavior Change Intervention (DBCI) was chosen as the delivery method since we 
hypothesized that a free, easily accessible platform might have a higher and longer-
term effect changing BZD prescription pattern.   
Both primary health care units allocated to intervention and control arm had an initial 
face-to-face visit to present to GPs the online platform, delivery of personal 
identification access code, explanation on how to access and use it. Informed consent 
was asked during this initial face-to-face visit.  
 
We expected GPs to start accessing and hence actively participating in the study in the 
weeks following the first face-to-face visit to their unit. Access to the platform was 
free, available at any convenient time and frequency, through any digital device with 
internet connection.  
 
We performed all face-to-face visits to allocated primary health care units in a six-
week time frame.  
 
We sent an email to GPs every three months, at three, six and nine months after 
implementation, as a reminder for participation and as a strategy to improve 
adherence to the intervention.  The platform was not tailored, personalized, adapted, 
or modified in any way during the intervention implementation period. 
 
To assess how well the intervention was actually implemented, and hence to evaluate 
the extension to which the intervention was delivered as planned, we performed 
another face-to-face visit to all participating primary health care units, 12 months after 
the initial intervention. During this visit, GPs included in the intervention arm were 
asked to answer a survey exploring their motivations and expectations regarding the 
use of the ePrimaPrescribe program; a questionnaire for evaluating the barriers and 
facilitators to the Implementation of the ePrimaPrescribe online platform; and to 
participate in an exploratory group discussion over the general perception of their 
study participation and platform implementation.  

 
All face-to-face visits, and hence intervention procedures explanation and monitoring 
were provided by the first author, who has a background as a psychiatrist and regularly 
performs her clinical work in the same geographical area where the intervention was 
implemented. 
 
We performed semi-structured in-depth interviews with an intentional sample of 
participants from the intervention arm to explore perceptions on the feasibility and 
implementation of the study.  
 
Outcome assessment 
 
Primary outcome measure 
 
We used as primary outcome measure the frequency of BZD prescriptions issued per 
month, the proportion of prescriptions issued by participants included in intervention 
and control units over the study time frame and more specifically at baseline, six and 
12 months after intervention.  
We included BZDs from the following Anatomical Therapeutic Chemical classification 
system-coded groups: N05B; N05C and N03AE.  
 
Secondary outcome measures 
 
Effectiveness 
 
We used the frequency of antidepressant prescriptions issued per month, the 
proportion of prescriptions issued by participants included in intervention and control 
units over the study time frame and more specifically at baseline, six and 12 months 
after intervention. 
We included antidepressants from the following Anatomical Therapeutic Chemical 
classification system-coded groups: N06A.  
 
To study the effect of ePrimaprescribe on diagnosis registration, we used the monthly 
registration distribution of psychological symptoms, complaints, and diagnoses coded 
at the same month as BZD and antidepressant prescriptions. The GP's diagnosis 
registration used the International Classification of Primary Care, second edition (ICPC-
2) developed and updated by the World Organization of Family Doctors' (WONCA) 
International Classification Committee (WICC).  
 
We further performed a cost analysis considering the monthly National Health Service 
spending with BZD co-payment. This cost was compared with the amount that would 
need to be spent to comply with needs and solutions suggested when evaluating GP’s 
perceptions of the feasibility and implementation. 
 
Implementation 
 
We studied the implementation process using quantitative and qualitative methods.   

 
Quantitative data 
We developed a standardized onsite survey aiming to explore the following themes: 
GP’s self-evaluation of knowledge about the management of patients with anxious and 
depressive symptomatology; reasons for prescribing BZDs and antidepressants; 
subjective assessment of the amount of BZDs and antidepressants prescribed; reasons 
for keeping long-term BZD prescription; difficulties for changing long-term BZDs 
prescription; the degree of concern with continued BZD prescription; knowledge and 
degree of adequacy of the existing Portuguese guidelines on BZD prescription; 
motivations and expectations regarding the use of the ePrimaPrescribe program in 
clinical practice and participation in the study. This questionnaire had 18 multiple 
questions and 14 short answer questions. The development and testing of this 
questionnaire is detailed in Part 5, section 1. 
We adapted the Barriers and Facilitators Assessment Instrument (BaFAI)(39) to the 
implementation of the DBCI ePrimaPrescribe program. This questionnaire is organized 
into four categories: barriers deriving from the characteristics of the 
practice/innovation; barriers deriving from the characteristics of the professionals; 
barriers due to patient characteristics; barriers arising from the intervention context. It 
has twenty-five questions of a five-point Likert scale type.  
 
We asked all participating GPs to complete the onsite survey and BaFAI questionnaires 
at the end-of-study face-to-face visit that was performed after completion of the 
twelve-month intervention period.  
 
At the end-of-study face-to-face visit, we also collected GPs sociodemographic data.  
 
Qualitative data 
During our end-of-study face-to-face visit, we performed a group discussion in each 
primary health care unit included in the intervention group, to explore GP's 
perceptions over their participation.  
 
We finally performed semi-structured in-depth interviews with an intentional sample 
of participants from the intervention arm to explore perceptions of the study's 
feasibility and implementation. The interview guide structure was developed after an 
exploratory analysis of the themes emerging from the answers to the onsite survey 
questionnaire, from the answers to the BaFAI, and from the group discussion occurring 
during the end-of-study visit. 
 
Data management 
 
Each prescription data was coded using an individual GP and patient numerical 
identification, in a secured and validated electronic database, directly extracted by the 
central Shared Services of the Portuguese Ministry of Health. 
 
Data concerning clinical diagnosis was extracted by the same method as the 
prescription database. Matching of prescription and diagnosis database was 

performed using the coded patient's numerical identification and 
prescription/diagnosis registration month. 
 
Data obtained through questionnaires and interviews was collected after participants 
signing an informed consent during the initial implementation face-to-face visit.  
 
Statistical analysis 
 
We performed an exploratory descriptive analysis using number of prescriptions as 
primary measure of outcome, considering as main influencing factors the patient’s age 
and sex, by units of intervention and control.  
 
We performed most analysis at the level of intervention versus control clusters (so 
compared together the set of intervened units vs the set of control units), since the 
available data did not allow for the author to identify each of the participating GP, 
hence also not allowing to distinguish in the intervention units, which GPs were 
compliant with the intervention (i.e. the doctors that actually used the DBCI), from 
those that, although initially agreeing to participate, finally didn’t use the platform.   
 
We tested for significant differences among the baseline characteristics of the 
intervention and control group. We performed descriptive analysis, with continuous 
variables summarized using means and standard deviations for normal distributions, 
and by medians and the 25th and 75th percentiles for non-normal distributions. 
 
Estimated effects were calculated by comparing number of prescriptions in the 
intervention and control groups at baseline, six months, and 12 months after 
intervention.  
 
All analyses were performed on an 'intention-to-treat' basis (i.e., all initially enrolled 
GPs were included in the analysis according to the group to which they were assigned). 
This approach reduces the bias that may occur when participants not receiving 
assigned treatments are excluded from analysis. 
 
The intervention and control groups were compared at the defined time points 
accounting for clustering by unit.  
 
We tested for significant differences in the baseline characteristics of the control and 
intervention groups using t-tests or one-way ANOVA. This included calculation of 
means and/or proportions with confidence intervals, and on robust standard 
deviations (to account for clustering).  
We performed a secondary analysis where we explored the association between the 
frequency of BZD and antidepressant prescription with diagnoses using Chi-Squared 
tests for testing independence in two-way contingency tables. The Cochran-Armitage 
trend test was employed to assess how the proportion of two ordinal successes varies 
across the levels of a binary variable. And when both variables in a contingency table 
had ordered categories, the linear-by-linear test was used instead (40).  
 

We finally performed a cost analysis considering the monthly National Health Service 
spending with BZD co-payment, using t-tests or one-way ANOVA. 
 
Statistical significance was considered for p-values < 0.05.  
 
The R statistical software (41,42) was used to perform all the statistical analyses within 
the RStudio integrated development environment for R, RStudio Team (2019). The 
graphs and plots were obtained with use of the ggplot2 R package (43). 
 
 
We performed a descriptive analysis to correlate data from the onsite survey 
questionnaire and BaFAI questionnaire with GP's sociodemographic characteristics 
using Chi-Squared tests for testing independence in two-way contingency tables.  
 
We used qualitative methods, namely content analysis, to explore data from in-depth 
interviews using ATLAS.ti.  
 
Data was reported elsewhere according to the 2010 Consolidated Standards of 
Reporting Trials guidelines.  
 
Discussion  
 
Strengths and limitations 
 
The concerning reality of excessive BZD prescription in Portugal suggests the need for 
effective interventions, at minimal cost and with a low need for professional time. A 
maximally effective withdrawal strategy is especially important in primary care settings 
because of budgetary limitations and the small amount of GP time available per 
consultation (44). 
BZD (de)prescription is considered a complex behavior. Behavior change is key to 
improving healthcare and health outcomes. Therefore, we have chosen to design our 
intervention based on a behavior changing theory, the Behavior Changing Wheel 
theory (32), hoping that using a validated and reviewed framework would strengthen 
the quality of our approach. 
Concerning our effectiveness trial, this paper describes the protocol for a cluster-
randomized trial to assess, primarily, whether a Digital Behavior Change Intervention 
(DBCI) would have a significant effect on BZD prescription. It also describes two 
secondary analyses regarding antidepressant prescription trend and reporting of 
diagnoses registration associated to BZD prescription, hoping that the implementation 
of our DBCI could secondarily translate in the improvement of anxiety disorder 
registration and adequate treatment (with an increasing antidepressant prescription).  
Randomized trials are the gold standard to assess intervention effects, and cluster-
randomized trials are an appropriate design when interventions need to be introduced 
to groups of individuals (in our case groups of GPs prescribing in the same primary 
health care unit), which are randomly allocated to different study arms (30,45).  

We chose as delivery method a DBCI because these online interventions have the 
potential for low unit-cost, high reach, effective and acceptable ways to benefit 
individuals and society (46).  
We chose to include both prescription-related outcomes and diagnosis outcomes. 
Regarding prescription-related outcomes, we recognize the limitation of presenting 
our prescription-related outcome as the frequency of prescription. In the specific case 
of BZDs, this limitation is minimized by the fact that, in Portugal, the prescription of 
BZDs cannot be placed in chronic prescriptions. This means that when a BZD 
prescription is issued it has to be dispensed during the next 30 days, and that each 
prescription might have a maximum of two boxes of drug. Hence, we considered that, 
since there is the need for frequent prescription renewal, the number of prescriptions 
issued is an adequate measure to estimate our intervention effect.  
Regarding clinical diagnosis outcomes, we recognize that our data might be a fairly 
inexact approach, since most prescriptions are not associated with a diagnosis, and 
also because even when this association was found, it didn’t mean that symptoms or 
disorders were identified at the moment of prescription. Despite this limitation, we 
considered relevant to report the analysis of the change in clinical diagnosis, since it 
might indicate significant secondary effects coming from the intervention 
implementation.  
We recruited GPs to participation taking into account all the available characteristics 
that might influence our primary and secondary outcomes: sex, years of clinical 
experience, type of primary health care unit and previous training in mental health. 
We recognize as a limitation and possible bias the fact that it was the first author that 
was mainly responsible for implementation, and at the same time responsible for 
collecting data from the questionnaires and in-depth interviews. We minimized this 
limitation by having all data, its categorization, content and theme identification 
reviewed by a researcher which was not involved with data collection.  
 
Relationship to other studies and expected contribution  
 
The excessive BZD prescription is a reality for many countries other than Portugal. For 
this reason, a large body of evidence already exists, with extended research assessing 
the effectiveness of interventions aiming to change BZD prescription pattern. From 
simple methodologies, such as sending letters to long-term BZD users, to more 
complex ones, such as multi-step tailored interventions, most studies show limited 
short-term effects. Most studies also lack a more profonde understanding of the 
facilitators and barriers to implementation.  
We consider that our trial is innovative since it presents a methodology that was 
carefully prepared to reach a maximum effectiveness, at minimal cost and low need 
for professional time, and it also has an important focus on factors influencing 
implementation. We also consider our research to be of particular national interest 
because no matter how well structured an intervention might be, if it is not accepted 
by the public for whom it is designed, the results are inevitably scarce and limited.  
When dedicating a significant effort to explore GP’s perceptions over their 
participation experience, we expected to contribute for two important areas of 
knowledge. On one hand, to better understand the factors influencing the act of BZD 
prescription, and possibly on a larger perspective, what this prescription 

means/represents in the management of mental health issues in the primary health 
care setting. On the other hand, in a time and setting where online interventions are 
becoming more common, to explore the factors influencing acceptance and 
practicability of our Digital Behavior Changing Intervention.  Concerning specifically 
this mode of delivery, we also expected for our research to contribute for an in-depth 
exploration of perceptions, liable to be applied on other areas besides mental health 
and hence helping to shape digital interventions that are adjusted to what GPs want 
and feel motivated to comply.    
 
 
Ethics and dissemination 
 
This trial was approved by the Ethics Commission for Health of the Regional 
Administration of Health for Alentejo Region (Portugal) (02/2016(CES) and the Nova 
Medical School, Nova University Lisbon, Portugal Ethics Commission (47/2016/CEFCM). 
The results of this study were disseminated via peer-reviewed publications and 
conference presentations. All data will be available on request. 
 
 
References 
 
1.  
Lader M. Benzodiazepines revisited—will we ever learn? Addiction. 
2011;106(12):2086–109.  
2.  
Brandt J, Leong C. Benzodiazepines and Z-drugs: an updated review of major 
adverse outcomes reported on in epidemiologic research. Drugs R D. 
2017;17(4):493–507.  
3.  
Khong TP, De Vries F, Goldenberg JSB, Klungel OH, Robinson NJ, Ibáñez L, et al. 
Potential impact of benzodiazepine use on the rate of hip fractures in five large 
European countries and the United States. Calcif Tissue Int. 2012;91(1):24–31.  
4.  
Donnelly K, Bracchi R, Hewitt J, Routledge PA, Carter B. Benzodiazepines, Z-
drugs and the risk of hip fracture: A systematic review and meta-analysis. PLoS 
One. 2017;12(4):e0174730.  
5.  
Thomas RE. Benzodiazepine use and motor vehicle accidents. Systematic review 
of reported association. Can Fam Physician. 1998;44:799.  
6.  
Carlsten A, Waern M, Holmgren P, Allebeck P. The role of benzodiazepines in 
elderly suicides. Scand J Public Health. 2003;31(3):224–8.  
7.  
Neutel CI, Patten SB. Risk of suicide attempts after benzodiazepine and/or 
antidepressant use. Ann Epidemiol. 1997;7(8):568–74.  
8.  
Cato V, Holländare F, Nordenskjöld A, Sellin T. Association between 
benzodiazepines and suicide risk: A matched case-control study. BMC 
Psychiatry. 2019;19(1):1–7.  
9.  
Kurko TA, Saastamoinen LK, Tahkapaa S, Tuulio-Henriksson A, Taiminen T, 
Tiihonen J, et al. Long-term use of benzodiazepines: Definitions, prevalence and 
usage patterns - a systematic review of register-based studies. Eur Psychiatry. 
2015/11/07. 2015;30(8):1037–47.  
10.  
Cullinan S, O’Mahony D, Fleming A, Byrne S. A meta-synthesis of potentially 
inappropriate prescribing in older patients. Drugs Aging. 2014;31(8):631–8.  

11.  
Furtado C, Teixeira I. Utilização de benzodiazepinas em Portugal continental 
(1999-2003). Acta Med Port. 2006;19(3):239–46.  
12.  
INFARMED. Benzodiazepinas e análogos 2016. INFARMED. 2017.  
13.  
Cormack MA, Sweeney KG, Hughes-Jones H, Foot GA. Evaluation of an easy, 
cost-effective strategy for cutting benzodiazepine use in general practice. Br J 
Gen Pr. 1994;44(378):5–8.  
14.  
Gorgels WJMJ, Voshaar RCO, Mol AJJ, van de Lisdonk EH, van Balkom AJLM, van 
den Hoogen HJM, et al. Discontinuation of long-term benzodiazepine use by 
sending a letter to users in family practice: a prospective controlled intervention 
study. Drug Alcohol Depend. 2005;78(1):49–56.  
15.  
Bueno MB, de Velasco Artaza ER, Uria JF, Iturbe AG. Benzocarta: intervención 
mínima para la desprescripción de benzodiacepinas en pacientes con insomnio. 
Gac Sanit. 2019;  
16.  
Richards D, Toop L, Graham P. Do clinical practice education groups result in 
sustained change in GP prescribing? Fam Pract. 2003;20(2):199–206.  
17.  
Vicens C, Bejarano F, Sempere E, Mateu C, Fiol F, Socias I, et al. Comparative 
efficacy of two interventions to discontinue long-term benzodiazepine use: 
cluster randomised controlled trial in primary care. Br J Psychiatry. 2014;bjp. bp. 
113.134650.  
18.  
Baker R, Farooqi A, Tait C, Walsh S. Randomised controlled trial of reminders to 
enhance the impact of audit in general practice on management of patients who 
use benzodiazepines. BMJ Qual Saf. 1997;6(1):14–8.  
19.  
Kollen BJ, van der Veen WJ, Groenhof F, Donker GA, van der Meer K. 
Discontinuation of reimbursement of benzodiazepines in the Netherlands: does 
it make a difference? BMC Fam Pract. 2012;13(1):111.  
20.  
Gentile G, Lapeyre-Mestre M, Micallef J. Combatting the misuse of 
benzodiazepines and related Z drugs in French general practice: a clinical 
review. BJGP open. 2020;  
21.  
Arensman E, Coffey C, Griffin E, Van Audenhove C, Scheerder G, Gusmao R, et al. 
Effectiveness of depression–suicidal behaviour gatekeeper training among 
police officers in three European regions: Outcomes of the Optimising Suicide 
Prevention Programmes and Their Implementation in Europe (OSPI-Europe) 
study. Int J Soc Psychiatry. 2016;62(7):651–60.  
22.  
Sketris IS, Ingram EML, Lummis HL. Strategic opportunities for effective optimal 
prescribing and medication management. J Popul Ther Clin Pharmacol. 
2009;16(1).  
23.  
Whyte SR, Van der Geest S, Hardon A. Social lives of medicines. Cambridge 
University Press; 2002.  
24.  
Abraham J. Pharmaceuticalization of society in context: theoretical, empirical 
and health dimensions. Sociology. 2010;44(4):603–22.  
25.  
American Psychiatric Association AP, Association AP. Diagnostic and statistical 
manual of mental disorders: DSM-5. Washington, DC: American psychiatric 
association; 2013.  
26.  
Stahl SM, Stahl SM. Stahl’s essential psychopharmacology: neuroscientific basis 
and practical applications. Cambridge university press; 2013.  
27.  
Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-
implementation hybrid designs: combining elements of clinical effectiveness 

and implementation research to enhance public health impact. Med Care. 
2012;50(3):217.  
28.  
Caldas de Almeida JMJ, Xavier M. Estudo Epidemiologico Nacional de Saúde 
Mental. 1o relatório. Nova Medical School. Faculdade de Ciências Médicas, 
Universidade Nova de Lisboa; 2013.  
29.  
Alves dos Reis T. Change in prescribing patterns of benzodiazepines after 
training of General Practitioners. Master in Clinical investigation. NOVA Medical 
School; 2015.  
30.  
Rutterford C, Copas A, Eldridge S. Methods for sample size determination in 
cluster randomized trials. Int J Epidemiol. 2015;44(3):1051–67.  
31.  
Eldridge S, Kerry S. A practical guide to cluster randomised trials in health 
services research. Vol. 120. John Wiley & Sons; 2012.  
32.  
Michie S, Van Stralen MM, West R. The behaviour change wheel: a new method 
for characterising and designing behaviour change interventions. Implement Sci. 
2011;6(1):42.  
33.  
NICE CKS. Benzodiazepine and z drug withdrawal. Retrieved from cks nice org 
uk/benzodiazepine-and-z-drug-withdrawal. 2013;  
34.  
DGS. Tratamento Sintomático da Ansiedade e Insónia com Benzodiazepinas e 
Fármacos Análogos. Direção Geral da Saúde. 2011.  
35.  
DGS. Tratamento sintomático da ansiedade e insónia com benzodiazepinas e 
fármacos análogos: norma de orientação clínica n. o 055/2011, de 27/10/2011, 
atualizada a 21/01/2015. Direção Geral da Saúde. DGS Lisboa; 2015.  
36.  
Pottie K, Thompson W, Davies S, Grenier J, Sadowski CA, Welch V, et al. 
Deprescribing benzodiazepine receptor agonists. Can Fam Physician. 
2018;64(5):339 LP – 351.  
37.  
All Wales Medicines Strategy Group. Educational Pack: Material to Support 
Appropriate Prescribing of Hypnotics and Anxiolytics across Wales. 2016.  
38.  
Hoffmann TC, Glasziou PP, Boutron I, Milne R, Perera R, Moher D, et al. Better 
reporting of interventions: template for intervention description and replication 
(TIDieR) checklist and guide. Bmj. 2014;348.  
39.  
Harmsen M, Peters M, Wensing M. Barriers and facilitators assessment 
instrument Introduction, instructions and instrument. IQ Healthc Sci Inst Qual 
Heal Radboud Univ Med Cent Nijmegen. 2005;  
40.  
Agresti A. An introduction to categorical data analysis. John Wiley & Sons; 2018.  
41.  
Team RC. R Foundation for Statistical Computing. R: A language and 
environment for statistical computing. vol; 2019.  
42.  
Team Rs. RStudio: Integrated Development for R. RStudio, Inc., Boston, MA. 
2015. URL https//www rstudio com/products/rstudio. 2019;  
43.  
Wickham H, Chang W, Wickham MH. Package ‘ggplot2.’ Creat Elegant Data Vis 
Using Gramm Graph Version. 2016;2(1):1–189.  
44.  
Vicens C, Socias I, Mateu C, Leiva A, Bejarano F, Sempere E, et al. Comparative 
efficacy of two primary care interventions to assist withdrawal from long term 
benzodiazepine use: a protocol for a clustered, randomized clinical trial. BMC 
Fam Pract. 2011;12(1):23.  
45.  
Pinto D, Heleno B, Rodrigues DS, Papoila AL, Santos I, Caetano PA. An open 
cluster-randomized, 18-month trial to compare the effectiveness of educational 
outreach visits with usual guideline dissemination to improve family physician 

prescribing. Implement Sci. 2014;9(1):10.  
46.  
Michie S, West R. A guide to development and evaluation of digital behaviour 
change interventions in healthcare. Silverback Publishing; 2016.