A Pairwise Randomized Study on Implementation of Guidelines and Evidence Based Treatments of Psychoses
Conditions: Psychotic Disorders
Sponsor: University Hospital, Akershus

IMP Project description 2015 1027 
1 
 
A pairwise randomized study on implementation of guidelines and 
evidence based treatments of psychoses 
1. Relevance relative to the call for proposals  
• The project is targeting patients with severe mental illness and is partly health services research.  
• It is “patient-near” as it addresses service delivery to patients (service users) and collects data on 
outcome in relation to implementation. Service users are involved in all stages in the project.  
• It is relevant for the health services as it will give new knowledge on current implementation of 
clinical guidelines, and on how implementation on evidence based treatments can be improved.  
• The project includes mental health clinics from six health authorities in three health regions of 
Norway, including three university hospitals.  
2. Aspects relating to the research project  
The objectives are described in the grant application form.  
2.1. Background and status of knowledge 
Persons with severe mental illness are among the key target groups for the mental health services, 
and it is of major importance to know whether the mental health services succeed in helping this 
group. The mental health services in Norway are giving services to 165 000 adults during a year, 
and 13 000 of these (8 %) have psychoses (F20-29 in ICD-10). Ten percent of the total patient 
population uses 80 % of the resources, and 28 % of these are persons with psychoses, indicating 
that many of these have great needs for comprehensive services [1]. Child and adolescent mental 
health services additionally meet 400 adolescents with psychoses during a year.  
Evidence based treatments and clinical guidelines 
Several treatment approaches and specific interventions have been developed for psychoses, and 
there is now research based evidence for several for these that they are effective. These include 
pharmacological treatments (such as antipsychotic drugs) and psychological therapies (such as 
cognitive behavioral therapy, work with families, psycho-education, supported employment, illness 
management and recovery, and integrated treatment of psychosis and substance abuse). 
During the last two decades an increasing number of clinical guidelines have been developed, based 
on reviews and meta-analyses of research. The aim has been to contribute to closing the gap 
between research evidence (what is known) and routine clinical practice (what is done). 
In spite of these developments and intentions to improve the services by practicing guidelines, the 
extent to which guidelines are implemented in everyday routine practice in Norway and elsewhere 
is largely unknown. We know even less whether implementation of clinical guidelines actually 
improves patient outcome. The few implementation studies reported in a recent Cochrane review 
[2] and other reviews [3] indicate that guideline implementation is typically fragmented or lacking, 
and that improved patient outcome may be limited even when guideline implementation occurs.  
There may be several reasons for this. One is that there is a roadblock to translate knowledge and 
new methods from optimized efficacy trials into everyday clinical practice [4]. The pathway from 
evidence generation to evidence synthesis and guideline development is well developed, but the 
pathway from evidence-based guidelines to evidence-based practice is much less developed. In 
efficacy studies clinicians may have been selected and especially skilled or motivated, and patient 
groups in such trials are often selected based on several exclusion criteria. But in everyday practice 
non-selected clinicians are supposed to change their practice to implement evidence based 
treatments with unselected patients with various comorbidity and additional problems that were 
excluded in the trials which the guidelines are based on.  
Fidelity measures for implementation of evidence based treatments for psychoses 
Fidelity scales have been developed for several of these, providing objective data on whether a 
clinical team or program have implemented key components of the evidence-based model for a 
treatment or service delivery.  “The rationale for using fidelity scales to guide practice is based on 
the working hypothesis that programs successfully replicating the core principles and procedures of 

IMP Project description 2015 1027 
2 
 
the program models rigorously evaluated in controlled studies will achieve similar outcomes as 
these earlier studies” [5].  
The US National Evidence-Based Practices project (US NEBP) made a major contribution to 
developing strategies and fidelity measures for research on implementation of evidence based 
treatments for this group. Five evidence based treatments (Assertive Community Treatment, Family 
Psycho-education, Integrated Dual Disorder Treatment, Illness Management and Recovery, 
Supported Employment/IPS) were chosen, and 53 CMHCs in 8 states each implemented one of 
these. Implementation was successful in various degrees across sites and types of evidence based 
treatment. US NEBP developed several fidelity scales, and designed and studies a comprehensive 
implementation support program [6-9]. 
Some European studies on evidence-based treatments for schizophrenia have developed indicators 
based on guidelines, but these have not been developed as fidelity scales [10, 11].  
Implementation support programs 
Research has shown that several strategies need to be combined both on the system level and for 
clinicians in order to achieve a successful implementation of a new practice. Frameworks for 
strategies are emerging to structure such combined approaches [12-15]. Strategies also define what 
is important in various stages in the implementation process and in relation to different target 
groups [16-20]. It is important to engage leaders on all levels to be involved in the decisions and 
process of implementation throughout the whole process. In order to change clinical practice, 
clinicians must understand and agree on the need to change practice, and receive supervision and 
feedback frequently over some time by supervisors taking part in small group discussions at the 
local site and communicating results from fidelity assessments and other relevant measures.  
According to a recent review of studies on implementation in mental health, US NEBP had the 
implementation program with the most comprehensive list of elements and activities [21]. Based on 
a review of the implementation literature the elements included in the US NEBP were toolkits with 
a practice manual and other tools to implement the practice, a trainer who made frequent onsite 
visits to the agency implementing the practice, a fidelity review process with feedback, a “kickoff” 
to build enthusiasm, initial training for the practitioners, consultations and technical assistance to 
the agency leadership, and consultation and technical assistance to the governmental authority 
responsible for mental health services. It was also emphasized that the implementation support must 
be reasonably intensive during the first months, must be sensitive to site-specific conditions, and 
helpful to implementers through three phases of implementation: building momentum for change, 
making the changes, and reinforcing the changes. The toolkit contained workbooks, research 
articles, introductory and instructional videotapes, PowerPoint lectures and the fidelity scales. 
Situation in Norway and collaborating partners in the project 
The Norwegian national guidelines on assessment and treatment of persons with psychoses (2013) 
give 74 different recommendations for assessment and treatment. Each of these is rated regarding 
level of evidence and importance. We do not know to which extent these are implemented. 
The mental health clinics of six health authorities (including three university hospitals) are partners 
in the project, representing three of the health regions in Norway and serving a total population of 
1,9 million (38 % of Norway’s population). Altogether they have 30 CMHCs (with outpatient 
clinics, day units, mobile teams, and inpatient wards). Each serves a specific area and population.  
Health authorities with mental health clinics that are collaborating partners in the project  
Health 
Region 
Mental health clinics from these health 
authorities are partners in the project 
CMHCs 
Population 
in area 
Patients 
F20-29* 
South-East 
Akershus University Hospital 
4 
500 000 
1 400 
The Hospital Innlandet 
5 
400 000 
1 120 
Sørlandet Hospital 
4 
290 000 
   810 
West 
Stavanger University Hospital 
5 
330 000 
   920 
Health Fonna 
4 
170 000 
   480 
North 
University Hospital of Northern Norway 
8 
230 000 
   640 

IMP Project description 2015 1027 
3 
 
Total 
 
30 
1 920 000 
5 370 
*) Estimated number of patients 16+ years old with F20-29 (ICD-10) seen during one year. 
2.2. Approaches, hypotheses and choice of method  
2.2.1.Research questions 
1. What is the current level of implementation in the mental health services of selected core 
elements in the national clinical guidelines for treatment of persons with psychoses? 
2. Is implementation of the selected core guidelines improved by a comprehensive implementation 
support program compared with no such support? 
3. Does implementation of selected core guidelines improve patient care pathway, patient outcome 
and satisfaction of patients and their families? 
2.2.2.Hypotheses 
1. Current implementation of evidence based approaches and treatments are low. 
2. A comprehensive implementation program for an evidence based treatment gives significant 
increase in implementation of the treatment compared to no such specific program. 
3. Higher implementation of core clinical guidelines is associated with better patient care 
pathways, patient outcome and higher satisfaction of patients and their families. 
2.2.3.Design and overview of the Implementation Study and the Patient Outcome Sub-study 
Implementation Study 
The design of the main study is a randomized paired design to answer research question 2. The unit 
of analysis for the implementation study is the CMHC, and the primary outcome measure is fidelity 
to recommendations/treatment packages. Each of the 30 CMHCs chooses two of packages that they 
will implement. For each center we randomize implementation support to one package and no 
support to the other one. 30 pairs of packages are randomized for implementation support or not.  
We have chosen the CMHCs because they are the clinical units giving the whole range of outpatient 
and outreach services over time for the population in a local area, including for persons with 
psychosis. Other departments serving patients with psychosis may also get implementation support 
for treatment packages, but they will not be included in the randomized design for CMHCs. 
We have selected four treatment packaged covering core recommendations in the Norwegian 
guidelines (see 2.2.4 below). A survey among the CMHCs and departments in May 2015 showed 
adequate interest for the four chosen treatment packages. The survey before the final selection of 
treatment packages was done according to our plan to wait with the final decision on design A or B 
is supported by the advices given by the Medical Research Council in UK for research on complex 
interventions [22].  
The level of implementation of two chosen packages at each CMHC/department is assessed with 
fidelity scales at baseline before implementation and after 6, 12 and 18 months. 
The fidelity measured at baseline (before the start of implementation support) will show the current 
implementation of the treatment packages across centers (research question 1).  
Necessary number of CMHCs: Based on data from the US NEBP project, we have calculated that 
we need 8 CMHCs in each branch to show that implementation support for a treatment package 
gives a significant increase in fidelity compared to baseline or low fidelity. In their project fidelity 
assessors reliably rated EBP fidelity on 5-points fidelity scales (1 = poor fidelity…5 = high fidelity) 
[8]. Average fidelity scores across all five EBPs and 51 sites receiving implementation support were 
2.28 (SD 0.95) at baseline 3.76 (SD 0.78) after 12 months (effect size 1.70). Our calculation is 
based on their data and 5% two-tailed significance and 90% power. If the differences are smaller in 
our study, we will need a larger sample. However, it is unlikely to have a large increase in fidelity 
without implementation support [23]. 
Patient Outcomes Sub-study 
A sub-study of patient outcome and satisfaction with services in relation to the targets of each 
treatment package aims to measure the effect of improved implementation of the packages (research 

IMP Project description 2015 1027 
4 
 
question 3). Unit of analysis is CMHC on the system level and patient on the individual level. 
Primary and secondary patient outcome measures are described in part 2.2.6. 
Inclusion criteria: Patients 16+ years old with psychoses (diagnosis F20-29 in the ICD-10). The 
number of adults with psychoses in the mental health services is 270 per 100 000 inhabitants during 
one year, with additional 10 adolescents. We aim to include 20% of these, which is 56 per 100 000 
population in each area, with a total of 1074 patients (96-280 per health authority). Only patients 
giving written informed consent will be included. There are no exclusion criteria.  
Inclusion procedures:  Patients are included at the level of the mental health clinic of the health 
authority, so that each person is included only once. The inclusion will start at a specific date and 
will be coordinated by the local coordinator. Eligible patients already being in contact with the 
clinic at that time will be invited to take part, and new referred patients assessed to have psychosis 
at admission will be asked consecutively until requisite number is met. Each included patient will 
be followed for 18 months from inclusion. 
2.2.4.Selection of recommendations and treatment packages to implement 
The 74 recommendations in the Norwegian guidelines are distributed on various domains of 
assessments or interventions. By combining related recommendations we have defined treatment 
packages (evidence based treatments) with corresponding available fidelity measures. 
Implementation of a package covering several recommendations makes the recommendations more 
manageable, and may be expected to give more effect than a single recommendation. 
To support local motivation and acceptance of the choice of treatment packages and the research 
design with limitations in packages to choose [22], the CMHCs/departments were invited to discuss 
and rate their interest for five different treatment packages in May 2015. The four chosen packages 
were the four with the widest support in this survey.  
Treatment packages for randomization 
We study the implementation of only four treatment packages in our project in order to make it 
feasible for the services to have a limited number of models to focus on, and to obtain the necessary 
power for the statistical analyses (see calculations under 2.2.3 above).  
Our selection of treatment packages has been based on these criteria: (a) They cover one or more 
recommendation with high evidence (1a/1b) and/or high importance (A) in the Norwegian 
guidelines, (b) they are considered by the participating collaborators to be among core elements in 
the services for persons with psychosis, (c) there are available fidelity scales that may be used or 
potential measures that may be adapted, (d) they may be relevant for most patients with psychosis, 
(e) the required competence is available or within reach by training in the collaborating health 
authorities, (f) they are feasible for the services, and (g) the related recommendations refer to 
treatments and not assessments, as we are focusing on treatments. Below we briefly describe the 
selected packages, also showing which recommendation (Rxx) each covers.  
Antipsychotic medication: Specific elements include using antipsychotics in treatment of psychosis, 
limiting polypharmacy, avoiding high doses and fast increase of doses, differentiating dosage in 
different phases and situation, and trying clozapine if two other antipsychotics have been tried 
without effect. Recently developed fidelity measures (MedMAP) are adapted to cover the core 
recommendations on antipsychotic medication in the guidelines [24-26] (R43-60). Efforts to 
improve adherence and observe side effects are also defined and included. 
Family interventions: Collaboration with carers/family, family psycho-education and meeting 
children of the patient are considered evidence based treatments. There are available fidelity scales 
for measuring family support and psycho-education [27]. We clarify the model and include a 
measure on following up children of patients. This treatment package covers six recommendations 
in the guidelines (R4, R28-32). 
Care for somatic health: The need for attention to somatic health is great due to the documented 
higher morbidity and mortality of persons with schizophrenia and related psychoses. Physical 
activity is also important for somatic health. This treatment package covers three recommendations 

IMP Project description 2015 1027 
5 
 
in the guidelines (R23-24,R39). Dartmouth researchers have developed a fidelity scale on 
implementation of a physical health program that we are adapting [28]. 
Illness management and recovery: Training in independent living and personal recovery is 
addressing some goals that many services users want most. Illness Management and Recovery 
(IMR) is a broad training package integrating several evidence based treatments. The manual, 
patient material and fidelity scales for IMR is translated into Norwegian and used in an ongoing 
project. Our research group has competence in using the fidelity scales for IMR. This package 
covers the recommendation in the guidelines on training in social and practical skills (R40).  
Elements that will be measured but not randomized 
There are also some overarching elements in mental health services delivery that are considered 
evidence based and are included among the recommendations in the national guidelines. These will 
not be randomized, but will be and included in data analyses also as possible moderators of patient 
outcome. These elements are emphasis on recovery with personal goals and service user preferences 
(R1), treatment alliance (R5,R62), shared decision making (R5), individually adjusted treatment 
(R26), individual care plan with crisis prevention plan (R74), care pathway (12), continuity of care 
(R6) and close collaboration between services, as emphasized in the national Collaboration Reform. 
2.2.5.Implementation support program 
Each health authority assigns one or more professional full time or part time to give implementation 
support to their CMHCs/departments, adjusting the capacity to number of CMHCs/departments and 
travel distances. The implementation trainers are experienced clinicians with experiences in giving 
supervision. Combination of trainers may be necessary if one trainer does not have competence to 
give implementation support on all packages. 
The trainer makes regular onsite consultations to each CMHC for 18 months, with visits every 
second week the first six months, and then once a months the next 12 months. The trainer will only 
give implementation support for the treatment packages randomly assigned to such support. With 4-
6 CMHCs/departments in the health authority and one day for each visit, the trainer will use two 
weeks a month the first six months and one week a month the next 12 months visiting CMHCs. 
The trainers will be trained in January-February 2016 to be ready to give implementation support 
from the beginning of March 2016. They will be trained by experts from Dartmouth and Norway in 
workshops in Norway. During the18 months of implementation the trainers from all the six health 
authorities will meet every third month for networking, discussion and supervision.  
We are adapting the implementation toolkits developed in the US NEBP project (see 2.1 above). 
Some content is specific for the treatment package and some is more general for the implementation 
process. The aim of the implementation support is to engage the clinicians in implementing the 
treatment package, to help identify and overcome barriers, and to help build systems to support the 
new practice so that it sustains itself over time.  
The trainer will be asked to keep notes on the process and make a written report for every 6 months 
on the process, barriers and work to overcome these. This may also include semi-structured 
interviews with leaders and clinicians on the implementation process. These reports will be used to 
analyze the factors contributing to successful or less successful implementation. 
2.2.6.Measures of implementation of treatment packages 
Measures in Implementation Study 
Measures on implementation will be fidelity scales for measuring to what extent a treatment 
package is implemented by the CMHC/department. Researchers will do the fidelity assessments in 
January 2016 (baseline, before start of the implementation support March 2016) and after 6, 12 and 
18 months. We are using fidelity scales developed in the US NEBP project and elsewhere [29, 30], 
adapting them to the Norwegian context, which is finished in November 2015. This is done in 
collaboration with Gary Bond and his colleagues at the Dartmouth PRC who developed several 
fidelity scales for the US NEBP project.  
 

IMP Project description 2015 1027 
6 
 
 
Measures in Patient Outcome Sub-study 
We aim to use robust measures from electronic patient records as primary outcome on patient level, 
especially whether patients continue to stay in contact with the mental health services or drop out. 
We will also measure continuity for services in relation to transfer of patients between units. We 
will seek approval and consent to get data from national official registers on health and use of 
health services (Norwegian Patient Register, HELFO for primary health and social care, NAV, 
Medication Prescription Register). This will also give data on concurrent use of several types of 
services, and partly on collaboration.   
Secondary outcome measures of patient outcomes will be filled in by the patients themselves 
(questionnaires) and by the clinicians involved in the treatment (rating scales). We have chosen 
brief instruments (or brief parts for longer instruments) to make the data collection feasible. The 
questionnaire to the patients will be limited to 4 pages, using a well designed layout developed by 
the Statistics Norway. We are also negotiating with the firm Checkware on possibility to make the 
questionnaires and forms available online on tablets/pcs (see more below). 
The questionnaire to patients covers quality of life (MANSA), mental health and problems (CORE-
10, SF-12), functioning (WSAS), substance abuse (parts of Audit and Dudit), participation in the 
community (parts of LCQ), relationship to most important clinician (STAR), continuity and 
collaboration of services (part of CONTINUUM), experience of recovery (part of RSA), shared 
decision-making (3 questions based on extensive recent research) and experiences and satisfaction 
with services (CSQ-8 and some specific questions related to the evidence-based treatments offered. 
With consent from the patient, the closest relative will receive a brief questionnaire on their 
experiences with the services (CSQ-8, part of FIA-Q). 
The form to be filled in by the clinician covers diagnoses (including information on any diagnostic 
procedures used), mental problems (HoNOS, CGI), physical health, functioning (LSQ, SIX), 
alcohol and substance abuse, relationship to the patient (STAR) and content of treatment last 6 
months (including specific information on evidence-based treatments).  
2.2.7.Data collection 
Data collection in Implementation Study 
Fidelity assessment every 6 month will be done by the postdocs and other researchers in the 
collaborating health authorities. To control bias and be able to calculate inter-rater reliability (from 
independent ratings) each assessment team will consist of a researcher from the health authority 
with the CMHC and a researcher from another health authority. The assessment will be done by site 
visits where the researchers gather information from the sources specified in the fidelity measure 
(usually written material, interviews, observations and reading randomly selected patient records).  
One fidelity assessment will take 2-3 work days for each researcher, including preparation, site visit 
and work afterwards to complete the assessment and a brief report. With 4-5 CMHCs/departments 
in a health authority and fidelity assessments every 6 months during two years, this will take 2-3 
weeks for a researcher every 6 months in his/her own health authority, and similar time in another 
health authority. Assessing fidelity of two or more packages at the same CMHCs/departments may 
require one more day per visit if the fidelity measures involves much work. A researcher will spend 
4-8 weeks a year on assessments of fidelity in 2016 and 2017. 
Data collection in Patients Outcome Sub-study 
Data collection on patients will be coordinated by the local coordinator, and done at baseline (at 
inclusion) and after 6, 12 and 18 months. Inclusion of patients will start in the beginning of March 
2016 and continue for six months. As the patients will be in various CMHCs/departments, this work 
will be distributed on various clinical units and clinicians. The clinicians will administer 
questionnaires to patients and do clinical ratings described above. Researchers will collect data from 
discharged patients who are not in contact with the mental health services any more.  

IMP Project description 2015 1027 
7 
 
We are negotiating with the firm Checkware to use electronic data collection by filling in 
questionnaires and forms online on tablets/laptops. Four of the six sites in our study are already 
making agreements with Checkware for such services after pilot testing. Our paper forms will be 
pilot tested in January 2016, and online data collection may eventually be piloted at the same time. 
The project will order data extraction from the official registers mentioned above. This will be done 
for the whole period (18 months before inclusion and 18 months during the study) when such data 
will be available in 2018. 
2.2.8.Plan for data analyses 
Analysis in Study of implementation of guidelines (research questions 1 and 2) 
To answer Research Question 1, we will examine baseline fidelity scores for the four treatment 
packages, using the benchmark of 4.0 as adequate fidelity.  We will calculate the percentage of sites 
achieving benchmark fidelity (4.0) at baseline. We will also analyze the distribution of fidelity 
scores and explore contributing factors the reasons, including any barriers that may be addressed. 
Analyses to answer Research Question 2 will partly depend on the final decision on design (see 
2.2.3 above). First, we will compare the experimental and control conditions on fidelity across time, 
controlling for baseline and ignoring the content of package. This analysis will consist of 30 pairs of 
sites, measured 4 points in time.  Baseline scores will be used as covariates. We consider various 
ANONVA analyses. Second, for each package we will compare all experimental sites to all control 
sites. The number of observation in this analysis will depend on how many CMHCs choose each of 
the EBPs. Baseline scores will be used as covariates. Third, we will examine within-group changes 
over time for each package receiving implementation support, calculating effect size and 
determining percentage of sites achieving benchmark fidelity (4.0) at each time period. 
Analysis in the Patient Outcome Sub-study (research question 3) 
Analyses of research question 3: Data analysis on primary and secondary patient outcomes will 
include multi-level analysis with CMHC/department as system level (including measured fidelity) 
and patient as individual level. The planned sample of 1074 patients (each living in catchment areas 
of one of the 30 CMHCs/departments), gives an average of 36 patients per CMHC/department, well 
suited for multi-level analyses. Correlations between fidelity and various patient outcomes may be 
done as secondary analyses. 
2.3. The project plan, project management, organization and cooperation 
2.3.1.Collaborating partners 
The clinics for mental health of the six health authorities that are collaborating partners from three 
of the health regions are listed in the table above. Our national and international collaborators are: 
Dartmouth Psychiatric Research Center, New Hampshire, USA: Professors Robert E Drake and 
Gary R Bond are leading international experts on developing evidence based treatments for people 
with severe mental illness, developing fidelity scales for such treatments, and designing and running 
studies on implementation of evidence based treatments for people with severe mental illness. Both 
will work 10% on the project. They have taken part in designing the study and making fidelity 
measures available, and they will collaborate on adapting measures and developing additional 
measures, as well as on training of staff and researchers, and on analyzing and publishing results. 
Mental Health Norway: Dagfinn Bjørgen, elected national leader, is a member of the joint project 
group. Local representatives of user organizations and user councils at the health authorities take 
part in the local project groups. 
2.3.2.Project organization 
Akershus University Hospital HF is coordinating the project and responsible towards the funding 
agencies, and Professor Torleif Ruud (head of the R&D Mental Health Department) is project 
manager (principal investigator). The project organization builds on the project manager’s 
experiences from managing other multi-center studies with similar complexity. 
A joint project group led by the project manager (PI) is established with one responsible person 
from each collaborating partner, including from participating user organizations. This group meets 

IMP Project description 2015 1027 
8 
 
regularly and takes all major decisions together. The group and the project manager are responsible 
to their organizations and to the signed agreement between the collaborating partners (with the 
project description and budgets as attachments).  
Each collaborating partner has established a local project group under the leadership of the local 
coordinaor representing the partner in the joint project group. This local group will consist of 
researchers, leaders and clinicians from participating departments/units, and persons from user 
organizations.  
2.3.3.Contents of the five work packages (WPs) 
Work package 
Activities and deliveries 
WP1. Project management 
(months 1-48) 
Negotiate final agreement with collaborating partners. 
Apply to regional committees on ethics and local privacy ombudsmen. 
Receive funding and distribute this to the partners.  
Organize meetings of project group and joint seminars for all. 
Monitor tasks and milestones of work packages and the project. 
Reports to collaborating partners and funding agencies. 
Manage data base, coordinate data analyses  
Coordinate publishing and dissemination of the results. 
Discuss and possibly agree on further collaboration and network. 
WP2.Preparation and 
training (months 1-12) 
Survey among leaders on experiences of suggested packages. 
Joint workshops for final decisions on packages and clinical units. 
Adapt and prepare implementation toolkits for packages. 
Adapt and pilot test fidelity scales and other measures. 
Train trainers in giving implementation support. 
Train researchers in fidelity assessment and clinical assessments.   
WP3.Implementation support 
(months 13-30) 
Distribute national guidelines to all departments in the clinics. 
Distribute implementation toolkits for treatment packages. 
Establish leadership steering groups in all CMHCs. 
Training in treatment packages using implementation toolkits. 
Site visits by trainer for 18 months (twice a month first, tapering). 
Collaboration to identify and overcome barriers. 
Feedback on fidelity after each fidelity assessment (every 6 months) 
WP4. Implementation study 
(months 13-36) 
Implementation/fidelity is measured before the start of the implementation 
support and then 6, 12 and 18 months after. 
Trainers keep notes on implementation process and barriers. 
Questionnaires to leaders/clinicians baseline and 6, 12, 18 months. 
WP5.Patient Outcome Sub-
study (months 13-30) 
Data extraction from electronic records and official registers. 
Include patients in sub-study according to plan. 
Questionnaires to patients/families baseline and 6, 12, 18 months. 
2.3.4.Plan of progress 
Preparations of training material and design of forms for data collection are finished in November 
2015, and training of implementation trainers and researchers is done in January 2016. The 
CMHC/department chooses two treatment packages in November/December 2015. Baseline 
assessment of implementation of the selected treatments is done in February 2016, and pairwise 
randomization of implementation support is done right after the baseline assessment of fidelity. 
Inclusion of patients for the patients sub-study starts 1 March 2016 with a 6 months inclusion 
period. Intensive implementation support also start in March and is given for 6 months, and then 
regular implementation support is given for 12 months. Implementation assessment is done again 6, 
12 and 18 months after the start of implementation support. 
2.3.5.Preliminary plan for postdocs and publications 
A postdoc for 2 years (50 % over 4 years) at each of the six collaborating partners is given high 
priority in the application to (a) be the researchers coordinating the data collection in the health 
authority, and to (b) support development of implementation research in each health authority. The 
focus for publications for each postdoc is agreed upon as a part of the preparation (WP2), also 

IMP Project description 2015 1027 
9 
 
taking into account the research interest and competence of the individual person in postdoc. These 
postdocs will be important persons for the study and will contribute to its success, as well as for 
building local competence in assessment of fidelity and research on implementation. 
We will agree on guidelines for publication and develop a publication plan describing each article 
(research questions, assigned data, analyses, authors), taking into account competences and interest 
of each collaborating partner. We have developed a model for this though several studies that 
include two or more partners. The project manager and others will support each partner in 
developing applications for PhD research fellows that may work on sub-studies or connected local 
studies that may be developed as a local extension of the present study.  
2.4. Budget.  Budget information is included in the application.  
3. Key perspectives and compliance with strategic documents 
3.1. Compliance with strategic documents  
Both national and regional health authorities have strategic documents and objectives to give high 
quality health services including evidence based assessments and treatments, and the clinical 
guidelines to be implemented are national guidelines from the national health authorities. 
3.2. Relevance and benefit to society  
The aim of implementing guidelines is to give better and more effective health services, and 
contribute to a better and more productive life for citizens with severe mental illness. Increased 
competence in implementing evidence based treatments for this group may also be used by the 
health authorities to give better implementation of evidence bases services for other groups.    
3.3. Environmental impact  
The project is not expected to influence the physical environment, but can contribute to the 
improvement of the psycho-social environment for people with severe mental illness. 
3.4. Ethical perspectives  
Application of approval of the study will be sent to the Regional Committee for Ethics in Research 
(REK), as well as to the Privacy Ombudsman at Akershus University Hospital and at the other 
health authorities. The project aims to contribute to the improvement of services for people with 
severe mental illnesses, which has positive ethical aspects. Data collection can be experienced by 
some service users as a burden, and perhaps positively by others. (See more in application.) 
3.5. Gender issues (Recruitment of women, gender balance and gender perspectives)  
Both women and men will be recruited for participation in the study. Statistical analyses will 
include whether implementation of guidelines is done equally for men and women, and whether 
there are differences in how men and women experiences current practice and implementation. 
4. Dissemination and communication of results  
4.1 Dissemination plan. The dissemination plan is written in the application.  
4.2 Communication with users  
Persons with service user competence are involved in the project both in the joint project group and 
in the local project groups. They are important collaborating partner in preparing the study, 
interpreting results, disseminating user experience and other results, and in influencing health 
authorities and services to larger scale implementation of new successful models for service 
delivery. The results will be shared by users and to users in collaboration with their organizations, 
including on their websites. 
 
References 
 
1. 
Helsedirektoratet, Pasienter i det psykiske helsevernet 2009. [Patients in the mental health services 2009]. 
2011: Oslo. 
2. 
Barbui, C., et al., Implementation of treatment guidelines for specialist mental health care. Cochrane Database 
Syst Rev, 2014. 1: p. Cd009780. 

IMP Project description 2015 1027 
10 
 
3. 
Weinmann, S., M. Koesters, and T. Becker, Effects of implementation of psychiatric guidelines on provider 
performance and patient outcome: systematic review. Acta Psychiatr Scand, 2007. 115(6): p. 420-33. 
4. 
Tansella, M. and G. Thornicroft, Implementation science: understanding the translation of evidence into 
practice. Br J Psychiatry, 2009. 195(4): p. 283-5. 
5. 
Bond, G.R., et al., Strategies for improving fidelity. Research on Social Work Practice in the National 
Evidence-Based Practices Project, 2009. 19(5): p. 569-581. 
6. 
Torrey, W.C., et al., Implementing Evidence-Based Practices for Persons With Severe Mental Illnesses. 
Psychiatric Services, 2001. 52(1): p. 45-50. 
7. 
Bond, G.R., et al., Individualization and quality improvement: two new scales to complement measurement of 
program fidelity. Adm Policy Ment Health, 2009. 36(5): p. 349-57. 
8. 
McHugo, G.J., et al., Fidelity outcomes in the National Implementing Evidence-Based Practices Project. 
Psychiatr Serv, 2007. 58(10): p. 1279-84. 
9. 
Torrey, W.C., et al., Evidence-based practice implementation in community mental health settings: the relative 
importance of key domains of implementation activity. Adm Policy Ment Health, 2012. 39(5): p. 353-64. 
10. 
Bollini, P., et al., Indicators of Conformance With Guidelines of Schizophrenia Treatment in Mental Health 
Services. Psychiatric Services, 2008. 59(7): p. 782-791. 
11. 
Lasalvia, A., et al., Psychosis Incident Cohort Outcome Study (PICOS). A multisite study of clinical, social and 
biological characteristics, patterns of care and predictors of outcome in first-episode psychosis. Background, 
methodology and overview of the patient sample. Epidemiol Psychiatr Sci, 2012. 21(3): p. 281-303. 
12. 
Proctor, E.K., et al., Implementation research in mental health services: an emerging science with conceptual, 
methodological, and training challenges. Adm Policy Ment Health, 2009. 36(1): p. 24-34. 
13. 
Proctor, E., et al., Outcomes for implementation research: conceptual distinctions, measurement challenges, 
and research agenda. Adm Policy Ment Health, 2011. 38(2): p. 65-76. 
14. 
Aarons, G.A., M. Hurlburt, and S.M. Horwitz, Advancing a conceptual model of evidence-based practice 
implementation in public service sectors. Adm Policy Ment Health, 2011. 38(1): p. 4-23. 
15. 
Rowlands, P., The NICE schizophrenia guidelines: the challenge of implementation. Advances in Psychiatric 
Treatment, 2004. 10(6): p. 403-412. 
16. 
Decker, S.E., M.T. Jameson, and A.E. Naugle, Therapist training in empirically supported treatments: a 
review of evaluation methods for short- and long-term outcomes. Adm Policy Ment Health, 2011. 38(4): p. 
254-86. 
17. 
Grimshaw, J.M., et al., Changing provider behaviour. An overview of systematic reviews of interventions. 
Madical care, 2001. 39(8, Supplement 2): p. II 2-II 45. 
18. 
Cheater, F., et al., Tailored interventions to overcome identified barriers to change: effects of professional 
practice and health care outcomes (Review). Cochrane Database Syst Rev, 2005(3, Art no: CD005470). 
19. 
Kimberly, J. and J.M. Cook, Organizational measurement and the implementation of innovations in mental 
health services. Adm Policy Ment Health, 2008. 35(1-2): p. 11-20. 
20. 
Wensing, M., H. Wollersheim, and R. Grol, Organizational interventions to implement improvements in 
patient care: a structured review of reviews. Implement Sci, 2006. 1: p. 2. 
21. 
Menear, M. and C. Briand, Implementing a continuum of evidence-based psychosocial interventions for people 
with severe mental illness: part 1-review of major initiatives and implementation strategies. Can J Psychiatry, 
2014. 59(4): p. 178-86. 
22. 
Craig, P., et al., Developing and evaluating complex interventions: the new Medical Research Council 
guidance. BMJ, 2008. 337: p. a1655. 
23. 
Shojania, K.G. and J.M. Grimshaw, Evidence-based quality improvement: the state of the science. Health Aff 
(Millwood), 2005. 24(1): p. 138-50. 
24. 
El-Mallakh, P., et al., Challenges of implementing a medication management evidence-based practice in a 
community mental health setting: results of a qualitative study. Issues Ment Health Nurs, 2014. 35(7): p. 517-
25. 
25. 
Howard, P.B., et al., Prescriber Fidelity to a Medication Management Evidence-Based Practice in the 
Treatment of Schizophrenia. Pschiatric Services, 2009. 60(7): p. 929-935. 
26. 
Taylor, A.C., et al., Scales to evaluate quality of medication management: development and psychometric 
properties. Adm Policy Ment Health, 2009. 36(4): p. 247-54. 
27. 
McFarlane, W.R., et al., Clinical and Functional Outcomes After 2 Years in the Early Detection and 
Intervention for the Prevention of Psychosis Multisite Effectiveness Trial. Schizophrenia Bulletin, 2014. 
28. 
Bartels, S.J., et al., Clinically significant improved fitness and weight loss among overweight persons with 
serious mental illness. Psychiatr Serv, 2013. 64(8): p. 729-36. 
29. 
Bond, G.R., et al., Measurement of Fidelity in Psychiatric Rehabilitation. Mental Health Services Research, 
2000. 2(2). 
30. 
Bond, G.R., et al., Psychiatric rehabilitation fidelity toolkit. 2000: Cambridge, MA.