The Added Value of an Internet-based Intervention for Treatment of Aggression in Forensic Psychiatric Outpatients: A Multicentre, Randomized Controlled Trial
Phase: NA
Conditions: Aggression, Psychosocial Problem, Forensic Psychiatry, Internet-based Intervention
Sponsor: University of Twente

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
 
 
 
 
 
 
 
 
 
 
 
 
 
RESEARCH PROTOCOL 
 
The added value of an internet-based intervention for treatment of 
aggression in forensic psychiatric outpatients 
Protocol for a multicentre, randomized controlled trial 
 
Hanneke Kip, PhD 
Lisa Klein Haneveld, MSc 
Yvonne Bouman, PhD 
Saskia Kelders, PhD 
 
 
NCT ID not yet assigned 
 
Date: April 29th, 2022 
 
 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
 
 
 
PROTOCOL TITLE: The added value of an internet-based intervention for treatment of 
forensic psychiatric outpatients: protocol for a multicentre, randomized controlled trial 
 
Protocol ID 
NL80846.091.22 
Short title 
Aggression  
EudraCT number  
- 
Version 
2 
Date 
29-04-2022 
Coordinating 
investigator/project leader 
 
Hanneke Kip, PhD  
h.kip@utwente.nl  
University of Twente 
Psychology, Health and Technology 
PO Box 217 
7500AE Enschede  
Principal investigator(s) (in 
Dutch: hoofdonderzoeker/ 
uitvoerder) 
<Multicenter research: per site> 
 
Lisa Klein Haneveld, MSc 
l.kleinhaneveld@transfore.nl 
Sites: 
Stichting Transfore 
De Woenselse Poort 
GGZ Noord-Holland-Noord  
Kairos 
See document I1 for the list of research locations and 
information about the first contacts.  
Sponsor (in Dutch: 
verrichter/opdrachtgever) 
Hanneke Kip, PhD  
h.kip@utwente.nl  
University of Twente 
Psychology, Health and Technology 
POBox 217 
7500AE Enschede 
Subsidising party 
Kwaliteit Forensische Zorg (KFZ) 
Project number 2021-157 
Independent expert (s) 
Lieke Christenhusz 
Laboratory sites <if applicable> 
Not applicable 
Pharmacy <if applicable> 
Not applicable 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
 
 
 
PROTOCOL SIGNATURE SHEET 
 
Name 
Signature 
Date 
Sponsor or legal representative: 
Hanneke Kip, PhD 
Assistant professor  
 
 
01-03-2022 
Principal Investigator: 
Lisa Klein Haneveld, MSc 
Researcher 
 
 
01-03-2022 
 
 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
 
 
 
TABLE OF CONTENTS 
1. 
INTRODUCTION AND RATIONALE .............................................................................10 
2. 
OBJECTIVES ................................................................................................................14 
3. 
STUDY DESIGN ...........................................................................................................15 
4. 
STUDY POPULATION ..................................................................................................19 
4.1 
Population (base) ...................................................................................................19 
4.2 
Inclusion criteria .....................................................................................................20 
4.3 
Exclusion criteria ....................................................................................................20 
4.4 
Sample size calculation ..........................................................................................21 
5. 
TREATMENT OF SUBJECTS .......................................................................................22 
5.1 
Investigational product/treatment ............................................................................22 
5.1.1 Intervention: Aggression  ........................................................................................22 
5.1.2 Treatment as usual  ................................................................................................24 
6. 
INVESTIGATIONAL PRODUCT ....................................................................................26 
7. 
NON-INVESTIGATIONAL PRODUCT ...........................................................................26 
8. 
METHODS ....................................................................................................................27 
8.1 
Study parameters/endpoints ...................................................................................27 
8.1.1 
Main study parameter/endpoint .......................................................................27 
8.1.2 
Secondary study parameters/endpoints (if applicable) ....................................28 
8.1.3 
Other study parameters (if applicable) .............................................................30 
8.2 
Randomisation, blinding and treatment allocation ..................................................31 
8.3 
Study procedures ...................................................................................................32 
8.4 
Time investment of participants ..............................................................................36 
8.5 
Withdrawal of individual subjects ............................................................................37 
8.6 
Replacement of individual subjects after withdrawal ...............................................37 
8.7 
Follow-up of subjects withdrawn from treatment .....................................................37 
8.8 
Premature termination of the study .........................................................................37 
9. 
SAFETY REPORTING ..................................................................................................38 
9.1 Temporary halt for reasons of subject 
safety………………………...………………………………38 
9.2 
AEs, SAEs and SUSARs ........................................................................................38 
9.2.1 
Adverse events (AEs) ......................................................................................38 
9.2.2 
Serious adverse events (SAEs) .......................................................................38 
9.2.3 
Suspected unexpected serious adverse reactions (SUSARs) .........................38 
9.3 
Annual safety report ...............................................................................................39 
9.4 
Follow-up of adverse events ...................................................................................39 
9.5 
[Data Safety Monitoring Board (DSMB) / Safety Committee] ..................................39 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
 
 
 
10. 
STATISTICAL ANALYSIS ..........................................................................................40 
10.1 
Primary study parameter(s) ....................................................................................40 
10.2 
Secondary study parameter(s) ...............................................................................40 
10.3 
Other study parameters ..........................................................................................41 
11. 
ETHICAL CONSIDERATIONS ...................................................................................42 
11.1 
Regulation statement .............................................................................................42 
11.2 
Recruitment and consent........................................................................................42 
11.3 
Objection by minors or incapacitated subjects (if applicable) ..................................42 
11.4 
Benefits and risks assessment, group relatedness .................................................43 
11.5 
Compensation for injury .........................................................................................44 
11.6 
Incentives (if applicable) .........................................................................................44 
12. 
ADMINISTRATIVE ASPECTS, MONITORING AND PUBLICATION .........................45 
12.1 
Handling and storage of data and documents ........................................................45 
12.2 
Monitoring and Quality Assurance ..........................................................................45 
12.3 
Amendments ..........................................................................................................46 
12.4 
Annual progress report ...........................................................................................46 
12.5 
End of study report .................................................................................................46 
12.6 
Public disclosure and publication policy ..................................................................46 
13. 
STRUCTURED RISK ANALYSIS...............................................................................47 
14. 
REFERENCES ..........................................................................................................47    
15. 
APPENDIX ................................................................................................................47      
   

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
6 of 57 
LIST OF ABBREVIATIONS AND RELEVANT DEFINITIONS  
 
ABR 
General Assessment and Registration form (ABR form), the application 
form that is required for submission to the accredited Ethics Committee; 
in Dutch: Algemeen Beoordelings- en Registratieformulier (ABR-
formulier) 
AE 
Adverse Event 
AR 
Adverse Reaction 
AVL-AV 
Aangepaste Versie van de Agressie Vragenlijst 
CA 
Competent Authority 
CCMO 
Central Committee on Research Involving Human Subjects; in Dutch: 
Centrale Commissie Mensgebonden Onderzoek 
CV 
Curriculum Vitae 
CVTRQ 
Corrections Victoria Treatment Readiness Questionnaire 
DSMB 
Data Safety Monitoring Board 
EU 
European Union 
EudraCT European drug regulatory affairs Clinical Trials  
FARE 
Forensisch Ambulante Risico Evaluatie 
GCP 
Good Clinical Practice 
GDPR 
General Data Protection Regulation; in Dutch: Algemene Verordening 
Gegevensbescherming (AVG) 
IB 
Investigator’s Brochure 
IC 
Informed Consent 
IMP 
Investigational Medicinal Product  
IMPD 
Investigational Medicinal Product Dossier  
METC  
Medical research ethics committee (MREC); in Dutch: medisch-ethische 
toetsingscommissie (METC) 
RCT 
Randomized Controlled Trial 
RESE 
Regulatory Emotional Self-efficacy scale 
(S)AE 
(Serious) Adverse Event  
SPC 
Summary of Product Characteristics; in Dutch: officiële 
productinformatie IB1-tekst 
Sponsor 
The sponsor is the party that commissions the organisation or 
performance of the research, for example a pharmaceutical 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
7 of 57 
company, academic hospital, scientific organisation or investigator. A 
party that provides funding for a study but does not commission it is not 
regarded as the sponsor, but referred to as a subsidising party. 
SUSAR  
Suspected Unexpected Serious Adverse Reaction 
TAU 
Treatment as usual 
TWEETS TWente Engagement with Ehealth and Technologies Scale 
UAVG 
Dutch Act on Implementation of the General Data Protection Regulation; 
in Dutch: Uitvoeringswet AVG 
WMO 
Medical Research Involving Human Subjects Act; in Dutch: Wet Medisch-
wetenschappelijk Onderzoek met Mensen 
 
 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
8 of 57 
SUMMARY 
Rationale: While internet-based interventions have been used for over ten years in Dutch 
forensic psychiatric outpatient care, no thorough evaluation study has been conducted yet. 
Studies on internet-based interventions in other mental healthcare sectors show promising 
results: they can increase the quality and efficiency of care. However, it is not clear if, why 
and for whom these interventions work in forensic mental healthcare. It is especially 
important to study whether these interventions are of added value for this complex patient 
population, known for its low treatment motivation, co-morbidity and low literacy levels. 
Objective: The primary objective of this study is to investigate whether the addition of the 
internet-based intervention ‘Aggression’ to forensic psychiatric outpatient treatment as usual 
(TAU) results in better treatment outcomes in terms of self-reported regulatory emotional 
self-efficacy, treatment readiness, and aggression. Additionally, it is investigated whether the 
experimental group requires fewer treatment sessions and improves more on dynamic risk 
factors. Furthermore, to gain more insight into for whom these interventions work, it is 
investigated whether engagement with the internet-based intervention predicts adherence 
and effectiveness. Finally, this study aims to explore reasons for the (in)effectiveness of the 
intervention according to patients and therapists.  
Study design: To investigate if the use of internet-based interventions is of added value for 
treatment of forensic psychiatric outpatients, a multicenter, non-blinded, parallel groups, 
randomized controlled trial design is used. Patients fill out three short self-report 
questionnaires four times: at baseline, mid-treatment (+6 weeks), post-treatment (+14 
weeks) and at follow-up (+26 weeks). Semi-structured interviews with a randomly selected 
sample of 20 patients from the experimental condition and with all participating therapists are 
conducted to explain the results of the RCT.  
Study population: The target group of this study consists of forensic psychiatric outpatients, 
treated in four Dutch organizations for aggression regulation problems.   
Intervention (if applicable): This study investigates the existing internet-based intervention 
Aggression, which was introduced in forensic mental healthcare over ten years ago. 
However, as is the case with other internet-based interventions, uptake in practice remains 
relatively low. ‘Aggression’ is used as an addition to treatment as usual, and thus does not 
replace any part of treatment in this study. The intervention is earlier developed by the CE-
certified company Minddistrict. The last couple of months the intervention has been 
cosmetically adjusted in close cooperation with therapists, patients and other stakeholders so 
that it fits the current design and lay-out of Minddistrict. The goals of this intervention are to 
(1) increasing the motivation to change, (2) acquiring skills for dealing with conflict, and (3) 
breaking the cycle of aggression by providing knowledge on situational, emotional, cognitive 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
9 of 57 
and physical triggers. It contains ten lessons, each containing written texts, videos and audio 
files, and short written assignments on which the therapist can give feedback.  
Main study parameters/endpoints: All participants fill out three validated, short self-report 
questionnaires on self-efficacy, treatment readiness and aggression at four points during the 
study. Participants in the experimental conditions also fill out an engagement questionnaire 
during the use of the intervention. Dynamic risk factors are assessed via a risk assessment 
instrument that is already used as a standard part of treatment according to Dutch guidelines 
and the number of treatment sessions are retrieved from existing systems. The patient and 
therapist perspective is take in into account by means of semi-structured interviews.  
Nature and extent of the burden and risks associated with participation, benefit and 
group relatedness: Since participants only have to fill out short questionnaires and are 
compensated for their time, participating in this study is not viewed as a major burden. 
Furthermore, the interventions of Minddistrict have been used for over ten years in mental 
healthcare, during which no risks or adverse events have been observed. Because  –not all 
forensic psychiatric patients receive internet-based interventions as part of their standard 
care and because these interventions are not yet evaluated and are thus not evidence-
based, participants in the control condition are not deprived of effective treatment. During the 
entire study, experienced therapists monitor the patient’s progress. Finally, multiple 
precautions are taken to ensure that patients are aware that the decision to participate does 
not have any effect on their treatment progress. Consequently, the burdens and risks 
associated with this study are low or even non-existent.  
 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
10 of 57 
1. INTRODUCTION AND RATIONALE 
Forensic mental healthcare is a complex branch of care which takes place at the intersection 
of psychiatry and law [1-3]. In forensic psychiatry, patients that show aggressive or sexual 
delinquent behaviour and have one or more concurrent psychiatric disorders are treated. 
Both in- and outpatients are often treated mandatory, as part of a sentence, which can result 
in overall low treatment motivation [4, 5]. Additionally, a relatively large share of the forensic 
psychiatric patient population has low cognitive and adaptive skills and often suffer from 
multiple psychiatric disorders [6-9]. These factors highlight the complexity of treating forensic 
psychiatric patients. While treatment, guided by for example the Risk-Need-Responsivity 
model, has resulted in lower recidivism rates, there numbers are not as low as desirable [10-
12]. Furthermore, when looking at the use of cognitive behavioural therapy (CBT) in this 
group, effectiveness rates for CBT for aggression are lower than those for treatment of for 
example anxiety [13, 14]. Besides these complicating factors, there also is a shortage of 
healthcare providers, while there also is an increase in forensic psychiatric patients that 
require treatment [15]. In general, it becomes clear that there is room for improvement in 
treatment for these vulnerable patients.  
 
In clinical practice, eHealth interventions are used to address the aforementioned 
challenges. The past ten years, Dutch forensic psychiatric organizations have especially 
invested in internet-based interventions, with the expectation that these interventions can 
increase efficiency and quality of forensic mental healthcare [16, 17]. Internet-based 
interventions are often based on evidence-based treatment models such as CBT and present 
the user with knowledge and skills via multiple lessons with e.g. multi-media such as videos, 
written assignments and short texts [18]. While users can work on these digital interventions 
independently, the therapist can provide written feedback on the assignments. Over the past 
years, different types of these interventions have been introduced and used in Dutch forensic 
mental healthcare [19, 20]. Existing internet-based interventions are focused on e.g. 
aggression, offense scripts and prevention plans, addiction, or violence in relationships [21]. 
Even though these digital interventions have been integrated in forensic mental healthcare 
for almost ten years, their effectiveness in Dutch outpatients has never been studied. In line 
with this, a scoping review on technology in treatment of offenders has shown that these 
interventions are also hardly evaluated in other countries [16]. The very few evaluation 
studies that were conducted focused on inpatients in for example prisons or clinics, while a 
large share of the patient population is treated at outpatient clinics. Consequently, this very 
complex and vulnerable target group is underrepresented in intervention research. 
 
Even though there is an obvious and urgent need for more research on the 
effectiveness of these  interventions in forensic mental healthcare, existing evidence 
suggests that this form of eHealth might have advantages for forensic patients. In the 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
11 of 57 
aforementioned review, 13 studies evaluated effectiveness of internet-based interventions in 
experimental designs [16]. Of these studies,10 showed that these interventions were at least 
as effective as standard care or more effective than a waiting list condition, while 3 studies 
found no evidence for effectiveness. This means that the most of the few existing studies 
have shown the effectiveness of these interventions in treatment of offenders. In addition to 
that, systematic reviews on digital interventions in mental healthcare in general show 
comparable findings. For example, if used well, internet-based interventions focused on a 
broad range psychiatric and somatic disorders result in comparable outcomes as face-to-
face interventions [22, 23]. Additionally, qualitative research with therapists and patients has 
identified multiple experienced benefits for patients. Examples are receiving better and more 
tailored information about the problematic behaviour via multimedia such as videos or 
assignments, attributing change to one’s own work on the intervention as opposed to just the 
therapist, and acquiring more tools on dealing with aggression by means of tips and 
assignments [19, 21]. Consequently, while earlier research suggests that these interventions 
might work for forensic mental healthcare as well [16], there is a need to study them in Dutch 
forensic outpatient care, since earlier findings cannot simply be generalized to this complex 
and highly specialized branch of care. Additionally, internet-based interventions are often 
studied in a fully online way and separately from treatment, while in practice, they are used in 
a blended way, i.e. integrated in treatment [24]. 
 
While no robust effectiveness studies have been conducted on the use of internet-
based interventions in forensic outpatient care, earlier research has provided several useful 
insights and recommendations. First, a straightforward recommendation is to study the 
effectiveness of internet-based interventions in clinical practice to gain insight into whether 
they work [17, 19]. However, merely gaining insight in effectiveness does not fully answer all 
questions that arise from research and practice. Consequently, a second recommendation is 
related to the use of these interventions. A recent study on implementation of these internet-
based interventions in forensic outpatient care has shown that adherence to them is low: 
most patients don’t complete all lessons of a intervention and a large share drops out after 
one session [21]. While an important reason for this is that therapists forget to introduce 
these modules to patients and do not integrate and discuss them in treatment sessions, 
explanations for non-adherence might also be found in patient characteristics. This highlights 
the importance of research that determines if and how we can predict if a patient drops out of 
the intervention. A third recommendation is related to the ‘what works for whom’ question. 
Therapists indicate that internet-based interventions do not seem to fit the entire forensic 
psychiatric population, but can be of added value for some patients. However, this is merely 
based on clinical experiences and subjective estimations: there has not been research on 
this topic yet. In other words: up until now, there is no clear explanation of why an 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
12 of 57 
intervention might work for one, but not for another patient. Socio-demographics do not 
provide satisfactory answers to questions regarding what works for whom. Research shows 
that engagement might be a novel and useful factor that might provide insight into why 
interventions work for specific patients [25, 26]. Engagement refers to the extent to which 
someone is involved or occupied with something, and is often related to a positive outcome. 
In relation to eHealth, it can serve as a predictor for adherence and engagement [25, 26]. 
However, the role of engagement in internet-based interventions has not been studied in 
forensic mental healthcare. Fourth, even if internet-based interventions are shown to be 
effective, it has proven to be very challenging to implement them in clinical practice. 
Research on the implementation of these interventions in forensic mental healthcare has 
indeed shown that implementation is suboptimal [21]. Qualitative, in-depth research into 
experiences and needs of not only patients, but also therapists is required to gain more 
insight into implementation barriers and potential activities.  
 
It is clear that there is a pressing need for research into if, why and for whom 
internet-based interventions work. Qualitative research has highlighted several potential 
benefits of internet-based interventions that require further research [19, 21]. These 
outcomes were mostly related to aggression regulation, since this is an important and large 
part of outpatient treatment. In general, therapists did see multiple advantages, but indicated 
that they did not expect any major improvements due to these interventions: they were 
mostly expected to slightly bolster the effectiveness of existing treatment, mostly by providing 
patients with more tools to work on their treatment independently. Because of this, changes 
might be attributed to the patient’s own effort as opposed to that of the therapist, which could 
benefit self-efficacy towards behaviour change. Additionally, internet-based interventions 
offer multiple ways of acquiring knowledge and skills that are tailored to the target group, 
such as multi-media, written assignments and peer-stories. This is done to accommodate 
different types of learning that better fit the patient than face-to-face treatment sessions and 
can thus also increase the patient’s confidence in regulating their aggression. Furthermore, 
therapists also expected that these intervention can positively affect treatment readiness of 
patients because they can support the patient in gaining insight into their problems and 
defining their own goals. Being responsible for their own treatment by working on an internet-
based intervention individually, and gaining more insight into their own behaviour, might 
influence their sense of ownership and motivation to work on their treatment. Therapists, 
patients and researchers expect that this potential increase in self-efficacy and treatment 
readiness could facilitate actual behaviour change: aggression might decrease due to the 
independence, knowledge, insights and skills that are acquired by means of these 
interventions [17]. This can result in decreased risk for recidivism, reflected in an improved 
score on risk assessment instruments. Finally, it is also expect that, if used well, these 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
13 of 57 
interventions might result in more efficient treatment because part of face-to-face treatment 
can be replaced by the internet-based intervention [21, 27]. However, for all of these 
expectations goes that they are based on qualitative research.  
 
Besides potential advantages of internet-based interventions, there are also critical 
views on these interventions. These are partly related to an expected misfit between the 
mostly language-driven interventions and relatively low educated and generally unmotivated 
patient population [21, 27, 28]. In general, therapists identify a broad range of improvements 
of these interventions, related to their content, the design, and the integration in clinical 
practice [21]. Robust evaluation research is required to investigate whether these internet-
based interventions are of added value for forensic outpatient care and to answer questions 
about adherence, engagement, effectiveness and implementation barriers 
 
The present study aims to fill the current gap in research on internet-based 
interventions in forensic psychiatric outpatient care. By evaluating an existing intervention 
that is already used in practice by means of a robust randomized controlled trial design, this 
study will have high clinical relevance and impact on practice. We will study the existing, 
evidence-based internet-based intervention ‘Aggression’, which contains 10 lessons and thus 
takes 10 weeks. Furthermore, our quantitative and qualitative results will provide valuable 
points of improvements for the content and design of the intervention and for its integration in 
forensic outpatient care. This type of research is especially relevant for forensic psychiatric 
patients: a vulnerable yet underrepresented target group that requires better, more fitting and 
evidence-based treatment to increase their quality of live, prevent recidivism and thus also 
protect society. 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
14 of 57 
2. OBJECTIVES 
 
Primary objective  
The primary objective of this study is to investigate whether the addition of the internet-based 
intervention ‘Aggression’ to forensic psychiatric outpatient treatment as usual (TAU) results in 
better treatment outcomes on the short and long term than patients that only receive TAU. 
 
Secondary objectives 
The secondary objectives are: 
1. To determine if there is more improvement in improvement regulatory emotional self-
efficacy, treatment readiness, and aggression in the experimental than control group 
during, directly after completing, and three months after completing the internet-
based intervention ‘Aggression’.  
2. To determine if there is more improvement on dynamic risk in the experimental than 
the control group after completing the internet-based intervention ‘Aggression’. 
3. To determine if the number of face-to-face treatment contacts over a period of 37 
weeks is lower in the experimental than the control group. 
4. To determine if the level of engagement with the intervention  measured at the 
beginning, halfway and at the end of the internet-based intervention Aggression, and 
the adherence to the intervention, is a predictor for effectiveness.  
5. To map experiences with the internet-based intervention Aggression of forensic 
psychiatric patients that followed (part of) the intervention. 
6. To map experiences with the internet-based intervention Aggression of therapists that 
used the intervention in at least five patients. 
 
 
 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
15 of 57 
3. STUDY DESIGN 
3.1 Study design  
Even though internet-based interventions are being used on a large scale and for a fairly 
long time, there have not been any robust evaluation studies in forensic mental healthcare. 
There have been small non-scientific pilots in single organizations and several mostly 
qualitative studies on their implementation [19-21]. A randomized controlled trial (RCT) in 
which the intervention as an addition to treatment as usual (TAU) compared to a control 
group that only receives TAU is a suitable research methods to evaluate whether the addition 
of these types of internet-based interventions adds to the effectiveness of treatment [29]. 
RCTs are viewed as a suitable research method - even the golden standard - when studying 
effectiveness of interventions. In order to also gain insight into why this might be the case 
and what points of improvements are, qualitative data will be collected as well [30]. In this 
study, interviews with patients and therapists are conducted after data-analysis of the RCT to 
ask them about reasons for the (in)effectiveness of the internet-based intervention and 
implementation conditions. In this way, the perspectives of patients and therapists– both 
essential for successful use of internet-based interventions – are integrated in the study. 
Furthermore, a multicenter approach is used to increase the feasibility of the study and to 
ensure that findings are not just based on one outpatient clinic and thus are more 
generalizable.  
 
Consequently, to investigate if the use of the internet-based interventions Aggression is of 
added value for in-person treatment of forensic psychiatric outpatients, a multicenter, non-
blinded, parallel groups randomized controlled trial design is used. 
  
3.2 Setting of the study 
This multicenter study will take place in four Dutch forensic psychiatric organizations that 
offer forensic psychiatric outpatient care: Stichting Transfore, Kairos, de Woenselse Poort 
and GGZ Noord-Holland Noord.  
 
This study is initiated and will be coordinated by Stichting Transfore, an organization that 
has the goal to be frontrunner in developing and providing eHealth in forensic care. Transfore 
has participated actively on the development of multiple Minddistrict modules and other 
eHealth interventions. Additionally, Transfore has been inducting multiple implementation- 
and evaluation research, to – among other things – internet-based interventions. GGZ 
Noord-Holland-Noord started experimenting with internet therapy in 2001. Nowadays, 
multiple applications of eHealth are integrated in treatment methods. Together with other 
GGZ-providers, GGZ Noord-Holland-Noord developed multiple (forensic) modules for the 
Minddistrict platform. De Woenselse Poort is part of GGzE and is also a frontrunner on 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
16 of 57 
innovation in (forensic) mental healthcare. Many of their projects are focused on eHealth: 
Minddistrict, mobile apps and virtual reality. Kairos is part of Pompestichting which has been 
inducting research on multiple areas for many years. Their main focus lies on technology and 
innovation, and other ways to improve treatment methods.  
3.3 Overview of the study design  
A schematic overview of the participant flow throughout the RCT, from screening up until 
follow-up assessment, is provided in Figure 1. The study design is briefly explained after the 
figure; a more detailed description of the procedure is provided in section 8.3.  
 
The flow of the participants throughout the RCT is visualized in Figure 1. The first step, 
therapists will first screen the patients on suitability when they are first discussed in a 
‘multidisciplinary consultation’ [multidisciplinair overleg; MDO]. Once the patient has been 
deemed suitable for participation, the therapists waits 8 weeks before introducing the study 
to the patient and asks for their consent to be contacted by a researcher. The main reason 
for not directly informing the patients during their intake or initial treatment sessions, is that 
patients already receive a lot of information at the beginning of their treatment, and thus often 
feel overwhelmed. Additionally, in practice, Minddistrict is often introduced later in the 
treatment process, because it often takes time to build a good therapeutic relationship, which 
is seen as a necessary requirement for starting with blended treatment, especially in a 
population known for its low treatment motivation.  
 
If the patient agrees to be contacted by a researcher for further information about the 
study and possible inclusion, they will be contacted by phone, videoconferencing or in-
person, depending on the preference of the patient. The researcher will inform them about 
participation and the consequences on being randomized to either the control or 
experimental condition. The researcher will clearly communicate that patients who are 
assigned to the control condition can still use an internet-based intervention after the study is 
finished. Furthermore, the researcher will clearly indicate that the decision to participate or 
drop out of the study will not affect their treatment or evaluation of their therapist. After 
informing the patient by means of a conversation and folder, the researcher will ask for their 
informed consent in participating. Next, the patient fills out the baseline assessment (T0) and 
is randomized to the control condition - which comprises forensic psychiatric treatment as 
usual (TAU) for aggression regulation, or the experimental condition, in which the internet-
based intervention ‘Aggression’ is used as a supplement to TAU.  
 
Patients will start with the Aggression module after approximately 12 weeks: this might 
exceed to a maximum of 14 weeks, if the patient for example is not able to start yet or 
misses several treatment sessions – which happens often in forensic mental healthcare.  
Participants are assessed four times throughout the study. They are first assessed at 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
17 of 57 
baseline (T0), i.e. in week 12. A mid-treatment assessment (T1) is conducted when 
participants are supposed to be halfway throughout the internet-based intervention, i.e. 6 
weeks after baseline, so in week 17. After completing the intervention, post-treatment 
assessment will be conducted. If participants complete a lesson per week, the intervention 
should be finished after 11 weeks. However, in practice, this often is not the case, for 
example because the patient misses treatment sessions or other psychosocial crises can 
arise that have priority. Consequently, post-treatment assessment will take place 14 weeks 
after baseline, i.e. in week 25. Finally, a follow-up assessment will be conducted. This 
assessment will take place 3 months after T2, so during week 37 of the entire study. 
 
At all assessment points, patients will fill out three self-report questionnaires via Qualtrics: 
Regulatory Emotional Self-efficacy (RESE) scale [31], the Corrections Victoria Treatment 
Readiness Questionnaire (CVTRQ): a validated self-reported questionnaire [32], and the 
Dutch version of the Aggression Questionnaire (AVL-AV) [33, 34]. Participants in the 
experimental condition also fill out the the TWente Engagement with Ehealth and 
Technologies Scale (TWEETS) [35] at the beginning, halfway and at the end of the internet-
based intervention Aggression. Furthermore, the scores of the risk assessment instrument 
‘Forensisch Ambulante Risico Evaluatie (FARE) [36] - which is part of standard procedures – 
will be used to assess change in dynamic risk factors in all participants .  
 
Additionally, a qualitative interview study with 20 patients will be conducted to gain insight 
into their experiences, perceived positive and negative points, and points of improvements 
regarding the internet-based intervention. These participants will be randomly selected from 
the sample that started the experimental condition by means of a random number generator. 
These interviews will take place within 5 weeks after post-treatment assessment (T2), i.e. 
during week 25-30 of this study.  Finally, all participating therapists will be sent an open-
ended questionnaire to ask them similar questions about experiences, perceived positive and 
negative points, and points of improvements. These questionnaires will be send at the end of 
the data collection period, i.e. in Q2 2023. 
 
 
 
 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
18 of 57 
Figure 1: Participant flow throughout the RCT 
Screening during MDO (therapist)
Week 0
Inclusion & informed consent (researcher)
Week 8 – 11 
Randomization (researcher)
Week 11
Start TAU
Week 12
Start TAU + Aggression + 
TWEETS
Week 12
 Mid-treatment assessment 
(T1) + TWEETS
Week 17
Mid-treatment assessment 
(T1) 
Week 17
Post-treatment assessment 
(T2) + TWEETS
Week 25
Post-treatment assessment 
(T2) 
Week 25
Follow-up assessment (T3) + FARE
Week 37
Post-intervention interviews 
(n=20)
Week 25-30
Informing patient about study (therapist)
Week 8 – 10
Baseline assessment
Week  11 
 
 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
19 of 57 
4. STUDY POPULATION 
4.1 Population (base)  
Description of the population base 
Forensic mental healthcare is focused on people who display aggressive or sexual 
delinquent behaviour that led or could lead to offending and simultaneously suffer from at 
least one psychiatric disorder, for example schizophrenia, alcohol abuse, antisocial 
personality disorder or post-traumatic stress-disorder [1-3]. Due to this combination of 
offending and psychiatric disorders, forensic mental healthcare - or forensic psychiatry - 
takes place at the intersect of mental healthcare and the law. Forensic mental healthcare 
encompasses treatment of both in- and outpatients. Inpatients reside in clinics with different 
levels of security, ranging from very high levels, where patients are not or almost never 
allowed to go on leave, to medium or low levels, where patients have more freedom and 
independence [37]. While relatively many forensic psychiatric patients receive treatment at 
inpatient clinics, a large share of the patient population comprises outpatients who live at 
home and receive treatment at an outpatient clinic [38]. Some patients receive treatment 
voluntarily and not as part of a sentence, for example when a general practitioner refers them 
to forensic mental healthcare because it fits their aggression regulation problems better than 
regular mental healthcare. Regardless of the differences between security levels, the main 
goal of forensic mental healthcare is always to prevent (re)offending and thus to protect 
society. Treatment of forensic psychiatric patients is viewed as challenging, which can be 
partly explained by characteristics of the patient population: most forensic psychiatric 
patients are not that motivated for their treatment [4, 5]. Furthermore, they often have 
followed no or very little education and many patients have low literacy levels [6-8]. 
Additionally, co-morbidity is common: patients often have multiple psychiatric disorders [9]. In 
general, the forensic psychiatric patient population is very heterogeneous, but the main 
common factor is that for all patients, there is a risk on (re)offending.  
 
This study is focused on people that receive forensic psychiatric outpatient care with a 
primary focus on aggression. To increase the feasibility and generalizability of this study, 
inclusion takes place at four large forensic psychiatric organizations that all treat patients at 
multiple outpatient clinics.   
 
 
 
 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
20 of 57 
4.2 Inclusion criteria 
In order to be eligible to participate in this study, a subject must meet all of the following 
criteria: 
1) The patient is 18 years or older 
2) The patient is treated at an outpatient clinic 
3) The patient receives one-on-one treatment 
4) During the intake, improvement of aggression regulation has been selected as one of 
the treatment objectives 
5) The patient indicates that they are able to read and write simple texts  
6) The therapist responsible for treatment of the patient indicates that participating will 
not result in any harm for the patient 
7) The patient voluntarily consents to participation 
4.3 Exclusion criteria 
A potential subject who meets any of the following criteria will be excluded from 
participation in this study: 
1) The patient has a current psychosis  
2) The patient resides in any type of psychiatric inpatient clinic – this can be a 
forensic, but also another type of clinic  
3) The patient receives group treatment focused on aggression regulation 
(specifically: AR [aggressieregulatie] op maat) 
4) The patient is analphabetic, i.e. being unable to read and write 
5) The responsible therapists identifies any other valid reason for exclusion 
 
As can be seen in the in- and exclusion criteria, the responsible therapist is actively involved 
in determining whether participating in the study is safe and possible for the patient. This will 
first be discussed during the MDO. If, after several treatment sessions, the therapist identifies 
any reasons for exclusion, this will be communicated to the researcher and the patient will 
not be invited to participate.  
 
Because the internet-based intervention ‘Aggression’ is transdiagnostic and is not focused 
on one psychiatric disorder, no psychiatric disorders were used to define the target group. 
This is also not feasible because forensic psychiatric patients are not enrolled in forensic 
mental healthcare for a diagnosis, but because of their delinquent behaviour. Additionally, 
many patients have multiple diagnosis: co-morbidity is relatively high in this target group. By 
keeping the in- and exclusion criteria relatively broad, patients from a large sample can be 
included, increasing the feasibility of this study.  
 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
21 of 57 
4.4 Sample size calculation 
This sample size calculation is conservatively based on the ability to detect at least a 
moderate effect of Cohen’s d =.50 on the primary outcomes at follow-up (T3). A power 
analysis was conducted with G*power. The analysis was conducted with a moderate effect 
size of 0.5, a β-power of .8, a p-value of 5%, and an independent two-sided t-test as method 
of analysis. This analysis showed that 64 patients need to be included in each condition. This 
means that a total of 128 patients needs to be included. Accounting for a drop-out of 20%, 
the goal for inclusion is set at a total of 154 patients. It is important to note that participants in 
the experimental condition are considered as a ‘drop-out’ when they don’t fill out the 
questionnaires at all measuring moments, not when they stop using the intervention. In other 
words: patients can stop using the intervention. In other words: patients can stop using the 
intervention, but can still complete their participation in the study if they fill out all 
questionnaires. Adherence to the intervention is not the same as drop-out from the study, 
since it is a different concept. The main reason for this is that earlier research has shown that 
patients often stop using these interventions before all lessons are completed, so it is not 
realistic to expect that all patients fully complete the intervention. In order to account for this, 
the number of completed lessons will be included as a moderator for effectiveness in the 
analyses to investigate whether adherence is related to effectiveness. Additionally, a sub-
goal of this study is to determine whether engagement is a predictor of adherence and 
effectiveness to incorporate non-adherence to the intervention in the study. The intended 154 
patients will be included at four different organizations that all have multiple forensic 
outpatient clinics.  
 
To the best of our knowledge, there are no other studies on the evaluation of internet-
based interventions in forensic outpatient care. Consequently, the effect size of .50 is based 
on similar studies. A recent systematic review on internet-delivered CBT for depression and 
anxiety disorders identified a mean effect size of .80 at follow-up after 3 to 36 months when 
looking at the post-test difference between treatment and control conditions [39]. However, in 
forensic mental healthcare, lower effect sizes are expected due to the complexity of 
improving treatment outcomes in this population. To illustrate: CBT appears to be more 
effective for treatment of specific disorders such as anxiety or depression than for treatment 
of aggression [13, 14]. Consequently, the more moderate effect size of the current RCT is set 
on .5.  
 
 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
22 of 57 
 
4.5 Feasibility of the study 
As is the case with any RCT in clinical practice, a potential barrier is the inclusion of enough 
participants. Especially in forensic mental healthcare, this can be problematic. However, 
there are multiple reasons and measures that are taken to increase the chance of successful 
completion of this RCT.  
 
First of all, the intervention that is investigated in the RCT is already introduced in 
practice, and thus is not new. Consequently, therapists do not require training, already have 
experience in using similar interventions, and ‘teething problems’ that often occur when 
introducing new interventions in practice will not occur because these interventions have 
been used for over ten years.  
Second, implementation research has shown that – while a large share of therapists don’t 
actively use these interventions, there are several therapists that do use these interventions 
relatively often and thus are quite experienced. In this study, therapists that have experience 
with and thus are able to successfully use these internet-based interventions will be included. 
Additionally, qualitative research into needs and wishes from clinical practice regarding 
technology in forensic mental healthcare has shown a high need for evaluation research of 
these internet-based interventions. [17, 19]. This finding, combined with experiences from 
clinical practice, shows that therapists see the urgency and relevance of this study, which is 
expected to positively contribute to their intrinsic motivation to participate in this study. Third, 
a common problem with these types of interventions is that they are not used as often as 
might be expected by most therapists, even though they express the intention to do so and 
are able to identify multiple potential benefits [21]. The main reason for this ‘intention-
behaviour gap’ is that therapists often forget to introduce these modules to patients, and if 
they are introduced, they often don’t remember to discuss them during treatment sessions. 
This problem can be addressed by regularly reminding therapists to introduce and keep on 
using the intervention [24]. Consequently, during data collection, researchers and the 
secretary of the clinic will send regular reminders to participating therapists via e-mail and 
phone. Also, the study and accompanying intervention will be integrated as a fixed topic in 
intakes and multidisciplinary meetings [multidisciplinair overleg, MDO]. Fourth, while this is 
not as common, it might be that patients do not see the added value of the internet-based 
intervention. This will be accounted for by providing patients with a flyer and short information 
video on the content of the intervention. The content of these information materials will be 
tailored to the reading skills and preferences of patients. Fifth, another barrier might be that 
patients are not motivated to participate in data collection. To account for this, patients in the 
experimental and control condition receive a financial compensation. Experience with other 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
23 of 57 
studies with this target group has shown that this can be an additional motivator for patients 
to participate. In all communication, these rewards will be mentioned to ensure that the 
patients are aware of the fact that they will be compensated for their invested time. To 
prevent that patients drop out during data collection, they will receive one voucher halfway, 
and one after completing all questionnaires. Additionally, when patients don’t fill out 
questionnaires at one of the measurement moments, they will be reminded via e-mail and if 
that is not effective, the researcher will contact the therapist, who will remind the patient of 
the questionnaires. Sixth, in line with this, to prevent drop-out due to high time investment, 
the four selected questionnaires are kept as short as possible and filling them out will only 
take a couple of minutes. Also, data from risk assessment instruments that are already being 
administered is used to prevent additional time investment from patients and therapists. 
Seventh, as is the case with any internet-based intervention, patients might not be fully 
adherent, meaning that they do not complete the entire intervention. While this is a realistic 
risk, this does not threaten the feasibility of this study, because this is also an interesting and 
relevant result. If, even if therapists and patients receive much support in using the 
intervention, patients still do not complete the intervention, this has implications for its 
application in clinical practice. Additionally, adherence to the intervention will be used as a 
predictor for effectiveness to investigate if there is a dose-response relationship. It is 
important to emphasize that patients that do not fully complete the intervention, but still fill out 
all questionnaires, are not considered as a drop-out. Finally, it can be challenging to gather 
large sample sizes in forensic mental healthcare. To overcome this, data collection will take 
place in four organizations, that all have multiple outpatient clinics where patients can be 
included. The feasibility of including approximately 38 patients per organization – which all 
have multiple outpatient clinics - is elaborately discussed with representatives from the 
organizations that also participate in the project team that coordinates this study. 
Furthermore, initial contact with therapists has underlined the expectation that the intended 
number of participants can be reached. 
 
To conclude: while any RCT on internet-based interventions in clinical practice is 
challenging, the set-up of this project is based on findings of and recommendations arising 
from research on implementation of these interventions in forensic mental healthcare. 
Additionally, several researchers with ample experience with RCTs on internet-based 
interventions and research in forensic mental healthcare are involved in this project, and the 
study design is based on their experiences and lessons learned from earlier projects. It is 
expected that, especially due to the fact that these interventions are not new for therapists 
and that they will receive ample support from the researchers, the intended number of 
participants will be reached. 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
24 of 57 
5. TREATMENT OF SUBJECTS 
5.1 Investigational product/treatment 
5.1.1 
Intervention: Aggression 
In this study, the internet-based intervention ‘Aggression’ is evaluated. This intervention is 
part of the eHealth-platform Minddistrict. Minddistrict is a platform that offers over 300 
different internet-based interventions, primarily targeted at mental healthcare. The platform 
contains a broad range of interventions, focused on for example attention deficit hyperactivity 
disorder (ADHD), mindfulness, autism, healthy lifestyle, and depression. These interventions 
are used by over 250.000 people in Europe. The entire Minddistrict platform is certified as 
ISO 27001:2017 and NEN 7510-1:2017 and is certified as a medical device under the Dutch 
Law on Medical Devices.  
 
All internet-based interventions of Minddistrict via the ‘Intervention Mapping’ approach 
[40]. Intervention Mapping is as a planning approach which uses theory and evidence as 
foundations for taking an ecological approach to intervening in health problems, following a 
systematic approach. Minddistrict also uses behavioural change techniques, or BCTs. These 
techniques are specific components designed to change behaviour, such as goal-setting, 
feedback or behavioural practice and rehearsal. Furthermore, persuasive design methods 
are applied. By incorporating persuasive features such as personalization, reminders or 
rewards, the chances of people being adherent to the internet-based intervention and 
consequent attitude and behaviour change increase. By combining BCTs and persuasive 
features, attention is paid to the content and design of Minddistrict interventions.  
 
This RCT is focused on the internet-based intervention ‘Aggression’. This intervention is 
used a blended way, i.e. integrated in face-to-face treatment. In this intervention, patients 
learn to deal with conflict and triggers in a constructive way. The intervention is focused on 
three main objectives: (1) increasing the motivation to change, (2) acquiring skills for dealing 
with conflict, and (3) breaking the cycle of aggression by providing knowledge on situational, 
emotional, cognitive and physical triggers. The module consists of 10 lessons:  
1. Motivation – where are you now? In this lesson, the patient is introduced to the 
module, amongst other things by means of a video with a therapist and an audio 
fragment of an expert by experience. The advantages and disadvantages are 
discussed by means of open-ended questions.  
2. Where do you want to go? In this lesson, the patient is asked to think about the 
future with and without aggression. They are asked about their goals by means of 
open-ended questions, on which feedback is provided by the therapist. Finally, the 
patients works on a diary by means of a daily reminders in which they can monitor 
their own aggressive behaviour and accompanying triggers.  

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
25 of 57 
3. Circle of aggression. In this lesson, the patient is informed about the circle of 
aggression by means of written text, illustrations and a video. Next, the patient thinks 
about their own risk situations and accompanying triggers.  
4. Thoughts. In this lesson, the patient receives information about convictions about 
their past and thinking styles by means of videos, written texts and short 
assignments. Attention is paid to different perspectives, convictions, thinking errors 
and the challenging of thoughts. 
5. Emotions. This section starts with information on emotions by means of text and 
animation. Next, different gradations of emotions are explained, using the metaphor 
of (over)cooking pans. Via videos, the patient learns about emotions that can lead to 
aggression.  
6. Bodily sensations. In this lesson, the patient learns more about bodily sensations 
and risky behaviours that are related to aggression by means of assignments. 
Additionally, attention is paid to relaxation exercises to deal with bodily sensations.   
7. Techniques for self-control. In this lesson, the patient works on their coping skills 
and self-efficacy for preventing aggression. The patient leans about potential coping 
skills such as avoidance, stepping out of the situation and time-outs by means of text 
and videos with experts by experience. 
8. Asking for help. In this lesson, the patient learns how to ask for help  from others 
and gains insight via text and video on how to deal with others. Furthermore, the 
patient is encouraged to practice with these coping strategies in real life.  
9. Assertiveness. In this lesson, the patient gains knowledge about reacting in sub-
assertive, assertive and aggressive ways. By means of quizzes, text and videos, the 
patient learns about and can practice with coping skills to deal with aggression.  
10. Relapse-prevention. In the final lesson, the patient learns about relapse prevention 
plans via texts and videos with experts by experience. The patient creates their own 
relapse prevention plan and makes plans for their future.  
As can be seen, each lesson consists of multiple components, i.e. written text, assignments 
and videos with therapists and/or experts by experience. After completing each lesson, the 
patient is rewarded by means of a ‘milestone’. Furthermore, after each lesson, the therapist 
sends written feedback on the assignments to the patient. Because ‘Aggression’ is a blended 
intervention, each lesson is briefly discussed in an in-person treatment session. This means 
that generally, participants work on one lesson per week. See the Appendix for screenshots 
of ‘Aggression’.  
 
The internet-based module Aggression was developed in 2012 and has been used for 
approximately 10 years in different branches forensic mental healthcare. Recently, the 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
26 of 57 
intervention has been updated cosmetically to ensure it is in line with current insights in 
terms of content and design. However, as is the case for most internet-based interventions in 
Dutch mental healthcare, it hasn’t been evaluated on effectiveness. Several years ago, a 
mixed-methods study on the use of Minddistrict modules in Dutch forensic psychiatric 
outpatient care was conducted [21]. This study showed that Aggression was one of the most-
used interventions of the Minddistrict platform within this organization – Transfore. However, 
interviews and data on usage showed that there is still much room for improvement in terms 
of uptake. One of the main reasons for this was that while therapists were able to identify 
benefits and expressed an intention to use the intervention, they often did not think about 
using the intervention because it was ‘not in their system’ . Additionally, if a patient started, 
therapists often did not integrate the intervention in treatment because of the same reason: 
they forgot about it. In order to increase the uptake of ‘Aggression’ in this study, therapists 
will be supported in remembering to introduce and keep on discussing the intervention with 
the patient. Finally, in qualitative research, therapists mentioned that limited evidence of 
these interventions was a barrier for their usage: if they do not know if the intervention works 
and is evidence-based, they indicated that they are less prone to use it [19, 21]. In line with 
this, therapists expressed a need for evaluation studies on internet-based interventions in 
treatment of forensic psychiatric patients to overcome this barrier. Consequently, based on 
these qualitative findings, it is expected that especially those therapists that require more 
research are intrinsically motivated to participate – which is underlined by observations from 
conversations with therapists that are eligible to participate.  
5.1.2 
Treatment as usual  
In this study, the control group will receive treatment as usual. In general, this refers to any 
form of psychotherapy that is delivered by a licenced psychologist to a patient in a one-on-
one setting. The main focus of treatment as usual should lie on aggression regulation. 
Additionally, patients should receive individual, one-on-one treatment and should not 
participate in group treatment of aggression during data collection.  
 
Because this is a multi-centre study, there are minor differences between the treatment as 
usual for aggression between the participating clinics and even between participating 
therapists. To illustrate, some clinics use the ‘AR op Maat’ program, while others use ‘Grip op 
Agressie’ as the main guidelines for their aggression treatment. While there are minor 
differences in the form of these types of treatment, they are all based on the same underlying 
principles and models.  All forms of treatment in forensic psychiatry target dynamic risk 
factors that have been identified by means of risk assessment instruments. In general, 
treatment of patients with aggressive behaviour is always focused on the improvement of 
self-regulation to reduce the chance of (re)offending. This is achieved by means of elements 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
27 of 57 
from cognitive behavioural therapy, often combined with techniques from occupational 
therapy.  
 
Furthermore, because the forensic psychiatric patient population is very heterogeneous - 
with much variation in e.g. diagnoses and type of offense, there is often also variation in the 
length of treatment and the exact way it is delivered to individual patients, even when 
following the same protocol. This is done to ensure that treatment is tailored – in forensic 
terms, responsive – to the individual patient [41]. Consequently, it is unfeasible and 
undesirable to offer exactly the same TAU to each patients since this is not in line with 
guidelines and treatment models that are used in practice. Despite the heterogeneity of the 
patient population and treatment, it is expected that there will be no major differences in 
outcomes between the different types of treatment because they are all based on the same 
principles. Forensic treatment in the Netherlands is always focused on risk assessment and 
management, grounded in evidence-based treatment protocols such as cognitive 
behavioural therapy, and underpinned by the risk-need-responsivity model. Additionally, all 
included patients in this study will receive in-person treatment with weekly sessions with their 
therapist, focused on aggression regulation, and underpinned by principles of cognitive 
behavioural therapy.  
 
To summarize: while in general terms, treatment of all patients is based on similar 
principles and guidelines, there will be minor differences in the way in-person treatment will 
be delivered between patients in both the control and experimental conditions, this is part of 
the ecological validity of this study and is not expected to influence results because of the 
randomization. 
 
 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
28 of 57 
6. INVESTIGATIONAL PRODUCT 
Not applicable. Minddistrict and the ‘Aggression’ module are already mentioned and 
explained in chapter 5. 
 
7. NON-INVESTIGATIONAL PRODUCT 
Not applicable. This study won’t be using any additional products. 
7.1 Name and description of non-investigational product(s) 
Not applicable. 
7.2 Summary of findings from non-clinical studies 
Not applicable.  
7.3 Summary of findings from clinical studies 
Not applicable. 
7.4 Summary of known and potential risks and benefits 
Not applicable. 
 
7.5 Description and justification of route of administration and dosage 
Not applicable. 
 
7.6 Dosages, dosage modifications and method of administration 
Not applicable. 
 
7.7 Preparation and labelling of Non Investigational Medicinal Product 
Not applicable. 
 
7.8 Drug accountability 
Not applicable. 
 
 
 
 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
29 of 57 
8. METHODS 
8.1 Study parameters/endpoints 
In this study, multiple types of data are collected to answer the six research questions. First, 
a short overview will be provided, after which the specific measurement instruments will be 
explained in more detail. 
 
The variables for the first research objective – focused on the improvement in regulatory 
emotional self-efficacy, treatment readiness and aggression – will be collected via short, 
validated self-report questionnaires that will be administered via Qualtrics. To answer the first 
research question, three primary outcome measures will be used. The main reason for 
selecting three outcome measures, is that this is the first RCT on internet-based 
interventions in forensic outpatient care [16]. Consequently, there are no other RCTs on 
which the choice for outcome measures can be based. The three main outcome measures 
are based on qualitative research, combined with insights of therapists and researchers [17, 
19, 21, 42]. However, based on these sources, it is not clear which of these outcome 
measures is best to identify change on the short and long term due to the intervention. The 
intervention is viewed as potentially effective if the experimental group improves more than 
the control group on at least one of the outcome measures on T1, T2, and/or T3. The 
intervention is viewed as effective in case of changes on all three outcome measures on T3. 
 
 For the second objective, data from risk assessment instruments that are already 
administered will be collected at baseline and follow-up. Since risk assessment with the 
FARE has to be performed twice per year, this form of data collection is in line with existing 
procedures and thus will not take up additional time of patient and therapist. For the third 
objective, therapists will be asked to indicate the number of treatment sessions that the 
patient participated in. Therapists can retrieve these data from their calendar, the electronic 
patient file or supported by the office assistants, the exact method depending on their own 
preference. Data for the fourth objective on engagement will only be collected from patients 
in the experimental condition. The TWEETS engagement questionnaire will be built into the 
internet-based intervention to ensure that it is filled out a the right point in the intervention, 
i.e. at the beginning, halfway and at the end. Another reason for this is that, if engagement is 
a predictor for effectiveness, Minddistrict will integrate the TWEETS in all interventions. The 
fifth and sixth objective will be addressed by means of qualitative data, collected by means of 
semi-structured interviews with patients and via questionnaires with open-ended with 
therapists. Finally, at baseline, patients are asked to fill out several questions on 
demographics, including their gender, age, marital and employment status and education 
level. Participants will also be asked if they received forensic and/or regular psychiatric 
treatment in the past. Additionally, at baseline therapists will be asked to provide information 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
30 of 57 
about the inpatient clinic in which the patient is treated, the main treatment goal, type of 
offense, and diagnoses. These data will be used to describe the sample and will be collected 
via Qualtrics.  
8.1.1 Main study parameter/endpoint 
Regulatory emotional self-efficacy – RESE  
A main advantage that is reported in qualitative studies on the use of internet-based 
interventions in forensic psychiatry, is that patients are working on part of their treatment 
independently. On top of TAU, internet-based interventions offer additional exercises and 
information to support the patient in dealing with their undesired aggressive behaviour. This 
is expected to provide them with more insights and tools to improve their aggression 
regulation than patients who do not follow an internet-based intervention outside of in-person 
treatment sessions. It is thus expected that the self-efficacy of patients that follow an 
additional internet-based intervention will increase more than in TAU. In other words: 
because they receive additional treatment via the intervention and work on it independently, 
their confidence in regulating their own emotions can increase.  
 
Consequently, the primary outcome measure of this study is regulatory emotional self-
efficacy. This construct is measured by means of the validated, 12-item Regulatory 
Emotional Self-efficacy (RESE) scale [31]. The RESE scale assesses self-efficacy in 
managing negative emotions (8 items) and in expressing positive emotions (4 items). 
Negative emotional self-efficacy refers to the perceived “capability to ameliorate negative 
emotional states once they are aroused in response to adversity or frustrating events and to 
avoid being overcome by emotions such as anger, irritation, despondency, and 
discouragement” [31]. Positive self-efficacy is defined as the perceived capability “to 
experience and to allow oneself to express positive emotions such as joy, enthusiasm and 
pride in response to success or pleasant events” [31]. Earlier research has demonstrated the 
validity and reliability of the RESE scale [31, 43, 44]. Because the RESE scale consists of 
only 12 items, it is easy to administer and does not require much effort of participants. 
Treatment readiness – CVTRQ  
Treatment readiness refers to the presence of factors that contribute to engagement with 
treatment and therapeutic change [12]. This concept is often used in forensic psychiatry and 
does not only cover motivation and willingness to change, but also responsivity, which refers 
to the suitability of the treatment program offered. It incorporates internal factors such as 
personality and cognitive-emotional features of the patient that are related to therapeutic 
change. Qualitative research has shown that therapists believe that, if used well, internet-
based interventions can contribute to treatment readiness because they might fit the needs 
and wishes of the patient, resulting in better responsivity to treatment. In other words: the use 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
31 of 57 
of internet-based interventions might increase the fit between the treatment and needs of the 
patient, resulting in higher treatment readiness. Additionally, the patient can gain new 
insights and skills individually as opposed to because of a therapist, which might contribute 
positively to their intrinsic motivation and willingness to change [4, 5]. 
 
Treatment readiness is assessed by the Dutch version of the Corrections Victoria 
Treatment Readiness Questionnaire (CVTRQ): a validated self-reported questionnaire [32]. 
The total scale consists of 20 items, divided into four subscales. The subscale Attitude and 
Motivation (AM) measures attitudes and beliefs about treatment programs and the desire to 
change. The scale Emotional reactions (ER) measures emotional responses to the 
individual’s offending behaviour. Offending beliefs (OB) refers to beliefs about personal 
responsibility for offending behaviour. The subscale Efficacy (EF) measures perceived ability 
to participate in treatment programs.  
 
Aggression – AVL-AV 
The main objective of forensic psychiatric treatment is to prevent (re)offending. 
Consequently, preventing aggression is an important goal of the entire treatment program. 
However, reaching this goal, even by means of long and intensive therapy has proven to be 
difficult. We did not select aggression as a primary outcome measure because a direct 
impact on changes in aggression might be too much to expect from an internet-based 
intervention. However, it might be that patients that the expected changes in self-efficacy due 
to the internet-based intervention result in actual behaviour change, so self-reported 
aggression is used as a secondary outcome. Self-reported aggression is assessed by means 
of the Dutch version of the much-used and validated Aggression Questionnaire (Aangepaste 
Versie van de Agressie Vragenlijst; AVL-AV) [33, 34]. The 12 items of the questionnaire are 
divided into four subscales: physical aggression, verbal aggression, anger, and hostility. The 
AVL-AV had been shown to be reliable and valid in earlier research and is easy to administer 
because it consists of only 12 items.  
8.1.2 Secondary study parameters/endpoints (if applicable) 
In order to gain more insight into if, how and why the internet-based intervention works, 
multiple other questionnaires are used. 
Risk assessment – FARE  
In treatment of all forensic psychiatric patients, risk assessment has to be conducted by 
means of evidence-based risk assessment instruments. In forensic psychiatric outpatient 
care, the Dutch standard is the Forensisch Ambulante Risico Evaluatie (FARE), version 2 
[36]. The FARE is not only used to estimate the risk on recidivism, but also to monitor 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
32 of 57 
changes in dynamic risk factors and risk of recidivism during treatment. Research into inter-
reliability and convergent validity has shown promising results [45]. The FARE is part of 
Dutch guidelines on risk assessment of forensic psychiatric outpatients. According to these 
guidelines, the FARE is administered twice a year, i.e. once each six months.  
 
In the FARE, 6 static and 11 dynamic risk factors are assessed. Static risk factors refer to 
‘unchangeable’ characteristics, such as age of the first offense. While they predict the risk of 
recidivism, they are not changeable by means of targeted interventions. Dynamic risk factors 
however are more related to the individual behaviour of the person and their social and living 
situation. Because these 11 factors are influenceable by interventions, they will be included 
in the analyses of this study. The 11 evidence-based dynamic risk factors that are 
incorporated in the FARE are: malfunctioning on education/work; financial mismanagement; 
delinquent social network; limited leisure activities; problematic (former) partner relationship; 
instable living situation; problematic substance use; limited impulse control; malfunctional 
coping skills; antisocial attitude; and rule-violating behaviour.  
 
Engagement – TWEETS  
Many studies have been conducted to identify why and for whom specific internet-based 
interventions work. Up until now, the use of sociodemographic factors as predictors has not 
provided answers to these questions. Engagement is a non-specific factor that can be used 
to predict effectiveness of and adherence to internet-based interventions [46]. Engagement 
refers to the extent to which someone is involved or occupied with something, and is often 
related to a positive outcome, such as effectiveness. In the context of digital behaviour 
change interventions, engagement contains an affective, behavioural and cognitive 
component [46]. The affective component is focused on positive or negative feelings towards 
the intervention and the way in which the intervention fits with a person’s identity: do people 
identify with the design of the technology and what it stands for? Behavioural engagement is 
focused on the routine that a user has regarding the use of an intervention and the extent to 
which it requires effort and is part of one’s daily life. The cognitive component of engagement 
refers to the extent to which an intervention supports the user in reaching their goals and is 
seen as useful for their behaviour change.  
 
In this study, engagement is measured with the TWente Engagement with Ehealth and 
Technologies Scale (TWEETS). The scale employs a definition of engagement that 
incorporates behaviour, cognition, and affect, and has been shown to have a good validity 
and reliability [35]. The TWEETS contains only 9 items – with three items per component of 
engagement - and has three slightly different versions: one for expected engagement, to be 
used when someone starts using an intervention, one for current engagement, to be used 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
33 of 57 
during the use of an intervention, and one for past engagement, to be used when a user has 
completed or stopped using an intervention. 
8.1.3 Other study parameters  
To describe the characteristics of the sample, information about treatment objectives, 
diagnoses, age, gender, and the inpatient clinic in which they are treated will be used. These 
data will be retrieved at T0 from the therapist by means of a Qualtrics questionnaire. In this 
way, the researchers do not need access to the personal patient file.  
 
 
Table 1: Overview of study parameters 
Variable 
Instrument 
Assessment point 
Invested time 
Regulatory 
emotional self-
efficacy  
Regulatory Emotional 
Self-efficacy (RESE) 
scale [31] 
T0, T1, T2, T3 
5 min. per assessment 
point, 20 minutes in 
total 
Treatment 
readiness 
Corrections Victoria 
Treatment Readiness 
Questionnaire 
(CVTRQ) [32] 
T0, T1, T2, T3 
5 min. per assessment 
point, 20 minutes in 
total 
Aggression  
Aggression 
Questionnaire [17, 
18]. 
T0, T1, T2, T3 
5 min. per assessment 
point, 20 minutes in 
total 
Risk assessment 
Forensisch Ambulante 
Risico Evaluatie 
(FARE) [36] 
T0 & T3* 
N.a. – part of standard 
procedures 
Engagement 
TWente Engagement 
with Ehealth and 
Technologies Scale 
(TWEETS) [35] 
T0, T1 & T2 (only 
in the experimental 
group) 
5 min. per assessment 
point, 15 min. in total 
(only for experimental 
group) 
Patients’ 
perceived 
effectiveness and 
points of 
improvement  
Semi-structured 
interviews 
T2 
Ca. 30 min 
Therapists’ 
perceived 
effectiveness and 
Open-ended 
questionnaires 
(Qualtrics) 
T2 
Ca. 15 min 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
34 of 57 
points of 
improvement 
* In this study, the FARE that is administered during the standard treatment trajectory are 
used to not interfere with standard procedures, meaning that the exact date of FARE 
administration can slightly differ from the T0 and T3 assessments. 
8.2 Randomisation, blinding and treatment allocation 
Because blinding is impossible due to the use of the internet-based intervention in treatment, 
this study is a non-blinded RCT. After a participant is screened and has given informed 
consent, they will be randomized by the principal investigator. The randomization will be 
supported by an online tool (https://sealedenvelope.com/). Participants are not randomized 
per participating organization or clinic because there are major differences in the number of 
patients treated by these clinics. Instead, once a patient is included, they receive a 
participant number, which will be aligned with a random sequence of treatment allocations, 
i.e. participants are either allocated to the intervention or control group. The first participant 
included in the study is allocated according to the first treatment allocation on the list, the 
second participant according to the second allocation, and so forth. 
 
Participating therapists are informed of the group the patient is allocated to and will 
explain this to the patient. If a patient is randomized into the control condition, the therapist is 
instructed to provide treatment as usual, and can be contacted by the researcher in case a 
patient did not fill out questionnaires at T0, T1, T2 or T3. If a patient is randomized into the 
experimental group, the therapist is asked to start with the intervention ‘Aggression’. 
Specifically, the therapist will shortly explain the goal, content and nature of the internet-
based intervention and explain to the patient that they will receive one lesson per week for 
the next 10 weeks. Additionally, the therapist will provide written feedback on the 
assignments of the patients. All participating therapists are trained in the use of these types 
of interventions and thus know how to provide feedback. To increase the feasibility of this 
study and reach the intended number of participants, therapists will receive additional 
support from researchers to remind them to introduce the intervention. The main reason for 
this is that implementation research on these types of internet-based interventions in Dutch 
forensic outpatient care has shown that, even though therapists have the intention to use the 
intervention, they often forget to introduce the intervention. Additionally, if they introduced the 
intervention, therapists often do not mention it in their treatment sessions. Consequently, 
participating therapists will be regularly reminded to introduce and keep on discussing the 
internet-based interventions with the patients in the experimental condition.  
 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
35 of 57 
8.3 Study procedures 
The flow of the participants throughout the study is visualized in Figure 1, in paragraph 3.3. 
In this section, each step is explained in more detail.  
 
Step 1: Screening – Week 0 
Before the inclusion starts, therapists are asked whether they want to participate. In each 
clinic of the participating organizations, the goal is to involve at least three therapists that 
want to participate in the study. This number has been discussed with representatives of the 
four participating organizations and is viewed as feasible.  
 
When a patient is enrolled in treatment, a ‘multidisciplinair overleg’ (MDO) always takes 
place in which the case is discussed and a plan for treatment is created. If the patient is 
allocated to a participating therapist, the treatment team that participates in the MDO 
discusses whether are patient is eligible for participation in this study based on the in- and 
exclusion criteria. If this is the case, researchers are notified. This entire process is 
supported by the office managers of the clinic: they remind therapists about the study before 
the MDO and notify the researchers.  
 
Step 2: Informing patient – Week 8 – 10  
Eight weeks after treatment has started, the therapist first introduces the study by means of a 
short explanation. The patient also receives a short information folder with information about 
the study. If this is the case, the patient will sign a consent form to indicate that they are 
willing to be contacted by one of the researchers, and enters their contact details and the 
preferred way of being contacted: by e-mail or phone. Therapists are reminded by the office 
manager to hand these materials and invite patients. 
 
There are multiple reasons for the ‘waiting time’ of approximately 8 weeks between 
screening and informing. First of all, therapists indicate that patients receive a lot of 
information during the first weeks of their treatment and have to fill out a lot of questionnaires 
that are part of standard procedures in forensic mental healthcare. Participating in the study 
is expected to be quite overwhelming during this initial phase of treatment. Second, 
therapists indicate that it is important to take some time to build a therapeutic relationship 
before starting with an internet-based intervention, which is especially relevant due to the 
often low treatment motivation of forensic psychiatric patients. Third, it often happens that 
when patients are enrolled in treatment, there are a lot of urgent problems that require 
attention: some more practical, e.g. homelessness, some more psychological, e.g. high 
levels of anxiety or denial of problems. This can make the first phase of treatment quite 
chaotic. In order to ensure that internet-based interventions are introduced at the right time, 
inclusion is initiated around week 8. If it is not possible or feasible to introduce the study in 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
36 of 57 
week 8 due to practical or treatment-related reasons, the therapist can also inform the patient 
up until, but no later than week 10.  
 
Step 3: Inclusion and informed consent – Week 8-11  
If the patient agrees to be contacted by the researcher, the researcher will contact them via 
their preferred way of communication, as indicated on the initial consent form. The patient is 
asked to participate by the researchers and not by the therapist to prevent that the patient 
feels obliged to participate if their therapist asks them – which is especially relevant for 
patients who follow treatment as part of a sentence. The patient will receive information 
about the goal and relevance of the study, the required effort, the rewards for participating, 
the consequences of randomization, and the nature of the internet-based intervention. The 
letter that contains all this information might be difficult to understand, so a short summary in 
simple language is added to make sure all patients are able to understand the goal and 
means of this study. Also a short video will be made to ensure that all the information is 
received and understood. The researcher will clearly communicate that during the data 
collection period, patients in the control group will not be able to use any of internet-based 
interventions offered by Minddistrict. However, once the study is finished, the patient can use 
the intervention. Consequently, if a patient is extremely motivated to use an internet-based 
intervention, they can decide not to participate in the study. The patient also will be given 
multiple opportunities to ask questions and to think five days about whether to participate. If it 
is possible for the researcher to visit the patient in real life at the treatment location, informed 
consent will be provided in person, after a waiting period of at least five days. If this is not 
possible, the informed consent form will be sent to the patient via post, after which a signed 
form an be returned by the patient via an included, stamped envelope. However, due to the 
psychosocial problems of this target group, it might not be possible or desirable for all 
patients to receive the informed consent form via post, for example because they don’t have 
a fixed living address at that point in time, or because they don’t want their housemates to 
know about participation in the study and/or in forensic care in general. If this is the case, 
and if both other options are not feasible, the informed consent will be e-mailed to the 
patient, after which they have time to read it. After five days, verbal informed consent will be 
obtained via a recorded videoconferencing session. In this way, the process of informed 
consent is adapted to the characteristics of the patient.  
 
Throughout the entire process, it will be clearly communicated that participating in this 
study does not affect the patient’s treatment progress or the evaluation of the therapist. It will 
also be clearly stated that the patient can decide to withdraw from the intervention and/or 
study at any point in treatment, without any consequences - other than not receiving the 
financial reimbursement that is part of the study. 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
37 of 57 
 
Step 4: Baseline assessment (T0) – Week 11 
Before they start with the intervention - thus before they are allocated to a condition - all 
participants will be sent a link to the self-report questionnaires by the researcher to assess 
self-efficacy, aggression, treatment readiness and to ask several short questions about 
sociodemographic variables. These questionnaires will be filled out online via the designated 
program Qualtrics. The patient is asked to fill out these questionnaires in their own time, but 
as soon as possible, preferably on the same day. If the patient has not filled out these 
questionnaire, an automated reminder will be sent after two days. If after that, the patient still 
did not fill out the questionnaires, the patient is contacted by the researcher via e-mail after 
five days, to remind them of filling out the questionnaires. If after those two reminders the 
patient has not completed the questionnaire yet, their therapist is contacted. In the upcoming 
session, the therapist will the patient for the reason for not completing the questionnaires. If 
this is due to for example difficulties with answering the questions, the researcher will contact 
the patient and fill the questionnaires out together. If the patient indicates that they do not 
want to fill out the questionnaire due to any other reason, the therapist will discuss if 
participation in the study is still an option. 
 
Besides the self-report questionnaires, the risk assessment instrument FARE will be 
administered by the therapist as part of standard treatment. As mentioned before, because 
the FARE is embedded in standardized structures of the organization, it might be that this 
risk assessment instrument is administered several weeks before the questionnaires. 
 
In this week, the therapist will also receive a short questionnaire via Qualtrics about 
background information regarding the patient (inpatient clinic in which the patient is treated, 
the main treatment goal, type of offense, and diagnoses). 
 
Step 5: Randomization and start intervention – Week 12 
Directly after the patient has provided informed consent, the patient is randomized by the 
researcher. This is communicated to the therapist, who will let the patient know whether they 
will start with the intervention ‘Aggression’ or if they will participate in the control group.  
 
After the first session of the internet-based intervention is finished, the TWEETS will be 
administered to participants in the experimental group via the Minddistrict platform, because 
the TWEETS is integrated in the intervention.  
 
Step 6: Mid-treatment assessment (T1) – Week 17 
In order to assess change during the intervention period, the three questionnaires to asses 
self-efficacy, aggression and treatment readiness are again administered to participants in 
the control and experimental group in week 17 of the study period. During treatment, the 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
38 of 57 
therapist indicates that the patient can expect the questionnaires in the upcoming week to 
ensure that the link to the Qualtrics questionnaires is not overlooked. After that, the 
researcher e-mails the link to the Qualtrics questionnaires to the patient. If the patient did not 
complete the questionnaire after two days, a reminder is sent via e-mail. If the patient does 
not fill out the questionnaire after this reminder, the same procedure as in the baseline is 
followed.  
 
Furthermore, the TWEETS is again administered, following the same procedures as at T0. 
Finally, participants will receive the first €10 voucher after completing the baseline 
assessment via e-mail. 
 
Step 7: Assessment post-intervention (T2) – Week 25 
 
After 15 weeks, the three questionnaires on self-efficacy, aggression and treatment 
readiness will be administered again, following the same procedures as described before. 
Furthermore, after finishing the final lesson of the intervention, the patient will fill out the 
TWEETS on the Minddistrict platform.  
 
Step 8: Qualitative data collection with patients - Week 25 – 30  
After the intervention is completed, a randomly selected sample of patients is invited to 
participate in a post-intervention interview. This sample is generated by means of 
(https://sealedenvelope.com/). Because the perspective of patients who dropped out of the 
intervention also is very relevant, the target group of this interview study will consist of 
patients who completed at least one session of the intervention, i.e. who completed the first 
TWEETS. Additionally, patients still have to be treated at the outpatient clinic in order to 
participate. Semi-structured interviews will be conducted in-person or via videoconferencing, 
depending on the preference of the patient and practical limitations.  
 
Step 9: Follow-up assessment (T3) – week 37 
37 weeks after start of treatment, the three questionnaires (self-efficacy, aggression and 
treatment readiness) will be again administered via Qualtrics. If, at this point, the patient is 
not being treated at the outpatient clinic anymore, he or she will be contacted by the 
researchers. This will be clearly communicated in the informed consent form and by the 
therapist. The FARE that will be administered about 6 months after treatment as part of 
standard assessment by clinics will be used for the follow-up assessment. Participants will 
receive the second €10 voucher after completing the this final assessment. 
 
 
 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
39 of 57 
Step 9: Qualitative data collection – therapists 
An open-ended questionnaire will be sent to all participating therapists at the final stages of 
the study period, i.e. in Q2 2023. In this questionnaire, they will be asked about their 
experiences with the internet-based interventions.  
8.4 Time investment of participants 
 
Participants in the experimental and control condition fill out three short questionnaires per 
measurement moment. This is estimated to take a maximum of 15 minutes. Data collection is 
kept short intentionally: forensic psychiatric patients are generally considered as hard to 
involve in research due to low motivation and short attention spans. To increase  the 
feasibility of data collection, only the most relevant questionnaires from a clinical and 
research point of view are used. Additionally, patients in the experimental condition 
individually work on the intervention outside of in-person treatment sessions. On average, 
this is estimated to take about 30 minutes per weekly lesson, but experiences in practice 
show that this can either take much longer or shorter, depending on the patient and lesson. 
Furthermore, to increase the feasibility of this study, participants will be compensated for 
their invested time by means of a voucher of €10 after the mid-treatment assessment (T1) 
and another voucher of €10 the final post-treatment assessment (T3). Experience with 
research with this target group has shown that financial rewards can serve as a motivator to 
participate and complete the study.  
Timeline 
Depending on the timing of ethical approval, inclusion of participants will start in April 2022 
up until February 2023. This means that the final follow-up assessment can take place up 
until September 2023. Participants will start with the study at any point during between Q2 
2022 and Q2 2023. To illustrate: the first participant can start with the study in April 2022, 
while the last participant can start 8 months later. In the meantime, there will be no changes 
to the internet-based intervention ‘Aggression’ to ensure consistency. This procedure 
ensures that patients do not have to wait to start treatment and also increases the feasibility 
of this study: including participants in forensic mental healthcare has been shown to be 
challenging. 
8.5 Withdrawal of individual subjects 
Subjects can leave the study at any time for any reason if they wish to do so without any 
consequences. The investigator can decide to withdraw a subject from the study for urgent 
medical reasons. Patients can also be withdrawn in case of a severe psychiatric crisis and/or 
admission to a (forensic) psychiatric hospital, other type of inpatient clinic, or prison. 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
40 of 57 
8.6 Replacement of individual subjects after withdrawal 
To account for withdrawal of subjects, 20% more participants than required according to the 
power analysis will be included. If patients drop-out, this will be accounted for due to this 
extra 20%. Patients are considered as a drop-out if they stop filling out the questionnaires 
before the end of the study. Consequently, if patients stop using the intervention but still fill 
out the questionnaire, they are not considered a drop-out. A reason for this is that 
investigating whether non-adherence to these interventions is problematic and related to 
engagement or effectiveness is part of this study. Data collection will be  ended after 
including the intended 154 patients. If throughout the study, there are signs that more than 
20% of the patients drop out, the researchers will re-evaluate the feasibility of the data 
collection with participating therapists and search for points of improvements.  
8.7 Follow-up of subjects withdrawn from treatment 
If patients withdraw from treatment entirely, they will be excluded from the study. The main 
reason for this is that the internet-based intervention has to be studied within the context of 
treatment, i.e. in a blended way. If participants in the experimental or control group do not 
receive treatment anymore, their data will not be relevant for the research questions of this 
study. As mentioned before, patients that stop using the intervention but still participate in 
data collection will not be considered as drop-outs. 
8.8   Premature termination of the study 
Based on the absence of risk associated with using internet-based interventions in treatment 
– which have been used in Dutch outpatient treatment for over ten years, without any 
incidents – we expect that there will be no reason to expect that the study has to be 
terminated due to detrimental effects of the intervention. 
 
This study is financed for two years. If the intended number of participants has not been 
reached after two years and if no additional financing has been found, the study will be 
terminated, even if the intended N has not been reached. In that case, the data from the 
already enrolled participants will still be collected. If the study has to be terminated before the 
desired number of participants is included, analyses will be conducted with the available 
quantitative and qualitative data of the participants that did complete the study.  

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
41 of 57 
9. SAFETY REPORTING 
9.1 Temporary halt for reasons of subject safety 
In accordance to section 10, subsection 4, of the WMO, the sponsor will suspend the study if 
there is sufficient ground that continuation of the study will jeopardise subject health or 
safety. The sponsor will notify the accredited METC without undue delay of a temporary halt 
including the reason for such an action. The study will be suspended pending a further 
positive decision by the accredited METC. The investigator will take care that all subjects are 
kept informed.  
9.2 AEs, SAEs and SUSARs 
9.2.1 Adverse events (AEs) 
Adverse events are defined as any undesirable experience occurring to a subject during the 
study, whether or not considered related to the study. All adverse events reported 
spontaneously by the subject or observed by the investigator or his staff will be recorded. All 
involved therapists will be required to report adverse events to the main researcher, who will 
record them.  
9.2.2 Serious adverse events (SAEs) 
A serious adverse event is any untoward medical occurrence or effect that  
- 
results in death; 
- 
is life threatening (at the time of the event); 
- 
requires hospitalisation or prolongation of existing inpatients’ hospitalisation; 
- 
results in persistent or significant disability or incapacity; 
- 
is a congenital anomaly or birth defect; or 
- 
any other important medical event that did not result in any of the outcomes listed 
above due to medical or surgical intervention but could have been based upon 
appropriate judgement by the investigator. 
An elective hospital admission will not be considered as a serious adverse event. 
If SAEs occur, the sponsor will report the SAEs through the web portal ToetsingOnline to the 
accredited METC that approved the protocol, within 7 days of first knowledge for SAEs that 
result in death or are life threatening followed by a period of maximum of 8 days to complete 
the initial preliminary report. All other SAEs will be reported within a period of maximum 15 
days after the sponsor has first knowledge of the serious adverse events. 
 
SAEs due to the internet-based intervention are not expected because it does not influence 
the physical state of the participant in any way.  

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
42 of 57 
If any SAEs occur, they will be reported by the sponsor the web portal ToetsingOnline to the 
accredited METC that approved the protocol, within 7 days of first knowledge for SAEs that 
result in death or are life threatening followed by a period of maximum of 8 days to complete 
the initial preliminary report. All other SAEs will be reported within a period of maximum 15 
days after the sponsor has first knowledge of the serious adverse events. 
9.3 Annual safety report 
Not applicable. 
9.4 Follow-up of adverse events 
All AEs will be followed until they have abated, or until a stable situation has been reached. 
Depending on the event, follow up may require additional tests or medical procedures as 
indicated, and/or referral to the general physician or a medical specialist. SAEs need to be 
reported till end of study within the Netherlands, as defined in the protocol  
9.5 Data Safety Monitoring Board (DSMB) / Safety Committee 
Not applicable.  

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
43 of 57 
10. STATISTICAL ANALYSIS 
10.1 Primary study parameter(s) 
Data will be analysed using IBM SPSS software (version 24.0) and significance was 
accepted at 0.05 or lower. Analyses will be done on intention to treat (ITT) basis.  
Research objective 1: Self-efficacy, treatment readiness, and aggression  
To identify whether there are changes in dependent variables between the experimental and 
control group on baseline (T0), during the intervention (T1), post-intervention (T2) and follow-
up assessment (T3), multilevel repeated-measures linear mixed models will be used. This 
analysis method accounts for autocorrelation within participants and missing data. Time and 
group will be used as fixed factors, along with their interactions, and participants are 
modelled as random factor. Interaction effects will be studied to identify whether the changes 
over time differed between groups. Main effects for time will be used to investigate whether 
the scores of all participants changed over time. In order to provide more insight into the 
main findings, Least-significant difference (LSD) post-hoc independent t-tests with Bonferroni 
correction will be performed to test for significant between-group differences and changes 
over time per group. 
Determining effectiveness  
In order to determine whether the internet-based intervention Aggression is effective, multiple 
scenarios are possible. 
 
First of all, it is possible that the scores of the experimental groups do not increase more 
than those of the control group on all three primary outcome measures at all measurement 
points (T1, T2 and T3). In that case, the intervention is viewed as ineffective. Second, there 
might be a significant improvement of the experimental group compared to the control group 
on one or more of the outcome measures on T1 (so halfway the intervention), but not on T2 
and T3. This indicates that the intervention shows some promise, but cannot be viewed as 
effective. Further research on why the effects are not maintained until the end of the 
intervention and on the long term is then warranted. These questions will be partly answered 
by means of the interviews at T3. Third, there might be a change on T2 - so directly after 
completing the intervention - on one or more of the outcome measures in the experimental 
group compared to the control group, which is not maintained at T3. In that case, the 
intervention is viewed as promising and potentially useful for clinical practice because 
participants did experience benefits, but not effective because the effects are not maintained 
on the long term. In that case, future research into why these effects are not maintained and 
what this means for the usefulness of these interventions in forensic outpatient care is 
necessary. Fourth, there might be an improvement of the experimental group compared to 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
44 of 57 
the control group on T3 on one or two outcome measures. In that case, the intervention is 
viewed as effective since it leads to benefits on the long term. However, future research is 
again warranted to investigate why these changes did not occur on the other outcome 
measure(s). Finally, if the experimental group outperforms the control group on all three 
outcome measures at T3, the conclusion will be that the intervention is effective and no 
future research is warranted. However, this of course does not mean that future research is 
not desirable for the sake of replicability.  
 
When interpreting the results and answering the first research questions, nuance will be 
added by investigating when and how long changes within and between groups take place. 
This will not just be done by post hoc analyses and visualizations of data, but also by means 
of qualitative research with patients and therapists, i.e. the interviews that will take place at 
T3. By means of these interviews, more insight will be gained into why changes are or are 
not observed, whether the selected outcome measures were suitable, if and how the 
intervention can cause changes on the short and long term, and whether it is of added value 
for clinical practice.  
 
10.2 Secondary study parameter(s) 
Research objective 2: Treatment contacts 
To determine if the number of face-to-face treatment contacts over a period of 37 weeks (i.e. 
9 months) is lower in patients who received the ‘Aggression’ intervention, compared to those 
who received TAU, a t-test with group as the independent variable and number of received 
treatment sessions as the dependent variable.  
Research objective 3: Dynamic risk factors 
To determine if there is more improvement on dynamic risk factors as measured by a risk 
assessment instrument in patients who received the internet-based intervention ‘Aggression’ 
as an addition to TAU, compared to those who only receive TAU. For this research 
objectives, the same analysis as for research objective 1 will be used.  
Research objective 4: Engagement 
To investigate whether engagement predicts effectiveness, simple linear regression analyses 
will be run for three variables: regulatory emotional self-efficacy, treatment readiness, and 
aggression. The score on the TWEETS on T0, T1 and T2 will be used as independent 
variables in three different regression analyses. The change scores (i.e. difference between 
scores on T2 and T0, and the difference between scores on T3 and T0) for all relevant 
variables (self-efficacy, treatment readiness and aggression) will be used as dependent 
variables for two regression analyses. These scores will be added in a second analysis to 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
45 of 57 
measure whether it adds predictive value on the dependent variables. Furthermore, in order 
to investigate if the TWEETS scores predict adherence, three analyses with the TWEETS 
score on T0 and T1 as dependent variables are run, this time with the number of finished 
lessons of the intervention as dependent variable.  
10.3 Other study parameter(s) 
Research objective 5 & 6: Patient and therapist perspective  
To explore reasons for the effectiveness or ineffectiveness of the internet-based intervention 
Aggression according to forensic psychiatric patients and therapists, qualitative data will be 
collected. Anonymized transcripts will be created of these data. These transcripts will be 
analysed via an inductive approach, using the method of constant comparison [47]. First, 
relevant fragments will be extracted from the anonymized transcripts. Based on these 
fragments, two researchers will create an initial version of a coding scheme, one per 
research question. These coding schemes will contain main and subcodes and a first version 
of a definition for each code. These  coding schemes will be applied to five transcripts by two 
researchers. After discussion amongst both coders, the coding scheme will be updated to 
ensure that it optimally fits the data. After that, both researchers will again code five 
transcripts and discuss whether any further changes are necessary. This process continues 
up until all transcripts are coded and the coding scheme covers all fragments. Intercoder 
reliability will be calculated, with a goal of at least 80% agreement. 
 
 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
46 of 57 
11. ETHICAL CONSIDERATIONS 
11.1 Regulation statement 
The study will be conducted according to the principles of the Declaration of Helsinki 2013 
(World Medical Association, 2013) and in accordance with the Medical Research Involving 
Human Subjects Act (WMO). 
11.2 Recruitment and consent 
The principal investigator is primarily responsible for recruitment of participants. Recruitment 
will take place in the four participating organizations and will start in April 2022. In order to 
ensure the feasibility of the study, researchers will actively work together with staff of 
Transfore, De Woenselse Poort, Kairos and GGZ Noord-Holland Noord. First, patients are 
informed about the study in a very general sense by their healthcare provider, after which 
they are asked if they consent to be contacted by a researcher. The researcher will contact 
the patient by the preferred means of communication – i.e. phone, e-mail or 
videoconferencing - and will fully inform them about the goal and nature of participating in the 
study. After this initial contact, the patient will have five days to consider participation, during 
which they are encouraged to contact the researcher if they have any remaining questions. 
After five days, the patient is contacted again by the researcher and is asked if they want to 
participate and if so, if they want to sign the informed consent form. The role of the 
researcher in this process is a deliberate choice to avoid any potential role conflicts. Patients 
might feel that their treatment progress can be affected by the decision to (not) participate, 
which might result in a situation in which they do not feel free to decline a request for 
participation from their therapist. To prevent this, informed consent is always asked by the 
researcher. Additionally, it is clearly communicated to the patient by both the therapist and 
researcher that participation does not affect their treatment progress in any way. 
Furthermore, it is made clear to potential participants that they can drop out of the study at 
any moment, without giving a reason for this. Communication about these conditions is done 
verbally and via the information folder that patients will receive from their therapist. To 
summarize, multiple precautions are taken to ensure that the patient does not feel forced in 
any way to participate in this study, and that the decision to participate is made on a 
voluntary basis.   
11.3 Objection by minors or incapacitated subjects (if applicable)  
Not applicable. 
 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
47 of 57 
11.4 Benefits and risks assessment, group relatedness 
11.4.1 Benefits for patients, therapists and forensic mental healthcare  
For participants in the experimental condition, the main advantage can be that they can 
follow a potentially effective intervention that can provide them with more skills to effectively 
deal with their aggression regulation problems. Explorative, qualitative research has shown 
that these internet-based interventions can have multiple potential benefits for patients, such 
as receiving more information and exercises on their disorder(s) or behaviour compared to 
TAU, an accompanying increase of knowledge and skills to deal with their problems, and an 
increased sense of ownership and accounting improvements to themselves instead of the 
therapist due to working on a part of their treatment independently [19, 21]. A tangible benefit 
for all participants – also those in the control condition -  is that they are compensated with 
two vouchers of a total of €20 to compensate for their invested time and effort.  
 
A more abstract, indirect benefit – especially relevant for participating therapists - is that 
this study can contribute to better forensic mental healthcare. The importance of this study is 
illustrated by a recent study amongst Dutch forensic mental healthcare professionals, which 
clearly showed the need for more robust evaluation studies of eHealth interventions [19].  
Especially considering the fact that internet-based interventions have been present in Dutch 
forensic mental healthcare for about ten years, is quite surprising that their effectiveness has 
not been studied yet. Consequently, it is essential to study the effectiveness of these 
modules, especially in a domain in which evidence-based practice is a core value. If the web-
based interventions prove to be effective, this can be an additional impulse for 
implementation in clinical practice. In that way, these modules provide an effective way to 
provide additional care to vulnerable patients, which might reduce the overall changes of 
recidivism and quality of care. In line with this, the effectiveness of these interventions might 
provide an additional impulse for implementation, since professionals working in forensic 
mental healthcare indicate that the absence of evidence is one of the experienced 
implementation barriers [19, 21]. If this study shows no proof for effectiveness, this should be 
a starting point for a broader discussion in forensic mental healthcare: is it worthwhile to 
invest much time, money and other resources in the implementation of these modules, or 
should these resources be focused on other, more promising technological additions to care? 
In any case, this study will contribute to evidence-based practice and thus the most optimal 
form of treatment for forensic psychiatric outpatients. 
11.4.2 Risk assessment 
 
As mentioned before, the web-based intervention Aggression has been used since 2012. 
No adverse effects have been reported by any of the organizations that have been using the 
module Aggression and other comparable interventions to developer Minddistrict. 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
48 of 57 
Furthermore, qualitative research in the Dutch forensic field via questionnaires and 
interviews has also shown no negative consequences of these modules [19]. A potential 
negative consequence that was mentioned was that working on self-reflection exercises 
might result in an excessive amount of negative feelings in patients [21]. While this can be  
therapeutic, a negative consequence might be that patients are not able to successfully 
manage these emotions. However, all exercises and texts in the Aggression intervention are 
developed by psychologists, in collaboration with experienced therapists to prevent these 
types of issues. Furthermore, the intervention is used in a blended way, which means that 
therapists regularly monitor how the patient is doing and can address unhelpful negative 
feelings and identify ways to deal with these. Finally, throughout all the years that these 
types of interventions have been used, no observable negative consequences of 
independently working on these modules have been identified.  
 
The control group of this study will not receive the Aggression intervention, or any other 
type of internet-based intervention, during the data collection period. After that, these 
interventions can be used, which will be clearly communicated. Not following these 
interventions is not expected to have a negative impact on the quality of treatment of the 
participants in the control condition. First of all, at this point in time, it is not yet clear whether 
internet-based interventions Aggression actually result in better treatment outcomes in Dutch 
forensic mental healthcare. This means that participants in the control condition are not 
denied evidence-based care, simply because these interventions are not evidence-based 
yet. Furthermore, research with log data and interviews has shown that at this point, many 
patients do not have access to internet-based interventions at all, mostly because their 
therapist does not introduce these modules. To specify: one study showed that only 54% of 
all therapists working at an organization used the modules, of which the largest part used it 
with very few patients [21]. This implies that the use of internet-based interventions is not a 
part of standard care yet, meaning that participants in the control condition are not denied 
standard care procedures.  
 
To summarize: in order to be able to deliver high-quality, evidence-based care, it is 
essential to investigate whether internet-based interventions that have been used for several 
years are indeed effective.  
11.5 Compensation for injury 
The University of Twente has a liability insurance which is in accordance with article 7 of the 
WMO. However, since there are no risk associated with the study, the University of Twente 
has asked for an exemption of the liability insurance in accordance with article 7 of the WMO. 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
49 of 57 
11.6 Incentives (if applicable) 
Participants in both groups will receive a total amount of €20 for their participation, of which 
€10 will be provided at T1, and €10 after completing the follow-up measurement (T3). An 
accompanying risk regarding the ecological validity of these results is that patients might 
complete the intervention due to the financial reward, even though they might not want to 
work on the intervention anymore. This is accounted for in multiple ways. First, it will be 
made clear that the financial reward is dependent on completing the questionnaires, and not 
the internet-based intervention. Patients thus can stop using the intervention, but can still 
receive the financial compensation if they complete all questionnaires. Second, adherence to 
these interventions in ‘real life’ is an important topic. In order to account for this, another 
study that it out of scope of this proposal will be conducted to determine the adherence to 
these types of interventions outside of a study context. By means of log data, insight will be 
gained into the way these interventions are used in clinical practice. These analyses will be 
conducted before this RCT is completed, which allows the researchers to use these findings 
on adherence from the ‘real world’ and compare them to the adherence of the participants in 
this study. Third, in line with this, researchers will account for problems with ecological 
validity in this study when interpreting these results.  

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
50 of 57 
12. ADMINISTRATIVE ASPECTS, MONITORING AND PUBLICATION 
12.1 Handling and storage of data and documents 
All data generated during screening and the study itself will be handled confidentially and 
anonymously in accordance with the EU General Data Protection Regulation and the Dutch 
Act on Implementation of the General Data Protection Regulation. Personal data will be 
coded with an individual ID-code, which is not relatable to the participant. A list will be 
created, which contains the ID-codes, stored on a separate drive on a computer and 
maintained by the principal investigator at her office in the Department of Psychology, Health 
and Technology of the University of Twente. Both the office, the drive, the computer, and the 
file containing the ID-codes will be locked with a key and password. All collected data will be 
stored in a file containing only the identification code to be able to match the data of the 
same person across different measurement points. These data will be stored and collected 
on a shared drive that is password protected and only accessible to the researchers. The key 
of the research data will be stored at the local centra that are participating.  
 
During the research period the coded research data will be stored at the University of 
Twente. After the research is completed, the data will be stored in a long time storage at 
Data Archiving and Networked Services (DANS) by the Royal Dutch Academy of Sciences 
(KNAW) after which the ID codes key file ownership and maintenance is transferred to the 
secretary of the department of Psychology, Health and Technology of the University of 
Twente. After 15 years the data will be deleted. 
 
Participants who want to be informed about their personal data or who want their data 
deleted can send a request to the principal investigator. For this, no reason is needed, since 
participants always have the right to be informed about their personal data. After the 
personal data is deleted, informing participants about their personal data is no longer 
possible. 
12.2 Monitoring and Quality Assurance  
The study is executed by L. Klein Haneveld (researcher). The study is closely monitored by 
dr. H. Kip (project leader/daily supervisor), dr. Y.H.A. Bouman (supervisor) and dr. S.M. 
Kelders (supervisor). This research group has contact on a weekly basis.  
To ensure the feasibility and fit with clinical practice, a project team with four practitioners 
from the four participating organizations was established, consisting of drs. B. Roelofsen 
(specialized nurse GGZ-NHN), drs. Inge Korfage (clinical psychologist & director Transfore), 
drs. John Cuijpers (clinical psychologist and manager de Woenselse Poort), and drs. Naomi 
Elders (psychologist Kairos). The entire project team, consisting of researchers and 
practitioners, meets on a monthly basis to discuss the progress of the study. Furthermore, 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
51 of 57 
every three months, a progress report is sent to dr. Premika Boedhoe, scientific advisor of 
funder KFZ, to monitor the study progress.  
 
12.3 Amendments  
Amendments are changes made to the research after a favourable opinion by the accredited 
METC has been given. All amendments will be notified to the METC that gave a favourable 
opinion.  
12.4 Annual progress report 
The principal investigator/sponsor will submit a summary of the progress of the trial to the 
accredited METC once a year. Information will be provided on the date of inclusion of the first 
subject, numbers of subjects included and numbers of subjects that have completed the trial, 
serious adverse events/ serious adverse reactions, other problems, and amendments.  
12.5 Temporary halt and (prematurely) end of study report 
The University of Twente will notify the accredited METC of the end of the study within a 
period of 8 weeks. The end of the study is defined as the last patient’s last measurement. 
The sponsor will notify the METC immediately of a temporary halt of the study, including the 
reason of such an action. 
 
In case the study is ended prematurely, the University of Twente will notify the accredited 
METC within 15 days, including the reasons for the premature termination. Within one year 
after the end of the study, the University of Twente will submit a final study report with the 
results of the study, including any publications/abstracts of the study, to the accredited 
METC. 
12.6 Public disclosure and publication policy 
The involved researchers aim to publish two papers about this study. First of all, in line with 
guidelines on open science, a protocol paper of the study design will be written. The second 
paper will report on the results of this study. Prior to any publications, the trial will be 
registered in the Netherlands National Trial Register (NTR) (TrialRegister.nl). All articles will 
be submitted to international, open access, scientific peer- reviewed journals. Finally, a 
Dutch report and accompanying factsheet will be written for the funder (Kwaliteit Forensische 
Zorg; KFZ) which can also be accessed freely.  

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
52 of 57 
13. STRUCTURED RISK ANALYSIS  
Not applicable.  
 
14. REFERENCES 
1. Bloem O, Nijman H, Bulten B. Psychopathologie onder gedetineerden. Groen, H; Drost, 
M; Nijman, HLI (ed), Handboek forensische geestelijke gezondheidszorg. 2011:153-62. 
2. Mullen PE. Forensic mental health. The British journal of psychiatry. 2000;176(4):307-11. 
3. Arboleda-Florez J. Forensic psychiatry: contemporary scope, challenges and 
controversies. World Psychiatry. 2006;5(2):87. 
4. Drieschner KH, Boomsma A. Validation of the treatment motivation scales for forensic 
outpatient treatment (TMS-F). Assessment. 2008;15(2):242-55. 
5. Drieschner KH, Lammers SM, van der Staak CP. Treatment motivation: An attempt for 
clarification of an ambiguous concept. Clinical Psychology Review. 2004;23(8):1115-37. 
6. Tervoort M. Behandelbaarheid. Handboek Forensische Geestelijke Gezondheidszorg 
Utrecht: De Tijdstroom. 2011:137-52. 
7. Deenik J, Tenback DE, Tak EC, Henkemans OAB, Rosenbaum S, Hendriksen IJ, et al. 
Implementation barriers and facilitators of an integrated multidisciplinary lifestyle enhancing 
treatment for inpatients with severe mental illness: the MULTI study IV. BMC health services 
research. 2019;19(1):740. 
8. Greenberg E, Dunleavy E, Kutner M. Literacy Behind Bars: Results from the 2003 
National Assessment of Adult Literacy Prison Survey. NCES 2007-473. National Center for 
Education Statistics. 2007. 
9. Goethals KR, Vorstenbosch EC, van Marle HJ. Diagnostic comorbidity in psychotic 
offenders and their criminal history: A review of the literature. 2008. 
10. 
Schmucker M, Lösel F. Sexual offender treatment for reducing recidivism 
among convicted sex offenders: a systematic review and meta‐analysis. Campbell 
Systematic Reviews. 2017;13(1):1-75. 
11. 
Papalia N, Spivak B, Daffern M, Ogloff JR. A meta‐analytic review of the 
efficacy of psychological treatments for violent offenders in correctional and forensic mental 
health settings. Clinical psychology: Science and practice. 2019;26(2):e12282. 
12. 
Lösel F, Schmucker M. The effectiveness of treatment for sexual offenders: 
A comprehensive meta-analysis. Journal of Experimental Criminology. 2005;1(1):117-46. 
13. 
Del Vecchio T, O'Leary KD. Effectiveness of anger treatments for specific 
anger problems: A meta-analytic review. Clinical psychology review. 2004;24(1):15-34. 
14. 
Saini M. A meta-analysis of the psychological treatment of anger: 
Developing guidelines for evidence-based practice. Journal of the American Academy of 
Psychiatry and the Law Online. 2009;37(4):473-88. 
15. 
Jansman-Hart EM, Seto MC, Crocker AG, Nicholls TL, Côté G. 
International trends in demand for forensic mental health services. International Journal of 
Forensic Mental Health. 2011;10(4):326-36. 
16. 
Kip H, Bouman YH, Kelders SM, van Gemert-Pijnen LJ. eHealth in 
treatment of offenders in forensic mental health: a review of the current state. Frontiers in 
psychiatry. 2018;9:42. 
17. 
Kip H, Oberschmidt K, Bierbooms JJPA. eHealth Technology in Forensic 
Mental Healthcare: Recommendations for Achieving Benefits and Overcoming Barriers. 
International Journal of Forensic Mental Health. 2020:1-17. 
18. 
Schueller SM. Mental health and eHealth technology.  eHealth research, 
theory and development: Routledge; 2018. p. 91-110. 
19. 
Kip H, Oberschmidt K, Bierbooms J, Dijkslag D, Kelders S, Roelofsen B. 
Technologie in de forensische zorg–Crossing borders. Kwaliteit Forensische Zorg. 2019;1. 
20. 
Bierbooms J, Bouman Y, Dijkslag D, Kimpen R, Muller J, Wieske R. Do’s 
en don’ts van e-health in de forensische ggz. Kwaliteit Forensische Zorg (KFZ). 2015. 
21. 
Kip H, Sieverink F, van Gemert-Pijnen LJ, Bouman YH, Kelders SM. 
Integrating People, Context, and Technology in the Implementation of a Web-Based 
Intervention in Forensic Mental Health Care: Mixed-Methods Study. Journal of Medical 
Internet Research. 2020;22(5):e16906. 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
53 of 57 
22. 
Carlbring P, Andersson G, Cuijpers P, Riper H, Hedman-Lagerlöf E. 
Internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic 
disorders: an updated systematic review and meta-analysis. Cognitive Behaviour Therapy. 
2018;47(1):1-18. 
23. 
Andersson G, Cuijpers P, Carlbring P, Riper H, Hedman E. Guided 
Internet‐based vs. face‐to‐face cognitive behavior therapy for psychiatric and somatic 
disorders: a systematic review and meta‐analysis. World Psychiatry. 2014;13(3):288-95. 
24. 
Kip H, Wentzel J, Kelders SM. Shaping Blended Care: Adapting an 
Instrument to Support Therapists in Using eMental Health. JMIR mental health. 
2020;7(11):e24245. 
25. 
Sailer M, Hense J, Mandl J, Klevers M. Psychological perspectives on 
motivation through gamification. Interaction Design and Architecture Journal. 2014(19):28-
37. 
26. 
Ludden GD, van Rompay TJ, Kelders SM, van Gemert-Pijnen JE. How to 
Increase Reach and Adherence of Web-Based Interventions: A Design Research Viewpoint. 
J Med Internet Res. 2015;17(7):e172. 
27. 
Kip H, Bouman YH. A perspective on the integration of eHealth in treatment 
of offenders: combining technology and the risk-need-responsivity model. Frontiers in 
Psychiatry. 2021:1493. 
28. 
Kip H, Bouman Y. Van denken en praten naar doen en ervaren: Op zoek 
naar de toegevoegde waarde van technologie voor de forensische psychiatrie. Sancties. 
2020:249-60. 
29. 
Sieverink F, Köhle N, Cheung K, Roefs A, Trompetter H, Keizer J, et al. 
Evaluating ehealth.  eHealth Research, Theory and Development: A Multi-Disciplinary 
Approach: Routledge; 2018. 
30. 
Creswell JW, Clark VLP. Designing and conducting mixed methods 
research: Sage publications; 2017. 
31. 
Caprara GV, Di Giunta L, Eisenberg N, Gerbino M, Pastorelli C, 
Tramontano C. Assessing regulatory emotional self-efficacy in three countries. Psychological 
assessment. 2008;20(3):227. 
32. 
Day A, Howells K, Casey S, Ward T, Chambers JC, Birgden A. Assessing 
treatment readiness in violent offenders. Journal of Interpersonal Violence. 2009;24(4):618-
35. 
33. 
Buss AH, Perry M. The aggression questionnaire. Journal of personality 
and social psychology. 1992;63(3):452. 
34. 
Hornsveld RH, Muris P, Kraaimaat FW, Meesters C. Psychometric 
properties of the aggression questionnaire in Dutch violent forensic psychiatric patients and 
secondary vocational students. Assessment. 2009;16(2):181-92. 
35. 
Kelders SM, Kip H, Greeff J. Psychometric Evaluation of the TWente 
Engagement with Ehealth Technologies Scale (TWEETS): Evaluation Study. Journal of 
Medical Internet Research. 2020;22(10):e17757. 
36. 
Van Horn J, Eisenberg M, Bouman Y, Van den Hanenberg F, Van der Put 
C, Bogaerts S. Forensisch Ambulante Risico Evaluatie (FARE). Utrecht: Kwaliteit 
Forensische Zorg; 2016. 
37. 
Veiligheid MvJ. Handboek Forensische Zorg. Ministerie van Justitie & 
Veiligheid; 2018. 
38. 
Drieschner KHH, J.M. & Weijters, G.M. . Recidive na forensische 
zorgtrajecten met uitstroom 2013-2015. Den Haag: Wetenschappelijk Onderzoek- en 
Documentatiecentrum; 2020. 
39. 
Andrews G, Basu A, Cuijpers P, Craske M, McEvoy P, English C, et al. 
Computer therapy for the anxiety and depression disorders is effective, acceptable and 
practical health care: an updated meta-analysis. Journal of anxiety disorders. 2018;55:70-8. 
40. 
Bartholomew LK, Parcel GS, Kok G, Gottlieb NH, Schaalma HC, Markham 
CC, et al. Planning health promotion programs: an intervention mapping approach: Jossey-
Bass; 2006. 
41. 
Bonta J, Andrews DA. Risk-need-responsivity model for offender 
assessment and rehabilitation. Rehabilitation. 2007;6(1):1-22. 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
54 of 57 
42. 
Kip H. The added value of eHealth in treatment of offenders: Improving the 
development, implementation and evaluation of technology in forensic mental healthcare 
[Dissertation]: University of Twente; 2021. 
43. 
Gunzenhauser C, Heikamp T, Gerbino M, Alessandri G, Von Suchodoletz 
A, Di Giunta L, et al. Self-efficacy in regulating positive and negative emotions. European 
Journal of Psychological Assessment. 2013. 
44. 
Gross JJ, John OP. Individual differences in two emotion regulation 
processes: implications for affect, relationships, and well-being. Journal of personality and 
social psychology. 2003;85(2):348. 
45. 
Eisenberg MJH, J.C.; van der Veeken, F.C.A.; Bouman, Y.H.A.; Bogaerts, 
S. & van Horn, J.E. Validatie FARE. Een multi-center studie naar de 
interbeoordelaarsbetrouwbaarheid, convergente validiteit, veranderbaarheid en de 
predictieve validiteit van de Forensische Ambulante Risico Evaluatie (FARE). Utrecht: 
Kwaliteit Forensische Zorg (KFZ); 2020. 
46. 
Kelders SM, Van Zyl LE, Ludden GD. The concept and components of 
engagement in different domains applied to ehealth: a systematic scoping review. Frontiers 
in psychology. 2020;11:926. 
47. 
Boeije H. A purposeful approach to the constant comparative method in the 
analysis of qualitative interviews. Quality and quantity. 2002;36(4):391-409. 
 
 
 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
55 of 57 
15. APPENDIX 
 
Screenshot 1: Table of contents ‘Aggression’ 
 
Screenshot 2: Example of written assignment in ‘Aggression’ 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
56 of 57 
Screenshot 3: Example of written text in ‘Aggression’  
 
Screenshot 4: Example of video fragment in ‘Aggression’ 

NL80846.091.22 / KFZ 2021-157 
 
Aggression 
Version 2 – 29/04/2022 
 
57 of 57 
 Screenshot 5: Example of audio fragment in ‘Aggression’ 
 
Screenshot 6: Example of visualization in ‘Aggression’