Face It Evaluation
Phase: NA
Conditions: Violence, Emotional Adjustment
Sponsor: The Behavioural Insights Team

Evaluation protocol
Face It: Pilot study
14/09/2023
Tom McBride
Alice Worsley,
Lilli Wagstaff,
Tess Moseley-
Roberts,
Jack Martin,

Acknowledgements
We are grateful to colleagues from Khulisa (including
Iman Haji and Derellanne Knowles) for their
assistance with and input into the early stages of this
evaluation. We are also grateful to colleagues from
the Behavioural Insights Team (particularly Dr Patrick
Taylor, and Leonie Nicks), the Youth Endowment Fund
(particularly Dr Mollie Bourne, Dr Daniel Acquah, and
Dr Amy Wells) and Dr Nick Axford (University of
Plymouth) for their helpful advice and feedback.
We are grateful to the Youth Endowment Fund
(originally funded by an endowment from the Home
Office) and Stuart Roden for the funding which
supports this work, and to the Behavioural Insights
Team and Nesta for their support and role in
administering this funding.
About
This work is being undertaken by the Ending Youth Violence Lab at the Behavioural
Insights Team. The Behavioural Insights Team (BIT) is a global social purpose company
that generates and applies behavioural insights to inform policy, improve public services
and deliver results for citizens and society.  With company number 08567792.
This report was first published in September 2023, and is available to download as a free
PDF at: https://www.bi.team/wp-content/uploads/2023/09/Face-It-Pilot-Study-
Protocol1.pdf

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
2
Contents
Contents
2
1. Study rationale and background
4
About the Ending Youth Violence Lab
4
Project overview
5
2. Intervention
6
Intervention overview
6
Intervention theory of change
7
How the intervention compares to other services and services-as-usual
8
3. Research objectives
9
4. Monitoring and success criteria
12
5. Design and methodology
14
Design
14
Participants
15
Sample size
17
Outcome measures
18
Methods and data collection
24
Data analysis
32
Risks to study validity
33
Qualitative research
34
6. Cost evaluation - data collection and reporting
41
7. Planned outputs
42
8. Ethics and data protection
42
Ethics
42
Safeguarding
44
Data protection
44
Data management
45
9. Racial diversity and inclusion
47
10. Risks
50
11. Timeline
52
Annexes
55
Annex A: Summary of Face It programme using the TIDieR framework
55
Annex B: Khulisa’s Theory of Change narrative and diagram
62
Annex C: Power calculations
70
Annex D: Face It milestones
71

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
3
Annex E: Evaluation team experience
75
Annex F: Schedule for recruitment, interventions and assessments
77
Annex G: BIT data protection policy summary
80
Project title
Face It Evaluation - Pilot study
Developer (Institution)
Khulisa
Evaluator (Institution)
Ending Youth Violence Lab
Principal investigator(s)
Tom McBride
Evaluation plan author(s)
Alice Worsley, Lilli Wagstaff, Tess Moseley-Roberts,
Jack Martin, Tom McBride
Evaluation setting
5 schools in London
Target group
13-15 year-olds living in London at risk of school
exclusion
Planned number of
participants
5 schools, approximately 160 young people
Funding source and
declaration of interests
Funding was received for this work from the Youth
Endowment Fund (YEF) and from philanthropist
Stuart Roden. There are no known conflicts of
interest associated with this publication, and there
has been no significant financial support for this
work that could have influenced its outcome.

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
4
1. Study rationale and background
About the Ending Youth Violence Lab
The Lab was founded in Summer 2022, bringing together expertise in intervention,
evaluation and youth violence. It is funded by Stuart Roden and the Youth Endowment Fund
(YEF), and is being incubated at the Behavioural Insights Team (BIT).
The Lab’s mission is to catalyse a step change in understanding and tackling violence. To do
this, we do 3 things: Firstly, we identify promising projects which seek to address youth
violence. Secondly, we fund the development and delivery of those projects. Thirdly, we
conduct research to assess how delivery has gone, ways to make it even better, and the
potential for further evaluation (with a focus on early-stage testing, to support the work of
YEF).
We prioritise three strands of activity:
1.
Supporting the importation, adaptation, and testing of well-evidenced
interventions from overseas - We will identify approaches with strong evidence of
improving youth violence outcomes or related upstream factors in other countries,
adapt these to the UK context, and deliver early-stage testing.
2.
Working with UK organisations to develop strong ideas into evaluable
interventions - We will work with the sector to find interventions that have strong
theoretical underpinnings and are committed to rigorous evaluation, and oversee the
development and early-stage testing needed to get them trial-ready.
3.
Working with developers, researchers, practitioners, and service users to
co-design new and innovative approaches - We will build partnerships and fund
the development of novel approaches to tackling youth violence, with a focus on
addressing underserved populations and unmet needs.
The project described in this protocol forms part of strand 2 of the Lab’s approach.

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
5
Project overview
Face It is a schools-based intervention, developed by Khulisa. It is an intensive therapeutic
group programme for young people, focused on developing social and emotional skills and
designed to explore the root causes of emotional distress. It combines creative techniques
like storytelling, art, debating, and role-play, using a model informed by neuroscience.
There is strong evidence that social and emotional skills are important for the positive
development of children and young people, and contribute to a range of important long-term
educational, economic, health, social and criminal justice outcomes
. There is also good
1 2
evidence that well-designed school-based social and emotional programmes can be
effective, that they are being successfully implemented in UK schools, and can have positive
impact on students’ social and emotional competencies and educational outcomes .
3
The Face It intervention is in its early stages in terms of evidence and evaluation, and there
is not yet a robust randomised control trial demonstrating that it is effective at improving
outcomes for children and young people. However, Face It indicates early promise as an
intervention to improve social and emotional skills. The intervention has demonstrated that it
can recruit and retain participants, and qualitative work indicates that the programme is
well-regarded by participants. An internal evaluation of the programme conducted by Nesta
indicated that Khulisa’s theory of change highlights relevant outcomes, and both quantitative
and qualitative insights suggested largely positive changes in the outcomes examined .
4
Khulisa is also currently conducting a quasi-experimental study which is scheduled to be
completed in the Autumn 2023. These studies collectively show that the intervention shows
promise.
However, the lack of a randomised comparison group in the existing evaluations limits the
conclusions that can be made about impact on outcomes for children and young people.
Before any future full-scale randomised control trial, it is important that a small-scale pilot
trial is conducted in advance to support and inform this work - to test and improve evaluation
procedures such as randomisation and data collection, and to generate useful information
around sample size determination.
4 Barber, A. (no date) An internal evaluation of Khulisa’s ‘Face It’ wellbeing programme in schools.
rep. Khulisa. Available at:
https://www.khulisa.co.uk/wp-content/uploads/images/NESTA-Evaluation-Report-2020.pdf.
3 Clarke, A. M., Morreale, S., Field, C. A., Hussein, Y., & Barry, M. M. (2015). What works in
enhancing social and emotional skills development during childhood and adolescence. A review of the
evidence on the effectiveness of school-based and out-of-school programmes in the UK. A report
produced by the World Health Organization Collaborating Centre for Health Promotion Research,
National University of Ireland Galway.
2 OECD. (2017). Social and emotional skills: Well-being, connectedness and success.
1 Goodman, A., Joshi, H., Nasim, B., & Tyler, C. (2015). Social and emotional skills in childhood and
their long-term effects on adult life.

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
6
2. Intervention
Intervention overview
This project is a pilot evaluation of Face It, a school-based intervention designed and
implemented by Khulisa. Face It is specifically designed for young people at risk of
offending, exploitation, and school exclusion. The programme builds self-awareness and
encourages pupils to reflect on the root causes and triggers of their disruptive or challenging
behaviour. Khulisa believes that early intervention breaks the school to prison pipeline, which
is exacerbated by exclusion, enabling young people to choose a safe and crime-free future.
The intervention is delivered over 6 weeks, including an intensive 5-day programme of
activities, and pre-programme and post-programme group and 1:1 sessions. Each
programme is tailored to participants’ needs and uses art, storytelling, 1:1 and group
experiential techniques, delivered by trained dramatherapists. Programme facilitators are
assigned to groups of young people ensuring that at least one person on each facilitation
team has a therapeutic qualification. This is typically a dramatherapy qualification, but may
also be relational therapy. A detailed description of Face It using the TIDieR framework can
be found in Annex A.
Face It aims to help young people learn new coping mechanisms and skills to identify
individual triggers (for example anger, conflict etc.) and alternative responses that can help
improve capacity for concentration and focus during lessons. It has a focus on developing
emotional regulation skills (planning, decision making, and communication) which can have
a positive impact on behaviour, sense of self-worth and confidence.
Figure 1 details a high level Theory of Change for the intervention. A logic model developed
by Khulisa can be found in Annex B. We have conducted an initial review of the Theory of
Change and discussed this in depth with Khulisa and have concluded that the outcomes
they specify and the underlying pathways to achieve them are sufficiently well considered
and plausible to justify further evaluation.

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
7
Intervention theory of change
Figure 1 - Face It theory of change

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
8
How the intervention compares to other services and services-as-usual
Face It can be distinguished from many other school-based social and emotional
programmes for similar cohorts on the basis that it is delivered by therapeutically qualified
practitioners and combines 1:1 sessions with group sessions. In particular:
1.
It takes a therapeutic and trauma-informed approach: Khulisa’s programmes are
led by therapists and follow trauma-informed methodologies including Perry’s
neurosequential model
.
5 6
2. Its activities are creative and experiential: It is theorised that children with limited
emotional literacy respond more effectively to non-verbal activities. Khulisa uses art,
drama therapy, role play and games to help participants observe themselves through
the experiences of others, which is intended to help them understand the behaviours
that benefit or hold them back.
3. The programme combines group and 1:1 support to embed learning: Face It
uses a multi-layered approach to learning. 1:1 sessions give facilitators an
opportunity to get an insight into participants’ levels of self-awareness and shed light
on how to help these participants build confidence, self-worth, and more positive and
healthy beliefs. Group sessions aim to enable participants to build relationships,
develop empathy and learn to respect other perspectives through group discussion,
and other activities.
6 Perry, B.D (2009) Examining child maltreatment through a neurodevelopmental lens: clinical
applications of the Neurosequential Model of Therapeutics, Journal of Loss and Trauma, 14, 240 –
255.
5 Perry, B.D (2006) ‘Applying principles of neurodevelopment to clinical work with maltreated and
traumatized children: the Neurosequential Model of Therapeutics’ in Webb, N.B (Eds.) Traumatized
youth in child welfare, Guildford Press, New York, 27 – 51.

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
9
3. Research objectives
This pilot trial is not primarily designed to estimate effect sizes or evaluate the impact of the
intervention, but is instead focused on assessing if Face It could and should progress to
more rigorous evaluation. If the pilot trial is successful (based on the criteria discussed in
Section 3), then there is a strong case for this to progress to a full-scale efficacy trial to
examine impact across a range of outcomes. Ideally this future evaluation would be
adequately powered to adjust for multiple comparisons, explore moderating and mediating
factors, and analyse differential impacts on pre-specified subgroups.
The pilot trial is designed to test the following questions:
●
Feasibility - Can Face It recruit and retain sufficient numbers of young people from
the target population, and be delivered with fidelity?
●
Acceptability - Do young people value the programme and feel that the content and
delivery are appropriate to their needs? Do teachers and caregivers find programme
delivery acceptable?
●
Evaluability - Do we have enough confidence in the feasibility of a randomised
controlled trial (RCT), particularly in terms of recruitment into evaluation,
randomisation and outcome data collection, to justify a future full-scale efficacy trial?
●
Outcomes - Does Face It show sufficient promise in terms of improvements in key
outcomes (resilience, emotional regulation, social and emotional wellbeing, school
attendance and school exclusions) to justify a subsequent efficacy trial?7
●
Mechanisms - Is it possible to collect data which would allow for the testing of
mediators, moderators and differential impact as part of any subsequent evaluation?
More detail on how we will address these questions is provided in Table 1.
7 Note that the sample size in this pilot is unlikely to be big enough to make strong causal claims
about effectiveness. However, it should be sufficient for us to produce descriptive statistics on key
outcome variables and make an estimate of effectiveness with low levels of confidence.

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
10
Table 1 Research questions
Overarching Question
Sub-question
Feasibility
Can Face It recruit and retain sufficient
numbers of young people from the target
population, and be delivered with fidelity?
Reach and recruitment
●
Can Khulisa recruit a sufficient number of schools to
participate in the programme?
●
Can Khulisa recruit and retain sufficient young people
at risk of youth violence?
●
What factors affect recruitment and retention?
●
Do recruitment and retention vary by ethnicity, gender
or other baseline characteristics?
●
How should the programme be adapted to improve
recruitment and retention?
Fidelity
●
Can Face It be delivered with fidelity?
●
What factors affect fidelity?
●
What variations in delivery are appropriate for effective
implementation?
Acceptability
Do young people value the programme
and feel the content and delivery are
appropriate to their needs? Do teachers
and caregivers find programme delivery
acceptable?
●
Is Face It acceptable to young people at risk of youth
violence?
●
Does participation in a randomised evaluation affect
participants’ views of the intervention? If so, how?
●
Does acceptability vary by ethnicity and race?
●
Is the programme acceptable to schools?
●
What factors affect acceptability?
●
How could the programme be adapted to increase
acceptability?
Evaluability
Do we have enough confidence in the
feasibility of a randomised controlled trial,
particularly in terms of recruitment into
evaluation, randomisation and outcome
data collection, to justify a continuation of
the efficacy trial?
●
Is randomisation feasible and adhered to?
●
Is data collection possible and sufficient?
●
What factors affect ease of data collection and
completeness and quality of data?
●
How could the approach to data collection be
improved to increase ease, completeness and quality?
●
Is it possible to retain schools and young people in the
control group?
●
What factors affect retention in the control group?
What are the best methods of achieving retention in
the control group?
●
Do practitioners or young people feel that the
evaluation impacts on intervention delivery? If so,
what elements of the intervention affect delivery
quality?
●
Is randomisation acceptable to practitioners &
teachers?
●
What factors affect acceptability?

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
11
●
How could the approach to randomisation be adapted
to increase acceptability?
Outcomes
Does Face It show sufficient promise in
terms of improvements in key outcomes to
justify a subsequent efficacy trial?
●
Is there preliminary evidence that Face It improves:
○
Resilience
○
Emotional regulation
○
Social and emotional wellbeing
○
Behavioural difficulties
○
Attendance
○
Exclusions
○
Self-reported delinquency
●
Will it be possible to link to offending data via the
Metropolitan Police?
Mechanisms
and
moderating
factors
Is it possible to collect data which would
allow for the testing of mediators and
moderators and differential impact as part
of any subsequent YEF evaluation?
●
Is it possible to gather data and permissions so that a
future RCT could assess:
●
If outcomes vary by:
○
School type or location
○
Gender
○
Age
○
SES/FSM
○
Race and ethnicity
●
Which
part
of
Face
It
leads
to
improved
outcomes?

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
12
4. Monitoring and success criteria
We will use Pause/Go criteria for two purposes:
1.
Throughout the evaluation to monitor if the pilot is proceeding as expected, allowing
for us to make real time adjustments or pause the work if needed. Criteria scoring in
red or amber zones will prompt changes to the pilot trial process, and, if criteria do
not improve, may prompt termination of the trial.
2.
At the end of the evaluation to make recommendations to YEF as to whether a full
RCT should be pursued.
Criteria used to monitor ongoing trial progress are as follows:
1) Recruitment I: Number of schools successfully recruited to the evaluation in the first 3
months of the pilot (addresses feasibility and evaluability)
a) Red: 3 or fewer
b) Amber: 4
c)
Green: 5
2) Recruitment II: Proportion of target number of eligible young people recruited within
schools in the first 3 months of the pilot (addresses feasibility)
8
a) Red: <60% of target
b) Amber: 60-79% of target
c)
Green: 80-100% of target
3) Randomisation: Proportion of recruited young people randomised to control or
treatment groups (addresses feasibility, evaluability and acceptability)
a) Red: <50%
b) Amber: 50-79%
c)
Green: 80-100%
4) Retention: Proportion of young people in the intervention arm completing the
intervention (addresses feasibility and acceptability)
9
a) Red: <50%
b) Amber: 50-79%
c)
Green: 80-100%
5) Fidelity: Assessed by the Lab through comparing facilitator self-report of activities
conducted during the 5-day programme with Face It’s list of key activities, found in
Annex D, which was developed in collaboration with the delivery team prior to the
pilot study (addresses feasibility)
9 Defined as attending both pre-programme and post-programme group and 1:1 sessions and all 5
days of the delivery
8 Assuming delivery in two cohorts over two school terms with a 3-month follow up for the first cohort
only, we anticipate that the trial will last approximately 6-7 months

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
13
a) Red: <50% of sessions meet at least 80% of criteria
b) Amber: 50-79% of sessions meet 80% of criteria
c)
Green: 80-100% of sessions meet 80% of criteria
Criteria used to contribute to determining recommendations to YEF are as follows:
1) Outcomes I: Data completeness for YP self-reported outcome variables post
intervention
for treatment and control (addresses evaluability)
10
a) Red: <35%
b) Amber: 35-89%
c)
Green: 90-100%11
2) Outcomes II: Data completeness for YP self-reported outcome variables at 3 months
post intervention for treatment and control (addresses evaluability)
a) Red: <35%
b) Amber: 35-89%
c)
Green: 90-100%
3) Outcomes III: Directional change in outcome variables for treatment and control
(addresses outcomes)
a) Red: at least one outcome measure indicates negative results and no positive
results12
b) Amber: null or mixed results
c)
Green: at least one outcome measure indicates positive results and no
negative results
Monitoring criteria will focus primarily on data completeness for young people, as our
primary outcome measure is the self-reported SDQ at post-programme. We will also monitor
response rates of parent surveys, although without specified thresholds. This will inform our
reporting to YEF, such as whether a full scale trial should incorporate parent-reported
metrics, and how best to approach this.
We will also record response rates for the 3-month follow up survey for young people. As
delivery will take place in two cohorts, one in the Autumn term and one in the Spring term,
the timeline for 3-month follow-up for the Spring term delivery cohort will fall towards the end
of the academic year. As such, we anticipate that response rates for this cohort may be
lower. Again, we will monitor this without stringent thresholds in order to inform
recommendations about the timing of delivery and data collection.
12 Null results: 95% confidence intervals that cross 0; negative results: 95% confidence intervals that
are entirely below 0; positive results: 95% confidence intervals that are entirely above 0
11 Thresholds based on the EIF evidence standards guidelines (available at:
https://guidebook.eif.org.uk/eif-evidence-standards)
10 We will have 100% completion for pre-test by definition, as students are not eligible for
randomisation if they did not complete pre-test assessment measures

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
14
5. Design and methodology
Design
This pilot trial will be a parallel two-armed randomised controlled superiority trial of Face It
compared to services as usual (SAU) for young people at risk of exploitation, offending and
school exclusion. The unit of randomisation will be individual pupils. A pragmatic sample of
state secondary schools in areas with high levels of violence in England (e.g. located in a
Violence Reduction Unit) will be selected. We will aim to recruit schools with a high
proportion of disadvantaged pupils (more pupils in receipt of the Pupil Premium than the
national average) and that are ethnically diverse (at least 30 per cent of pupils not of white
British heritage). This will not be a waitlist control study, as this pilot study is designed to
assess the feasibility of a full-scale efficacy trial including offending as an outcome. This
requires long-term follow-up, hence a waitlist would not be possible. This may affect
recruitment and retention of schools in the study, as this design will mean that many eligible
pupils will not receive the intervention, and schools will have no control over which students
receive the programme. However, we will work with Khulisa to mitigate this risk by
emphasising the importance of the trial to schools and participants, encouraging attendance
at data collection activities through food incentives, and ensuring that the process is simple
and straightforward for participants to minimise friction.
The pilot trial will use a within-schools randomisation design, with randomisation taking
place at the level of the individual student. Baseline assessment will be conducted by
researchers who are blind to treatment allocation as young people will not yet have been
allocated to control or intervention groups. We will endeavour to blind the researcher
conducting post-treatment data collection by ensuring that data collection is arranged by a
different researcher. This means that the researcher will not have been involved in the
scheduling, and will not know who is in their session and their treatment allocation. However,
it is possible that young people may ask questions of the evaluator, such as seeking
clarification on survey questions, that may indicate their allocation. Young people, teachers
and families are not blind to treatment allocation.
We will attempt to recruit an overall sample with an approximately 80:20 split of participants
displaying primarily externalising and internalising behaviours respectively, as Khulisa views
this balance as an essential component of building an optimal group dynamic for the
programme. We will then use stratified randomisation to ensure that participants within the
treatment group and control group are balanced on these characteristics and the 80:20 split
is maintained in each. Pupils will be classified into groups with internalising or externalising
behaviour based on the referral form completed by schools. We will also stratify by gender
to ensure the treatment and control group are balanced on this characteristic - and
particularly so that there is a sufficient balance within the treatment group to facilitate
mixed-gender intervention groups, which the programme developers argue improves group
dynamics.

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
15
If schools refer a number of young people that exceeds the desired sample size (as
discussed below), we will conduct a two stage randomisation process. First, we will
randomly select a subsample of the referred students, limited to the maximum sample size
per school. Second, we will randomly assign members of this subsample to either treatment
or control. Those not randomly selected will not participate in the programme or the study.
This will ensure that we do not over sample in either the treatment or control group and that
allocation to the treatment group is entirely random.
Randomisation and assignment to intervention or control groups will be implemented by the
Ending Youth Violence Lab using a random number generator in Stata. The allocation
mechanism will be young people’s Unique Pupil Number (UPN).
Randomisation will occur in batches for each school after students are referred to the
programme and assessed for eligibility, and after consent has been obtained and baseline
assessment has taken place, but before any contact with the programme such as 1:1
discussions. This means that after Khulisa has received referrals and assessed eligibility, the
Ending Youth Violence Lab will conduct baseline data collection and randomise eligible
students to either treatment or control on a 1:1 ratio in each school in turn. The Ending Youth
Violence Lab will then inform Khulisa about the allocation of young people to treatment or
control.
Individuals allocated to the control group will receive services as usual (SAU). As part of
data collection, we will investigate what this entails through surveys of control group
participants.
As part of the development of this pilot study, we will conduct a participatory panel with a
separate group of young people who have experienced the programme in the past few
years. We will seek input from the panel on the design of our evaluation materials and in
interpreting our findings.
Participants
Participants will be young people aged 13-15 in years 9 and 10, who are referred to Face It
by schools and who meet programme eligibility criteria. Referrals will be made within-school,
typically by teachers with awareness of the pupil’s behaviour, such as a Head of Year or
Form Tutor, and based on recruitment criteria shared with schools by Khulisa. Pupils will be
identified as at risk due to internal school data and assessed for eligibility using information
from Khulisa’s referral form. Participants will then be enrolled by Khulisa.
The programme will be delivered within 5 schools, twice within each school, to two different
groups of young people (to one group of Year 9s, and to one group of Year 10s).
Participants will predominantly be pupils displaying externalising behaviours. This indicates
pupils at risk of youth violence, proxied by risk of exclusion, history of verbal or physical
violence and/or bullying perpetration. Participants will also include pupils displaying

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
16
internalising behaviours. This includes young people with low self esteem and at risk of
bullying victimisation. Programme groups will include young people displaying externalising
and internalising behaviours with an approximate ratio of 80:20 as Khulisa views this
balance as an essential component of building a group dynamic.
All participants will be asked to consent to be part of the evaluation for ethical purposes
before data collection and randomisation. We will seek consent for participation in the
evaluation first from the Head Teacher or other member of the senior leadership team who is
capable of acting in loco parentis. This individual will provide written consent for pupils within
the school to participate in the evaluation and receive the programme.
Parents/caregivers of young people will be given an information sheet explaining the
evaluation purpose and process and will be given the opportunity to withdraw their child from
the evaluation. Young people will be given a verbal explanation of the purpose and process
of the evaluation as well as a written information sheet. Young people will be asked for active
written consent prior to survey data collection, and will be given the opportunity to opt out of
data processing of routinely collected school data. Participation in the intervention is
contingent on participation in the evaluation, hence opting out of data collection prior to
intervention delivery would involve withdrawing from the programme. Young people will also
have the opportunity to opt out of data processing and/or the evaluation at a later stage by
speaking to a teacher.
Young people will also be asked for written consent in order to participate in interviews.
The same pathway for consent as detailed above will be applied for participation in the
participatory panel: the Head Teacher, or other appropriate staff member, will consent for the
school to participate. Parents/caregivers of young people will then be given the opportunity
to withdraw their child, and young people will give written consent to participate.
Participating schools will be identified by Khulisa using convenience sampling. The
limitations of this approach will be highlighted in the final report. All schools identified will be
in London. Individual recruitment will take place following Face It’s usual processes with the
additions of explanations of the trial and randomisation.
Figure 2 below outlines the proposed participant flow for the project.

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
17
Figure 2 - Participant flow diagram for pilot study
Sample size
Khulisa has significant experience delivering Face It and has determined that the ideal group
size is between 8-10 students. Through our discussion with them, it became clear that

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
18
recruiting more than 5 schools would not be feasible in the timelines of the pilot. However,
they are able to deliver the programme to two cohorts of students in each school. We
therefore will aim to recruit 160 eligible participants across five schools (80 young people in
the intervention group and 80 in the control group). Khulisa will ask schools to refer anyone
who is eligible for the intervention. This will give us a stronger understanding of the true
demand for interventions like Face It in schools.
We will use a pragmatic approach to achieve an overall sample size of 160 eligible
participants: if some schools are unable to refer a sufficient number of pupils, we will aim to
oversample from other schools, noting that Khulisa can deliver the programme in groups of
up to 12.
Using pupil level randomisation, a sample size of 160, power of 0.8, and estimates for
standard deviation on the SDQ from previous papers , we anticipate that we would be able
13
to detect a Cohen’s D of 0.5 (or a mean difference of 3 on the SDQ). Whilst this effect size is
larger than those detected in many YEF efficacy trials (or used in the power calculations),
our primary objective in this study is not to test the impact of Face It, but rather to establish
its evaluability and generate information that will enable a full-scale impact evaluation in the
future.
It is worth noting that during this pilot trial we are making no assumptions about pre-test
post-test correlation, which means our power calculations are conservative estimates. As
part of this study we aim to collect sufficient data to make an estimate of this value (and the
standard deviation of the outcome) to enable more accurate power calculations for any
full-scale efficacy trial.
Outcome measures
We intend to have one primary outcome (the self-assessment Strengths and Difficulties
Questionnaire at post-programme), and a number of secondary outcomes. Given the high
number of outcomes and comparisons we will be making, we will be adjusting for multiple
comparisons. While this is not strictly necessary in a pilot study where the aim of the trial is
not to estimate impact, we feel it is important to replicate the analytical approach which
would be used in a full scale efficacy trial. More details on how we will adjust for multiple
comparisons can be found in the Statistical Analysis Plan. All outcome measures (see Table
2) will be collected at baseline (prior to the intervention), at post-test (after delivery of the
intervention), and at 3-month follow-up (with the exception of attendance and exclusions,
which will only be collected at baseline and follow-up).
13 Two papers found that the standard deviation for the SDQ was between 5-7, so we ran our power
calculations assuming 6.5. The papers are cited as follows: Cortina, Melissa & Fazel, Mina. (2014).
The Art Room: An Evaluation Of A Targeted School-based Group Intervention For Students With
Emotional And Behavioural Difficulties.. The Arts in Psychotherapy. 11. 10.1016/j.aip.2014.12.003;
Vugteveen, J., de Bildt, A. & Timmerman, M.E. Normative data for the self-reported and
parent-reported Strengths and Difficulties Questionnaire (SDQ) for ages 12–17. Child Adolesc
Psychiatry Ment Health 16, 5 (2022). https://doi.org/10.1186/s13034-021-00437-8

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
19
Table 2 Outcome measures
Type of
outcome
Outcome
measured
Instrument
Completed
by
Number
of items
Age
suitability
(young
person)
Subscales to be used
Scoring14
References
Primary
Behavioural
difficulties
Strengths and
Difficulties
Questionnaire
(SDQ) -
post-programme
self-assessment
Young
person
25
11-17
All subscales including:
●
Emotional symptoms
●
Conduct problems
●
Hyperactivity/inattention
●
Peer relationships
problems
●
Prosocial behaviour
Total difficulties
score:
A score from 0-40 is
generated by
summing scores from
all the subscales,
except the prosocial
subscale.
While the total
difficulties score is the
primary outcome, we
will also examine the
total difficulties score
when broken down into
the externalising score
(the sum of the conduct
and hyperactivity
scales), and the
internalising score (the
sum of the emotional
and peer problems
scales).
Goodman,199715
15 Goodman R (1997) The Strengths and Difficulties Questionnaire: A Research Note. Journal of Child Psychology and Psychiatry, 38, 581-586.
14 Data will be collected and outcome measures scored by the Ending Youth Violence Lab

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
20
Secondary
Behavioural
difficulties
SDQ - 3-month
follow-up
self-assessment
Young
person
25
11-17
All subscales including:
●
Emotional symptoms
●
Conduct problems
●
Hyperactivity/inattention
●
Peer relationships
problems
●
Prosocial behaviour
Total difficulties
score:
A score from 0-40 is
generated by
summing scores from
all the subscales,
except the prosocial
subscale.
Summed score of
subscales give a score
ranging from 0-40s
Goodman,199716
Behavioural
difficulties
SDQ - parent
assessment
Parent/care
giver
25
4-17
All subscales including:
●
Emotional symptoms
●
Conduct problems
●
Hyperactivity/inattention
●
Peer relationships
problems
●
Prosocial behaviour
Total difficulties
score:
A score from 0-40 is
generated by
summing scores from
all the subscales,
except the prosocial
subscale.
Goodman,199717
Offending
The Self-Report
Delinquency
Scale
Young
person
19
10-17
Does not have subscales
Variety of
delinquency score:
Sum the number of
items the respondent
answers ‘yes’
to:
• Yes = 1
• No = 0
Produces a score that
Thornberry and
Krohn, 200018
18 Thornberry, T.P., & Krohn, M.D. (2000). The self-report method for measuring delinquency and crime. Measurement and Analysis of Crime and Justice, 4, 33-83.
17 Goodman R (1997) The Strengths and Difficulties Questionnaire: A Research Note. Journal of Child Psychology and Psychiatry, 38, 581-586.
16 Goodman R (1997) The Strengths and Difficulties Questionnaire: A Research Note. Journal of Child Psychology and Psychiatry, 38, 581-586.

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
21
ranges from 0-19.
Volume of
delinquency score:
Summing the point
values when
respondents report a
number of times. Point
values are assigned as
follows:
• Once = 1
• Twice = 2
• 3 times = 3
• 4 times = 4
• 5 times = 5
• Between 6 and 10
times = 6
• More than 10 times =
11
Victimisatio
n
Problem
Behaviour
Frequency Scale
Young
person
10
Adolescents
●
Overt victimisation
●
Relational
victimisation
A score ranging from
6-36 achieved by
summing scores of the
two subscales
Farrell et al.,
201619
Resilience
Children’s Hope
Scale
Young
person
6
8-16
All subscales
●
Agency
●
Pathways
A score ranging from
6-36 achieved by
summing scores of the
Snyder et al.,
199720
20 Snyder, C. R., Hoza, B., Pelham, W. E., Rapoff, M., Ware, L., Danovsky, M., ... & Stahl, K. J. , "The development and validation of the Children’s Hope Scale," Journal of Pediatric Psychology, Vol.
22, No. 3, 1997, pp. 399–421.
19Farrell, A. D., Sullivan, T. N., Goncy, E. A., & Le, A. T. H. (2016). Assessment of adolescents’ victimization, aggression, and problem behaviors: Evaluation of the Problem Behavior Frequency
Scale. Psychological assessment, 28(6), 702.

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
22
two subscales
Emotional
regulation
The Emotional
Regulation
Questionnaire
Young
person
10
10-18
●
Cognitive reappraisal
●
Expressive suppression
Summed score of
subscales. The higher
the score, the greater
the use of emotion
regulation strategies,
conversely lower
scores represent less
frequent use of such
strategies
Ioannidis &
Siegling, 201521
Gross & John,
200322
Gullone & Taffe,
201223
Social and
emotional
wellbeing
The Short
Warwick-Edinbur
gh Mental
Well-being Scale
(SWEMWBS)
Young
person
7
11+
Does not have subscales
The SWEMWBS is
scored by first summing
the scores for each of
the seven items, which
are scored from 1 to 5.
The total raw scores
are then transformed
into metric scores using
the SWEMWBS
conversion table
resulting in a score
ranging from 7-35.
Melendez-Torres
et al., 201924
24Melendez-Torres, G., Hewitt, G., Hallingberg, B. et al. Measurement invariance properties and external construct validity of the short Warwick-Edinburgh mental wellbeing scale in a large national
sample of secondary school students in Wales. Health Qual Life Outcomes 17, 139 (2019). https://doi.org/10.1186/s12955-019-1204-z
23 Gullone, Eleonora; Taffe, John (2012). The Emotion Regulation Questionnaire for Children and Adolescents (ERQ–CA): A psychometric evaluation.. Psychological Assessment, 24(2), 409–417.
doi:10.1037/a0025777
22 Gross, J.J., & John, O.P. (2003). Individual differences in two emotion regulation processes: Implications for affect, relationships, and well-being. Journal of Personality and Social Psychology, 85,
348-362.
21 Ioannidis, C. A., & Siegling, A. B. (2015). Criterion and incremental validity of the emotion regulation questionnaire. Frontiers in Psychology, 6, Article 247. https://doi.org/10.3389/fpsyg.2015.00247

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
23
School
attendance
ImpactEd data
Admin data
N/A
N/A
N/A
Number of absences
during the pilot period
N/A
School
exclusions
ImpactEd data
Admin data
N/A
N/A
N/A
Number of exclusions
during the pilot period
N/A
Additionally, while a subsequent efficacy trial will evaluate violence as an outcome, it will not be measured in this pilot trial due to the short timeframes: data
from Police National Computer (PNC) records, which represent the most reliable data, will not be available for several years. We will, however, test that we
can collect data and obtain consent (for ethical reasons) for analysis of PNC data to provide assurance to YEF that analysis of the impact on offending would
be feasible in any subsequent efficacy trial. We will also collect self-reported offending data from young people using the Self-Report Delinquency Scale.

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
24
Methods and data collection
Quantitative research activities
We will be conducting quantitative research activities within all five participating schools. We
will use:
a) Young person survey at baseline, end-of-programme and 3-months post-programme
b) Parent/caregiver survey at baseline, end-of-programme and 3-months
post-programme
c)
Programme administrative data
d) ImpactEd data
e) Facilitator report
a) Young person survey
We will conduct surveys with young people at baseline (pre-randomisation),
end-of-programme and at 3-months post-programme. Surveys will be conducted in person in
groups during the school day. Since all outcome measures we have chosen can be
self-administered, young people will complete the survey themselves using technology
provided by schools such as computers or tablets. However, to ensure the data that is
collected is of high quality, a trained member of the evaluation team will be present at all
data collection sessions to respond to any clarification questions. To ensure the conditions
are as similar as possible across all data collection sessions, Lab staff will be trained on how
to facilitate these sessions, highlighting what questions from participants they can answer
and what questions they can’t. While online completion reduces the likelihood of
encountering certain issues relative to paper completion (e.g. errors with digitisation), there
will still be opportunities for errors, e.g. participants accidentally skipping questions. We have
decided not to make questions mandatory - this is because giving participants the chance to
not answer questions that they do not want to was highlighted as a key part of making the
survey ethical and fair for participants. To mitigate the potential for accidental skipping of
questions, we have included a reminder at the bottom and top of each survey page for
participants to check each page is complete before submitting. In addition, Lab staff will
remind participants before they begin the survey and before they submit it. Another
advantage to online surveys is that we can track the time-to-completion. For details on how
we will use this to understand data quality, please see the Statistical Analysis Plan.
We have decided to collect survey data from control and intervention participants separately,
as part of separate data collection sessions, rather than collecting survey data from all
participants as part of the same groups in the same data collection sessions. There are pros
and cons to each of these approaches. On balance, we have opted for the former as our
partners at Khulisa felt that the latter would not be feasible, and that it would cause potential
issues for the control group to interact with pupils who have received the programme in this
context. From an evaluation perspective, there are associated risks around resentful
demoralisation and compensatory rivalry.

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
25
As the surveys are self-completed by participants, there is arguably limited potential for bias
introduced by researchers not being blind to treatment allocation. However, we will
endeavour to blind the independent researcher supervising survey completion to treatment
allocation by ensuring that data collection is arranged by a different researcher. This means
that the researcher will not have been involved in the scheduling, and will not know who is in
their session and their treatment allocation. However, it is possible that young people may
ask questions of the evaluator, such as seeking clarification on survey questions, that may
indicate their allocation.
Each survey collection session will start with a short icebreaker activity. The aim of this
activity is to ensure that young people are comfortable in the setting prior to any data
collection activities in order to minimise any potential distress from survey questions
regarding delinquency, emotional regulation and wellbeing which may be distressing for
young people.
b) Parent/caregiver survey
We will conduct surveys with parents/caregivers of young people at baseline
(pre-randomisation), end-of-programme, and at 3-months post-programme. Surveys will be
completed online: parents/caregivers will receive a link to complete the survey via email.
Parents/caregivers will be offered a shopping voucher as an incentive to participate.
Since this survey will be completed independently without a trained researcher, the steps we
can take to ensure data quality are limited. However, we will inform participants at the
beginning of the survey about what the ideal conditions will be. Additionally, we will include
reminders at the top and bottom of each page reminding them to check the page is complete
before moving on. Finally, we will use the time-to-completion as a check for data quality.
c) Programme administrative data
Khulisa will send referral data from referral forms and session attendance lists to the Ending
Youth Violence Lab. Programme data will also be recorded and shared, including:
●
Number of young people referred;
●
Number of eligible participants consenting to participate in the intervention and
evaluation;
●
Number of young people in the intervention and control arms; and
●
Complaints submitted to Khulisa and school staff.
d) ImpactEd data
We will use the ImpactEd platform to record school attendance and exclusions. This will be
collected at baseline and 3-months post-programme, and linked to participant data using
young people’s UPNs.
e) Facilitator report

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
26
Programme facilitators will report activities conducted during the 5-day programme. This will
be compared to the Face It milestones (Annex D) to assess programme fidelity (dose and
adherence).
Table 3 provides an overview of our proposed methods of data collection. Further detail can
be found below. Information on the timings of data collection can be found in Annex F.

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
27
Table 3 Data collection - brief overview
Purpose
Topic
Method25
Data collected
Data analysis
Feasibility
Recruitment
Programme administrative data
Number of eligible young people agreeing to
intervention and participating in the
evaluation
Descriptive statistics
In depth interviews with key school staff who
manage relationship with Khulisa: one interview per
case study school26
Perception of ease of recruiting eligible young
people
Framework analysis
Written reflection from Khulisa on the process of
school recruitment including the number of schools
approached and how schools were successfully
approached
Number of schools approached, successfully
engaged and recruited
Descriptive statistics
and/or framework
analysis
Retention
Programme administrative data
●
Number of young people in the
intervention arm completing the
intervention and completing data
collection
●
Number of young people in the control
completing the data collection
Descriptive statistics
Fidelity
Facilitator report
Programme facilitator data on activities
delivered compared to Face It milestones
-
27
examines dose and adherence
Descriptive statistics
27 Annex D
26 See Qualitative Research section for more details
25 Orange: quantitative data; Blue: qualitative data; Green: quantitative and/or qualitative

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
28
Interviews with young people
Discussion of young person engagement with
programme - examines engagement
Framework analysis
In-depth interview with one programme facilitator
per case study school (between 1 - 3 depending on
whether facilitator runs session in more than 1
school)
Details on activities delivered and
comparisons to ethos of delivery and
experiences running the programme -
examines adherence, engagement and
quality of delivery
Framework analysis
Acceptability
Acceptability of
randomisation
Programme administrative data
Compliance with randomisation measured via
attendance sheets
Descriptive statistics
Programme administrative data
Complaints submitted to Khulisa and school
staff
Descriptive statistics
Programme administrative data
Content analysis of complaints filed
Framework analysis
Acceptability of
content
Young person post-programme survey
Responses to questions measuring
experiences in the intervention
Descriptive statistics
Interviews with 4-6 young people per school across
three schools
Young people’s perception of the quality of
the content and delivery of Face It
Framework analysis
Acceptability of
delivery
In depth interviews with key school staff who
manage relationship with Khulisa: one interview per
case study school
School staff acceptance of programme
Framework analysis
Evaluability
Completeness of
data collection
Young person baseline, post-programme and
3-month post-programme follow-up survey
Admin data on completion of outcome
measures from treatment and control
Descriptive statistics
Service as usual
Young person post-programme survey
Control group responses about services
received
Descriptive statistics
Feasibility of
Short parent/caregiver interviews held at a drop in
Engagement rates for interviews with
Descriptive statistics

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
29
interviewing
caregivers
evening at one case study school
parents/caregivers within one case study
school
Feasibility of
linking to
Metropolitan
Police dataset
Discussion with Metropolitan police and/or other
relevant departments and agencies
Discussion about the feasibility of linking data
to the Police National Computer (PNC)
Reporting of
outcomes of
discussion
Outcomes
Resilience
Young person baseline, end-of-programme and
3-month post-programme follow-up survey
The Children’s Hope Scale
Pre-specified
statistical analysis28
Interviews with young people
Perceptions of behaviour change related to
resilience
Framework analysis
Short parent/caregiver interviews held at a drop in
evening at one case study school
Perceptions of behaviour change related to
resilience
Framework analysis
Emotional
regulation
Young person baseline, end-of-programme and
3-month post-programme follow-up survey
The Emotional Regulation Questionnaire
Pre-specified
statistical analysis
Interviews with young people
Perceptions of behaviour change related to
emotional regulation
Framework analysis
Short parent/caregiver interviews held at a drop in
evening at one case study school
Perceptions of behaviour change related to
emotional regulation
Framework analysis
Social and
emotional
wellbeing
Young person baseline, end-of-programme and
3-month post-programme follow-up survey
Short Warwick-Edinburgh Mental Wellbeing
Scale (SWEMWBS)
Pre-specified
statistical analysis
Interviews with young people
Perceptions of behaviour change related to
social and emotional wellbeing
Framework analysis
28 See Data Analysis section for more details

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
30
Short parent/caregiver interviews held at a drop in
evening at one case study school
Perceptions of behaviour change related to
social and emotional wellbeing
Framework analysis
Behavioural
difficulties
Young person baseline, end-of-programme and
3-month post-programme follow-up survey
SDQ (self assessment)
Pre-specified
statistical analysis
Caregiver baseline, end-of-programme and 3-month
post-programme follow-up survey
SDQ (parent assessment)
Pre-specified
statistical analysis
Interviews with young people
Perceptions of behaviour change related to
behavioural difficulties
Framework analysis
Short parent/caregiver interviews held at a drop in
evening at one case study school
Perceptions of behaviour change related to
behavioural difficulties
Framework analysis
Offending
Young person baseline, end-of-programme and
3-month post-programme follow-up survey
Self-report delinquency scale
Pre-specified
statistical analysis
Victimisation
Young person baseline, end-of-programme and
3-month post-programme follow-up survey
Problem Behaviour Frequency Scale (overt
and relational victimisation subscales only)
Pre-specified
statistical analysis
Attendance
ImpactEd data collected at baseline and 3-months
post-programme
Attendance data
Pre-specified
statistical analysis
Exclusions
ImpactEd data collected at baseline and 3-months
post-programme
Exclusion data
Pre-specified
statistical analysis
Mechanisms
Individual
demographics
Programme administrative data (via referral forms)
Data on completeness of data provided
Descriptive statistics
Intervention
Facilitator report
Analysis of the completeness of responses to
the Face It Milestones
Descriptive statistics
Feasibility of
Interviews with young people
Analysis of sign-up rates and attendance at
Descriptive statistics

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
31
conducting
interviews
interviews
Interviews with key school staff who manage
relationship with Khulisa
Analysis of sign-up rates and attendance at
interviews
Descriptive statistics
Interview with one programme facilitator per case
study school (between 1 - 3 depending on whether
facilitator runs session in more than 1 school)
Analysis of sign-up rates and attendance at
interviews
Descriptive statistics

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
32
Data analysis
The following section provides a high-level overview of our data analysis plan - we will
provide a more detailed overview in a full statistical analysis protocol which will be produced
at a later date.
This pilot trial is not primarily designed to estimate effect sizes or evaluate the impact of the
intervention in depth, but is instead focussed on assessing if Face It could and should
progress to further stages of more rigorous evaluation.
Feasibility, acceptability and evaluability analysis
The primary objective of the pilot trial is therefore to establish the feasibility, acceptability,
and evaluability of evaluating Face It. This will be done based on framework and descriptive
analysis of quantitative and qualitative data, as described in Table 2, which will be assessed
based on the monitoring criteria discussed in Section 4 of this protocol.
In addition to the analyses needed to assess our monitoring criteria, we will conduct analysis
of programme administrative data to test whether uptake and retention varies by key
participant characteristics, and to test correct treatment assignment. Analysis will be
pre-specified in the full statistical analysis plan.
We will conduct descriptive analysis of programme administrative and implementation data
to inform assessments of trial arrangements and practice model and suitability to proceed
with a full evaluation.
Effectiveness analysis
In addition to the analysis of the feasibility, acceptability and evaluability of Face It, we will
conduct analysis on the outcomes of the pilot study using the following approach, in order to
further inform monitoring criteria and our resulting recommendation to YEF.
Our analysis will help us understand if there is sufficient evidence of impact to justify a larger
and more robust efficacy trial. Because this is a pilot study with a small sample size, we will
have to interpret any statistical results with caution.
We intend to have one primary outcome (the self-reported, post-programme SDQ), and a
number of secondary outcomes. These are measured using the following scales:
Primary outcome:
●
SDQ (total difficulties, post-programme self assessment)
Secondary outcomes (all at both post-programme and 3-month follow up:
●
SDQ (3-month follow-up self assessment)
●
The Self-Report Delinquency Scale

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
33
●
Problem Behaviour Frequency Scale
●
SDQ (parent assessment)
●
The Children’s Hope Scale
●
The Emotional Regulation Questionnaire
●
SWEMWBS
●
School Attendance
●
School Exclusions
We have conducted preliminary assessments of validity and reliability of these scales and
judged these to be appropriate for this pilot trial.
All outcome data will be analysed using an intention to treat (ITT) analysis and linear (or
logistic where relevant) regressions - high-level equation model below. We will collect
pre-intervention outcomes for all students to increase power and adjust for regression to the
mean. Our control vector will include gender, ethnicity, FSM status, pre-intervention
outcomes, and allocation reason .
29
𝑌𝑖 = α + β1 * 𝑇𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡 + β2 * 𝐶𝑜𝑛𝑡𝑟𝑜𝑙 𝑣𝑒𝑐𝑡𝑜𝑟 + ε𝑖 
We will conduct subgroup analysis for the 80% of young people showing externalising
behaviour to test whether these outcomes vary from the 20% displaying internalising
behaviour.
Additionally, we will produce descriptive statistics for individual survey questions that are
most relevant to our outcomes (the full statistical analysis plan will clearly define these).
Risks to study validity
Spillover
The risk of spillover arises from the potential for students in the intervention group to discuss
the programme with students in the control group, thereby providing the control group with
some of the learnings from the intervention. However, we have assessed that the risk of
spillover is low, given the intense and targeted nature of the intervention. That is, any effects
are likely to emerge as a result of participation in the full programme; we do not expect that
students discussing the content outside of programme sessions would provide the same
benefits. We do not anticipate that an awareness of parts of the programme by those in the
control group, or interactions between those in the treatment and the control is likely to
impact on the outcomes being assessed.
School staff are not involved in the delivery of this programme, and the likelihood of spillover
through staff will be further reduced by ensuring that teachers do not sit in on or observe
29 Defined as whether they fall into the 80% of externalising behaviour or 20% of internalising
behaviour

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
34
sessions, and by working with new schools who have not experienced the Face It
programme before. For example, pupils in the control group of a school which has worked
with Khulisa before may experience elements of the intervention if teachers have previously
been exposed to programme elements and include them in their lessons and classroom
approach.
Risk that assessment for eligibility will result in control group not receiving
service-as-usual (SAU)
We assess that the risk of this is medium. Many teachers and other intervention providers
will not be aware that the student has been referred, but there is a risk that some may be
aware and adapt their behaviour as a result. We will assess the impact of this by recording
what interventions the control and intervention groups received and whether these are
different, as well as looking for evidence as to whether this is influenced by the eligibility
assessment.
Attrition
Some degree of young person attrition from the evaluation is to be expected. We will attempt
to minimise the impact of this by:
●
Providing incentives such as food to all participants when filling out surveys;
●
Avoiding unnecessary follow-up by conducting two periods of follow-up (immediately
after programme completion and three months after programme completion) rather
than three;
●
Ensuring that evaluation activities are designed to be low-impact in terms of burden
and time;
●
Informing schools with sufficient notice about any planned activities with appropriate
information about how useful the evidence created by the study will be; and
●
Where needed, utilising the relationships that Khulisa has built with LAs to facilitate
access and cooperation.
We will monitor rates of attrition and whether there are differences in rates of attrition based
on gender, ethnicity, SES (proxied by FSM status) and baseline outcomes.
Qualitative research
Participatory panel
We propose adopting a participatory approach to the design of our evaluation materials and
in understanding our results. To do this, we will conduct a young person participatory panel
in one school to inform our evaluation. This will be a school where Khulisa has previously
delivered Face It, but where no delivery as part of this evaluation is taking place.
We will involve a small group of up to five young people who have participated in the Face It
programme within the last two years to help co-construct elements of our research. Their

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
35
experiences and understanding of the programme will inform our decision-making processes
and ensure that we appropriately capture participant experiences of the programme. This
process will consist of two in-person sessions as follows:
●
Session 1: 2-hour session prior to any qualitative data collection to introduce the
evaluation and the key aims and sense check evaluation materials including, but not
limited to, interview information sheets and consent forms, and to ask for advice and
input into improvements.
●
Session 2: 3-hour session after completion of data collection to help the evaluation
team understand and analyse the qualitative data
The young people will be compensated for their time and expertise with shopping vouchers.
As a result of this approach, some of the evaluation tools may change following the first
participation panel. We do not anticipate any fundamental changes to the evaluation plan.
Qualitative research activities
We will be conducting qualitative research activities within three case study schools. These
three schools will be selected from the five participating schools using a convenience
sampling approach, depending on schools’ availability and willingness to participate. This
method will be used to explore and examine experiences and perceptions of the
interventions in depth, in three different contexts and using multiple sources of data. We will
explore site-specific characteristics and circumstances that influence the implementation of
Face It and how this might vary between schools. We will use:
a) In-person interviews with young people in the intervention group
b) In-person interviews with parents/caregivers of young people in the
intervention group
c)
Online interviews with key school staff who manage the relationship with
Khulisa
d) Online interviews with programme facilitators
a) In-person interviews with young people in the intervention group
We will conduct in-person interviews with young people from the intervention group within six
weeks after the programme finishes to understand young people’s own perceptions and
personal experiences of the programme. We will identify 12-18 students (4-6 students per
case study school) to participate using a combination of convenience and purposive
sampling. As part of the convenience approach we will draw interviewees based on
recommendations from the teacher responsible for the relationship with Khulisa and
students’ availability and willingness to be interviewed. In addition to this, using purposive
sampling, we will aim to sample participants across a range of key criteria, namely, gender,
school, year group, ethnicity, FSM status, and whether young people display primarily
internalising or externalising behaviours. The interviews will either be paired or individual

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
36
depending on the feedback that the participatory panel gives and dependent on the young
people’s own preference.
Interviews will be between 45-minutes and 1-hour in length, will take place at school during
the school day, and will be audio-recorded (with consent) and transcribed. Young people will
be offered a £10 shopping voucher for compensation for their time and knowledge. Interview
guides will be developed in collaboration with Khulisa, and we will seek input from the young
person participatory panel (see above) regarding key areas to cover and elements to
consider or avoid during interviews. The topic guides for young people will be participant-led
to allow the researcher to form a stronger rapport with the participant and generate more
in-depth data. This means that topic guides will include a series of prompts relating to the
research objectives, rather than representing a predefined list of questions, allowing the
researcher to shape the interview based on the participant’s responses.
Interview topic guides are likely to focus on the following research questions:
1.
Acceptability of the programme (Do young people value the programme and feel the
content and delivery are appropriate to their needs?)
a.
What participants’ experiences of participating in the programme were
b.
What participants thought about programme content
c.
What participants thought about programme delivery
2.
Programme outcomes (Does Face It show sufficient promise in terms of
improvements in key outcomes (resilience, emotional regulation, social and
emotional wellbeing, school attendance and school exclusions) to justify a
subsequent efficacy trial?)
a.
What are the perceived impacts of the programme for young people on
resilience, emotional regulation, social and emotional wellbeing, school
attendance and school exclusions?
3.
Mechanisms (What do young people’s perceptions of the programme suggest about
possible mediators and moderators?)
a.
What the perceived mechanisms of change are for young people
b.
Whether other factors (separate from the programme) contributed to identified
changes
4.
Feasibility: Fidelity of programme (Can Face It recruit and retain sufficient numbers of
young people from the target population and deliver the programme with fidelity?)
a.
What is the degree of young people’s engagement with programme activities?
b) In-person interviews with parents/caregivers
We will conduct in-person interviews with parents/caregivers of young people in the
intervention group in one school around six weeks after the programme finishes to
understand the perceived evaluability, acceptability and impact of the intervention. We will
use a voluntary response sampling approach - parents/caregivers will be invited to a drop in
session, with shopping voucher incentives included as compensation for parents/caregivers’
time and knowledge. Evaluation staff will be on hand to conduct up to five interviews with
caregivers.

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
37
The time frame stated above is indicative - we will be flexible with scheduling the drop in
session, aiming to align it with a time where caregivers are already in school (e.g. for parents
evenings) to maximise participation. This represents an exploratory part of our approach:
this activity will be tested in only one case study school to determine the feasibility of
including parent/caregiver views in the larger trial and to understand potential barriers to
using this approach. We recognise that there is a risk of a low response rate, and are
conducting this activity to assess whether it might be possible in any future trial.
Interviews will be audio-recorded with each participant’s permission and then transcribed.
Interview topic guides will be developed in collaboration with Khulisa, but are likely to focus
on the following research questions:
1.
Evaluability: Acceptability of data collection (Do we have enough confidence in the
feasibility of a randomised controlled trial (RCT), particularly in terms of recruitment
into evaluation, randomisation and outcome data collection, to justify a continuation
of the efficacy trial past this pilot trial?)
a.
Are interviews and surveys seen as acceptable?
b.
How can we optimise qualitative and quantitative data collection from
caregivers?
c.
Why participants chose to participate in interviews
d.
Why participants think others may choose not to participate
2.
Acceptability: (Do parents/caregivers find programme delivery acceptable?)
a.
Is the programme content acceptable to parents/caregivers?
b.
Is the programme delivery acceptable to parents/caregivers?
c.
What participants’ impressions of YP experience of the programme are
3.
Programme outcomes: (Does Face It show sufficient promise in terms of
improvements in key outcomes (resilience, emotional regulation, social and
emotional wellbeing, school attendance and school exclusions) to justify a
subsequent efficacy trial?)
a.
What are the perceived impacts of the programme on young people’s
resilience, emotional regulation, social and emotional wellbeing, school
attendance and school exclusions?
b.
What the perceived mechanisms of change for identified outcomes are
c) Online interviews with key school staff who manage the relationship with
Khulisa
The successful delivery of interventions is often reliant on the engagement of key
stakeholders, so it will be important to explore the attitudes of key school staff towards the
programme as well as key behaviours and decisions that may have affected programme
implementation.
We will interview three members of staff (one from each case study school) around eight
weeks after the programme finishes to understand the perceived feasibility, acceptability,

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
38
evaluability and impact of the intervention. Interviews will be conducted online,
audio-recorded with participants’ permission and then transcribed. Interview topic guides will
be developed in collaboration with Khulisa, but are likely to focus on the following research
questions:
1.
Feasibility: (Can Face It recruit and retain sufficient numbers of young people from
the target population and deliver the programme with fidelity?)
a.
What school staff perception of the referral process was
b.
Whether staff were able to identify eligible participants easily
2.
Acceptability: (Do teachers find programme delivery acceptable?)
a.
Is programme content and delivery acceptable (e.g. taking children out of
lesson time)
b.
What the facilitators and barriers to effective programme delivery are
3.
Evaluability: (Do we have enough confidence in the feasibility of a randomised
controlled trial (RCT), particularly in terms of recruitment into evaluation,
randomisation and outcome data collection, to justify a continuation of the efficacy
trial past this pilot trial?)
a.
Acceptability of randomisation
b.
What staff members’ experiences of participating in the evaluation were
4.
Programme outcomes: (Does Face It show sufficient promise in terms of
improvements in key outcomes (resilience, emotional regulation, social and
emotional wellbeing, school attendance and school exclusions) to justify a
subsequent efficacy trial?)
a.
What are the perceived impacts of the programme on young people’s
resilience, emotional regulation, social and emotional wellbeing, school
attendance and school exclusions?
b.
Whether they or their colleagues have reported any differences in behaviour
among young people
d) Online interviews with programme facilitators
We will conduct interviews with 3-6 programme facilitators approximately two weeks after the
programme finishes to help us to understand whether the programme was delivered as
intended, and to identify any facilitators and barriers to delivery.
Interviews will be conducted online, audio-recorded with participants’ permission and then
transcribed. Interview topic guides will be developed in collaboration with Khulisa, but are
likely to focus on the following research questions:
1.
Feasibility: Fidelity (Can Face It recruit and retain sufficient numbers of young people
from the target population and deliver the programme with fidelity?)
a.
Can Face It be delivered with fidelity?
b.
What factors affect fidelity?
c.
What variations in delivery are appropriate for effective implementation?

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
39
2.
Acceptability: (Do young people value the programme and feel the content and
delivery are appropriate to their needs? Do teachers and caregivers find programme
delivery acceptable?)
a.
Is Face It acceptable to young people at risk of youth violence?
b.
Does participation in a randomised evaluation affect participants’ views of the
intervention? If so, how?
c.
Does acceptability vary by ethnicity and race?
d.
Is the programme acceptable to teachers?
e.
What factors affect acceptability?
f.
How could the programme be adapted to increase acceptability?
3.
Evaluability: (Do we have enough confidence in the feasibility of a randomised
controlled
trial
(RCT),
particularly
in
terms
of
recruitment
into
evaluation,
randomisation and outcome data collection, to justify a continuation of the efficacy
trial past this pilot trial?)
a.
Is data collection possible and sufficient?
b.
What factors affect ease of data collection and completeness and quality of
data?
c.
How could the approach to data collection be improved to increase ease,
completeness and quality?
d.
Do practitioners feel that the evaluation impacts on intervention delivery? If
so, what elements of the evaluation affect delivery quality?
e.
Is randomisation acceptable to practitioners & teachers?
f.
What factors affect acceptability?
Qualitative analysis
All interview recordings will be transcribed verbatim to ensure accuracy and facilitate
subsequent analysis. Transcripts will be anonymised by assigning unique identifiers to each
participant, replacing their names or any identifying information.
The research team will immerse themselves in the data before starting the data
management and analysis approach. The team will adopt a framework approach to analysis,
whereby the key themes emerging from both the research questions and the data are
incorporated into an analytical framework where each column represents a subtheme and
each row a participant. The interview data is then charted (summarised) into that framework
in a comprehensive and systematic way. Once the data management stage is complete, the
research team will use the framework to carry out the analysis. This will involve looking at
each theme in turn and exploring the range of views held under that theme with a view to
developing categories. This will be done by grouping the views into clusters and exploring
the properties of each of these clusters until clear categories can be developed. Given the
nature of the pilot study and the size and likely diversity of the sample, it is highly likely that
the majority of the analysis will be descriptive and aim to clearly map out the range and
diversity of views that exist within each participant population on the key areas relevant to
the research questions. However, where possible the researchers will proceed to a higher

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
40
level of analysis and aim to look for patterns and linkages in the data, as well as
explanations.
Verbatim participant quotations and case examples will be used to provide evidence and
exemplify the theme(s) discussed.
Quality assurance will be provided by an experienced qualitative researcher.

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
41
6. Cost evaluation - data collection and
reporting
We will report the cost of delivering the intervention in the final report, following YEF costing
guidance. We will:
●
Use a bottom-up costing approach and break costs down into: prerequisites, set-up
costs, and recurring costs.
●
We will report the total cost for a typical single cohort receiving the intervention for
one round of delivery and the costs per participant for one round of delivery,
assuming full compliance. The programme will be delivered within 5 schools, twice
within each school, to two different groups of young people (to one group of Year 9s,
and to one group of Year 10s). Depending on heterogeneity in costs across cohorts,
we will either report average costs, or select a case which we think is most
representative of the costs we expect to be incurred in future, typical rounds of
delivery.
The organisations and practitioners involved in delivery are Khulisa and the intervention
schools. To report cost at the end of this study, we have produced a template for these
organisations to complete, covering staff costs, equipment/materials costs, programme
procurement costs, and buildings and facilities costs. This template has been reviewed by
Khulisa.
We expect most costs to fall within the following two categories:
●
Staff cost: cost of practitioners, supervisors, and managers involved in delivering
Face It.
●
Programme procurement costs: the cost for Khulisa to train and supervise staff, as
well as to produce and provide the programme-specific materials required to deliver
the programme.

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
42
7. Planned outputs
The final output of this pilot trial will be a publicly accessible report summarising the
evidence for each of the research questions and making a recommendation to YEF about
whether to proceed to a full-scale efficacy trial based on a holistic look at the evidence as a
whole, highlighting any discrepancies between data sources.
This will involve triangulating information from a range of qualitative and quantitative data
sources, from a range of relevant stakeholders comprising young people, parents, school
staff, and intervention delivery staff.
8. Ethics and data protection
Ethics
Overview
This trial was self-assessed as being high risk due to the inclusion of high risk participants in
the form of vulnerable young people. As a result we sought ethical approval from an
independent panel of external experts with experience of working with vulnerable children
and experience with safeguarding and child protection.
The independent ethics review committee (ERC) reviewed the following information:
●
Ethical review form
●
Consent forms and information sheets for young people and parents/caregivers of
young people
●
Student interview topic guide
●
Safeguarding and distress protocol
The ERC discussed any issues raised by the research with The Lab with the aim of finding
solutions that meet ethical requirements. The reviewers and the project manager agreed
solutions to any outstanding issues, and the resulting changes to the way the project is being
implemented have been included on the ethics form. The ERC was happy to approve the
project with the inclusion of these amendments.
If there are substantial changes while the research or evaluation is being implemented, the
ethics form will be revised and the revisions agreed with the ERC.

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
43
We conducted a separate internal ethical review for the materials for the participatory panel,
as we have judged this to be a separate package of lower risk work. This is because:
●
We will be asking young people to comment on our proposed materials and
approach rather than on the intervention in detail. This means that any personal
reflections and experiences will be limited
●
Panel members’ views and/or data won't be written up into outputs in the same way
as material from the evaluation. Instead, they will just shape the evaluation that we
deliver and the analysis that we conduct.
The participatory panel materials were approved through an internal ethics review process.
Informed Consent
The Head Teacher or other suitable member of each participating school’s Senior
Leadership Team will provide written consent for the school to participate in this study. This
will take place prior to the Lab/Khulisa receiving any referral data.
We will provide parents/caregivers of young people with an information sheet and withdrawal
form, providing the opportunity to withdraw their child from the evaluation prior to seeking
consent from young people. Any parents/caregivers who don’t want their child’s data to be
used in the evaluation will be able to withdraw their child from the trial prior to any data
collection. No data will be gathered on the child and the child will not participate in the Face
It programme.
We will obtain written consent from young people to take part in the evaluation prior to
baseline data collection. We will then also obtain written consent at every data collection
activity (embedded within surveys, at the beginning of YP interviews, at the beginning of all
other stakeholder interviews - parents/caregivers, facilitators, key school contact). The only
exception to this will be in special circumstances (e.g. where participants have accessibility
needs, such as severe dyslexia); in these instances we will seek verbal recorded consent.
We will take steps to ensure consent is fully informed, including comprehensive, but
accessible, information sheets, going over the information sheet content verbally, and giving
plenty of opportunity for participants to ask any clarifying questions. The information sheets
will include information about the nature and purpose of the study, the organisations
involved, what data will be collected, what will happen to it and why, and offer participants
the opportunity to opt out of the research. The information sheets will be sent via email and
will include a link to the privacy notice and will explain that we will ensure confidentiality and
anonymity in reports, providing all information in clear, accessible language. Where we think
consent is not being made on an informed basis we will not continue with data collection
activities for that participant.

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
44
We will review the information sheets and consent forms with the young person participatory
panel to ensure that the information is clear and accessible.
At any point until 1st May 2024, participants or their caregivers can withdraw their or their
child’s participation from the trial. If participants withdraw from the trial before any of their
personal data is shared with the Lab, then the young person will not participate in the
evaluation and no personal data will be shared.
Should we be informed that participants would like to withdraw from the trial after some
personal data has been collected we will not collect any additional personal data and will,
where possible, delete all of the participant’s personal data collected up to that point (other
than interview responses that have already been analysed and anonymised).
We will make it clear to participants that we will use their information to inform the findings of
our evaluation, which will be incorporated into a report, or other publicly publishable
materials. However, no identifying information will be disclosed in any such materials.
Safeguarding
Given the potential vulnerability of the young people involved in this evaluation, we have
ensured that we have a stringent safeguarding plan in place throughout.
All researchers with any contact with children will have an enhanced DBS check, will have
completed NSPCC Safeguarding training, and will familiarise themselves with the BIT Group
Safeguarding Policy and the project safeguarding and distress protocol.
Researchers conducting in-person data collection in schools will comply with all school
requirements for visitors.
Data protection
We will follow appropriate data protection processes in accordance with BIT processes,
including completing a Data Protection and Security Checklist and Data Protection Impact
Assessment which have both been reviewed and approved by BIT’s legal team.
The legal basis for processing personal data will be legitimate interest and the legal basis for
processing special category data will be scientific interest. Article 6(1)(f) of UK GDPR states
that “processing is necessary for the purposes of the legitimate interests pursued by the
controller or by a third party except where such interests are overridden by the interests or
fundamental rights and freedoms of the data subject which require protection of personal
data, in particular where the data subject is a child.”
The Lab has determined that there is a genuine purpose to process this data. The purpose
of processing personal data is to assess whether it will be possible to conduct a full-scale
randomised controlled trial evaluation of the Face It intervention, through assessing
feasibility, acceptance, and evaluability of the intervention, and estimating initial outcomes of

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
45
the intervention. Answering this question will help us, YEF, and the wider education and
youth violence prevention sector to better understand effective support to vulnerable young
people at risk of exclusion and becoming involved in violence. This will inform the evidence
around what works to improve the health, behaviours and life outcomes of young people,
particularly those at risk of, or who have engaged in, violent behaviours. Data processing is
necessary to complete a robust evaluation. The Lab does not consider that collecting and
gathering data for this trial will interfere with individuals’ interests, rights or freedoms. The
data subjects will include: at-risk youth, caregivers of at-risk youth, the delivery team at
Khulisa, and school staff responsible for the relationship with Khulisa.
All data shared with BIT will be processed in line with its data protection policy. A summary
of this policy can be found in Annex G. In the analysis, BIT will promote data quality and
security through the following measures.
●
All variables will be clearly named, coded and labelled before analysis.
●
Checks on the data received will be carried out for valid values, range, and
consistency against already held data.
●
Any modifications to datasets will be recorded in the analysis code, which will be
well-annotated.
●
Original raw datasets will never be amended.
●
Access to the project data will be restricted to project personnel.
●
All data stored by BIT will be backed up.
A project privacy notice for this project will be shared with participants and published on the
Lab’s website.
Data management
All quantitative and qualitative data will be stored in a secure Google Folder where access is
restricted to only researchers conducting the analysis. Data will be deleted upon completion
of the project in March 2025.
After the project has been completed, data will be shared with YEF, pseudonymised, and
stored in the YEF Evaluation Data Archive. All projects funded by YEF store data in this way.
Data in the archives is separated from any identifiable information.
Approved researchers may apply to access YEF data via the Office of National Statistics
(ONS) secure research service. This will be accessed via their own project space created in
the ONS secure research environment by the ONS. They may apply to the Department of
Education and Ministry of Justice to access the linked National Pupil Database-National
Police Computer data, and if successful it will be made available here to combine with the
YEF evaluation data, using pupil matching reference numbers. Researchers will only be able
to access pseudonymised data. All results will be published in a de-identified form.
This enables YEF to assess the impact of their projects long-term, as well as to conduct
quality assurance, reanalysis and methodological exploration across the outputs and results

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
46
published in the YEF funded evaluation reports; and pooled analysis of data from multiple
interventions, to explore what works in different geographic areas and for particular
subgroups (e.g. gender, ethnicity, child who have experienced periods of care etc) that may
be identified from the data collected from the projects themselves or from the linked
administrative datasets.
Further information about this process can be found on the YEF website.30
Quantitative data
Survey data
We will use SmartSurvey to collect the survey data. SmartSurvey produces a spreadsheet
where one row is a survey response. This will be used to code the survey outcomes using
the methods outlined in the outcome measures table.
Surveys will ask participants to record their name. This enables us to link survey responses
with demographic data and other outcome measures. Once survey responses have been
linked, participants’ names will be removed.
ImpactEd and programme administrative data
Khulisa is responsible for providing us with both the ImpactEd and programme administrative
data. All data shared with the lab by Khulisa will be received via a secure transfer link (Virtru
or Quatrix).
ImpactEd data will be collected via ImpactEd, shared with Khulisa who will in turn share it
with us. They have used ImpactEd before in the past and are confident the data will be of
high quality.
Programme administrative data includes the referral data and the programme delivery
data (e.g. attendance sheets, fidelity checklists). Referral data will be collected via an online
form (located on FormAssembly) completed by school staff. Khulisa will download the data
in a spreadsheet and share the relevant data with the lab. Programme delivery data will be
collected via Khulisa, and shared with the lab.
Qualitative data
Interview transcripts
Interview recordings will be uploaded to McGowan for transcription. All interview recordings
will be transcribed verbatim to ensure accuracy and facilitate subsequent analysis.
30https://youthendowmentfund.org.uk/wp-content/uploads/2021/07/YEF-Data-Guidance-Projects-and-
Evaluators.pdf

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
47
Transcripts will be anonymised by assigning unique identifiers to each participant, replacing
their names or any identifying information. Transcripts, observation notes, and any additional
relevant documents will be securely stored in a password-protected file area. Access to the
data will be restricted to only project team members involved in the analysis. Recordings will
be deleted after completion of the project, in March 2025.
9. Racial diversity and inclusion
The Lab is committed to conducting research in which equality, diversity and inclusion
principles are firmly embedded across all stages of this evaluation, including the design,
recruitment, data collection, and analysis.
i) Groups included in the programme and evaluation. Prior to eliciting student referrals
from schools, Khulisa will conduct work against unconscious bias with relevant staff
members using material developed with UNLRN. This will consist of educating staff about
unconscious bias and sharing strategies to minimise this when considering which students to
refer to the programme. Khulisa will also work to refine their target population and materials
communicating this to schools to ensure that the most appropriate young people are referred
to the programme.
We will work with Khulisa to develop a final sampling frame that meets the primary needs for
our evaluation. For example, our primary sampling criteria for interviews will be young
people who have engaged with Face It, and we will purposively sample young people across
the following key characteristics (as far as is possible within our sample size): ethnicity,
gender (including non-binary descriptors), year group, school, free school meal status (as a
proxy for socio-economic status), and whether young people display primarily internalising or
externalising behaviours.
ii) Inclusivity during the design of the evaluation
We propose adopting a participatory approach to the design of our evaluation materials and
in understanding our results. By including a small group of young people (up to five) who
have previously participated in the Face It programme within the last two years, we will
protect against researcher bias in the design of elements of our research. Their experiences
and understanding of the programme will inform our decision-making processes and ensure
that we appropriately capture participant experiences of the programme.
These young people will be selected using convenience sampling, but, if possible, we will
aim for the group to represent a range of demographic characteristics.

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
48
iii) Inclusivity during recruitment. The Lab will ensure that inclusive practices are central
to our recruitment process and that participant wellbeing is promoted by (i) being considerate
of the sensitivity of the topic area during recruitment - e.g. providing the option of a paired or
individual interview, within a school setting or online (ii) providing young people with
welcoming and accessible information sheets, co-designed with young people on your
participatory panel, that gives all of the information about data security, anonymity and the
reasons for undertaking research (iii) ensuring that information is accessible to young people
who struggle to engage with written text, such as through offering a hard copy version of any
documentation with a yellow overlay for individuals with severe dyslexia, or providing an
audio recording of information sheets and other relevant documentation (iv) we will offer
flexible interview times that allow for different groups and individuals the opportunity to
participate (see next section).
iv) Inclusivity during data collection
For the delivery of Face It, Khulisa works to ensure racial diversity of practitioners in a
manner that is representative of the demographics of the target school. For the evaluation,
we will:
1.
Minimise power imbalance: Negative effects of power-imbalance can lead to false,
distorted or limited information being disclosed, or the interviewee relating what they
think the interview wants to hear. To protect against this BIT will: (i) collect data
through individual or paired interviews; (ii) include tools within our enquiry process
that ensure that people participate in a more equitable way that is not simply through
verbal response. Researchers will endeavour to ensure inclusive facilitation through
displaying empathy and respect for participants’ views and experiences.
2.
Mitigate access issues: Enabling inclusive access to interviews requires
considering the factors that limit attendance. Access issues could include a lack of
time during the day to attend sessions, or distance from an in-person session. The
Lab aims to enable wide access to the sessions by offering to conduct interviews
within a school setting wherever possible. We could also offer remote sessions if
young people prefer, making accessing the session easier. The use of Google Meets
does not require new software to be installed and participants can join Google Meets
meetings through telephone connection, so do not need internet access. We will offer
to use other software such as Microsoft Teams or Zoom if this is more accessible to
the individual.
3.
Training for researchers: Prior to conducting interviews, researchers will complete
the NSPCC’s Introduction to safeguarding and child protection training and complete
a pre-interview workshop on interviewing best-practice with NJ Research Ltd.
iii) Wellbeing and safety during interviews. The Lab is conscious that young people who
engage in the evaluation could be vulnerable to negative and stressful impacts of the
research process. The Lab will work to ensure the wellbeing and psychological safety of
individuals during data collection by:

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
49
1.
Designing interview questions to minimise harm and maximise comfort: The
Lab will maximise wellbeing and minimise harm during qualitative data collection by
(i) structuring topic guides to begin the interview with simpler questions which can
build in complexity and difficulty as the participant becomes more comfortable and
rapport develops, (ii) depersonalising questions to elicit comfort and stronger
answers (e.g. instead of ‘what do you hate about X’, ask ‘If you had a magic wand,
what 3 things would you change about X to make it easier?’), (iii) being aware of
tension, discomfort or distress during the interview, repeating that the interview can
be stopped if participants wish, and repeatedly asking participants if they want to
continue, (iv) ensuring that researchers are aware of places to signpost participants
and offering this information, and (v) auditing the questions for their sensitivity within
the context before the interview.
2.
Allowing the participants to choose their environment for participating: Where
possible, the Lab will allow the interviewees to make decisions about the interview
setting - at their home, a public place or over the phone, enabled by the online
conference format.
3.
Reminding participants of anonymity and data security: The Lab will endeavour
to minimise stress by reminding participants that data collection is fully anonymous
and all identifiable information will be removed from the transcripts and report. It often
helps to say what this means, e.g. ‘We will describe you as a year 9 student from a
school in London’. The Lab can also offer the chance for them to check over the
anonymised transcript.
4.
Incorporating the views of the participatory panel: The Lab will seek the views of
the participatory panel on elements relating to interview logistics, such as whether to
proceed with individual or paired interviews, in order to ensure that arrangements are
acceptable to young people.
Inclusivity during data analysis: The Lab feels it is also important to preserve EDI
principles within the data analysis stage too. This will be achieved by engaging the
participatory panel to cross check our analysis and emerging themes, to ensure that we
have an inclusive and informed approach to analysis.

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
50
10. Risks
We have identified the following risks.
Risk
Impact
Likelihood
Mitigation
Exclusion outcome
confounded with
intervention (i.e. if
schools are less likely
to exclude pupils
while they are on the
programme)
Medium
Low
Unlikely as schools have previously excluded
pupils receiving Face it, and many teachers will
not be aware of the programme.
Spillover between
treatment and control
participants
Medium
Low
Ensuring that teachers do not sit in and
observe programme sessions will reduce the
chance of spillover. Working with new schools
who have not experienced the Face It
programme before, where teachers will not be
familiar with the programme content will also
reduce the likelihood of spillover. For example,
pupils in the control group of a school which
has worked with Khulisa before may
experience elements of the intervention if
teachers have previously been exposed to
programme elements and include them in their
lessons and classroom approach.
Unconscious bias in
pupil referrals (i.e.
disproportionately
referring children of
minority ethnic
backgrounds based
on unconscious bias
Medium
Medium
There is potential for unconscious bias due to
the use of teacher referrals. However, we will
aim to minimise the likelihood of this by using a
detailed referral form developed and used by
Khulisa. The form prompts referrals to consider
multiple factors such as students’ current needs
and adverse childhood experiences, reducing
the likelihood of referral solely due to bias.
In addition, prior to referrals, Khulisa will
provide schools with resources developed with
charity partner Chasing Prospects and
consultants UNLRN which aim to tackle
unconscious bias. As part of the pilot
evaluation, we will monitor this risk by
monitoring the characteristics of young people
being referred to the programme, and discuss
this concern with practitioners as part of
qualitative interviews.

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
51
Non-compliance by
staff with random
assignment
High
Medium
We will ensure that school and delivery
partners are aware of the implications of
non-compliance with randomisation prior to
recruitment. We will conduct randomisation
close to the start of delivery, by conducting
randomisation separately for each school. We
will also issue schools with an information
sheet reiterating the importance of compliance.
We will also encourage Khulisa to
communicate the importance of compliance
with randomisation in discussions with schools.
Low recruitment and
retention of schools
Medium
Medium
School recruitment and retention will be a
responsibility of Khulisa. Khulisa will aim to
reduce this risk by initiating discussions early
with schools and working to maintain positive
relationships with schools.
Low recruitment of
eligible young people
within schools
Medium
Low
Face It is typically oversubscribed and there
are processes in place to monitor the trajectory
of recruitment and retention
High dropout rate
from evaluation
among young people
Medium
Medium
Provision of incentives for survey completion
sessions for both the intervention and control
groups.
Sites (or participants
within sites) will not
engage with
evaluation activities
Medium
Medium
Some amount of attrition is to be expected in all
evaluations. We have several strategies that
aim to minimise
attrition, which include:
●ensuring buy-in from the Headteacher and/or
senior leadership team;
●identifying a specific individual responsible for
the relationship between the school and
Khulisa, and for communications with the Lab;
●ensuring that evaluation activities are
designed to be low-impact in terms of burden
or time, by avoiding unnecessary data
collection, and incentivising activities where
appropriate;
●informing sites with sufficient notice about any
planned activities with appropriate information
about how useful the evidence created by the
study will be; and
●where needed, utilising the relationships that
Khulisa has built with LAs and schools to
facilitate access/cooperation.

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
52
11. Timeline
This pilot study is expected to last 18 months from April 2023 to September 2024, subject to
appropriate school recruitment and young person referral. While the Lab will endeavour to
facilitate this, these processes are ultimately out of our direct control.
Phase
Activity
Description of activities during
phase
Led by
Target date
Project set up
Delivery
Planning
Internal delivery plans and project
management tools developed
Khulisa
July 2023
Recruitment and training completed
Khulisa
Aug 2023
Evaluation
design
Protocol published and detailed
internal plans and project
management tools development
The Lab
July 2023
Development of surveys, interview
guides and other data collection
tools
The Lab
July 2023
Statistical Analysis Plan developed
The Lab
Sep 2023
School
recruitment
Development of materials for
recruitment of schools explaining
evaluation, randomisation etc.
Khulisa
April 2023
Target number of schools agreed to
participate in the evaluation
Khulisa
By July 2023
Participatory
panel
Participatory panel session 1
The Lab
September
2023
Young
person
recruitment
At least 160 eligible young people
referred to the programme
Khulisa
Oct 2023
At least 160 young people
randomised into intervention or
control
Joint
Nov 2023
Project
delivery and
data collection
Baseline
data
collection
Young person baseline survey for
cohort 1 (Autumn term)
The Lab
Sep 2023
Young person baseline survey for
cohort 2 (Spring term)
The Lab
Jan 2024
Parent/caregiver baseline survey for
The Lab
Sep 2023

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
53
cohort 1
Parent/caregiver baseline survey for
cohort 2
The Lab
Jan 2024
Baseline attendance and exclusion
data
Khulisa
Dec 2023
Delivery
Delivery for schools in cohort 1
Khulisa
Oct 2023
Delivery for schools in cohort 2
Khulisa
Feb 2024
Post-interve
ntion data
collection
Programme data on retention, fidelity
etc. for cohort 1
Khulisa
Dec 2023
Young people end-of-programme
survey for cohort 1
The Lab
Dec 2023
Interviews with young people in
cohort 1
The Lab
Jan 2024
Programme data on retention fidelity
etc for cohort 2
Khulisa
Feb 2024
Young people end-of-programme
survey for cohort 2
The Lab
Feb 2024
Interviews with young people in
cohort 2
The Lab
Mar 2024
3 month
follow up
Young person follow-up survey
cohort 1
The Lab
March 2024
Parent/caregiver follow-up survey
cohort 1
The Lab
March 2024
Attendance and exclusion data
cohort 1
Khulisa
March 2024
Young person follow-up survey
cohort 2
The Lab
May 2024
Parent/caregiver follow-up survey
cohort 2
The Lab
May 2024
Attendance and exclusion data
cohort 2
Khulisa
May 2024
Data analysis
and report
write up
Analysis of
quantitative
data
Produce descriptive statistics of
admin data
The Lab
3 months after
final data
collection
Apply Statistical Analysis Protocol to
The Lab
3 months after

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
54
outcomes data
final data
collection
Internal and external quality
assurance
The Lab
4 months after
final data
collection
Analysis of
qualitative
data
Analysis of transcripts from
interviews with young people
The Lab
3 months after
final data
collection
Analysis of transcripts from
interviews with parents/caregivers
The Lab
3 months after
final data
collection
Analysis of transcripts of interviews
with practitioners and pastoral leads
The Lab
3 months after
final data
collection
Participatory
panel
Participatory panel session 2
The Lab
Following initial
analysis, prior
to reporting
Quantitative
and
qualitative
analysis
Quality assurance
The Lab
4 months after
final data
collection
Drafting and
publication
Draft report submitted for external
quality assurance
The Lab
6 weeks after
completion of
QA of analysis
Quality assurance
The Lab
2 weeks after
above
Final revisions and publication
The Lab
1 month after
above

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
55
Annexes
Annex A: Summary of Face It programme using the TIDieR
framework
Name: Provide a name or
phrase that describes the
intervention.
Face It
Why: Describe any
rationale, theory, or goal of
the elements essential to the
intervention.
Due to the high levels of trauma typically experienced by
Khulisa’s participants, Face It is scaffolded around
trauma-informed methodologies like Bruce Perry’s
neurosequential model (Perry 2006, 2009). According to this
model, in order to manage the impact of trauma in the brain,
participants first need help to ‘regulate’ their emotions, to calm
and ground them sufficiently to reduce their level of arousal
(amygdala hijack). This then enables them to ‘relate’ in a way
that enables them to trust others to keep them safe to try new
things. Only then is it possible to ‘reason’ with them, to work
with the cognitive part of the brain. The process of how
change happens set out by Bruce Perry’s model also ties into
our Theory of Change.
Khulisa’s theory of change supports the importance of six
clusters of social and emotional skills outlined in the EEF’s
Spectrum Framework to support young people’s social and
emotional wellbeing and positive longer-term life outcomes
(Education Endowment Foundation, 2017). These
competencies are inter-connected and at times overlapping.
The aim of Khulisa’s programme is to develop this broad
spectrum of social and emotional skills, but improved
resilience, coping skills, and self regulation are prioritised as
key short-term outcomes as in order to improve social and
emotional wellbeing, we must first improve young people’s
coping skills, emotional regulation and resilience.
Heading straight for the ‘Reasoning’ part of the brain is
unlikely to be successful if the young person is dysregulated,
stressed and disconnected. First, the person needs help to
regulate and calm their flight / fight / freeze responses and
build resilience and coping skills to lay the foundation for them
to engage with more challenging parts of the programme and
wider life, in order to improve levels of wellbeing.

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
56
What - Materials: Describe
any physical or informational
materials used in the
intervention, including those
provided to participants or
used in intervention delivery
or in training of intervention
providers. Provide
information on where the
materials can be accessed.
Before they ever deliver a programme, facilitators are given
access to:
1.
The full programme methodology
2.
The scaffolding process
3.
An example high level lesson plan that gives a
skeleton overview of the process and content of a
programme, to enable them to gradually immerse
themselves in the detail that sits behind it.
4.
A recommended reading list on each of the theoretical
concepts that underpin the programme.
In the set up phase of the programme, they have access to
the following documents:
●
Our participant profile- shared with schools to support
the referral process.
●
A draft letter to parents - sent out by schools.
●
A safer recruitment doc that explores our initial
safeguarding processes and procedures with Partners
●
A Programme Venue Health and Safety Risk
Assessment
Supporting materials for the delivery of the programme:
●
Lesson plan templates - these are edited for each
programme.
●
A programme fidelity checklist highlighting the core
elements every Face It programme should have
●
A list of icebreakers and games.
●
The poems and games we use on the programme.
●
1 page guides to key activities.
●
Handouts to share with participants as part of the
programme activities.
What - Procedures:
Describe each of the
procedures, activities, and/or
processes used in the
intervention, including any
enabling or support
activities.
Pre-programme one-to-one session: Discussion of
pre-programme survey
Pre-programme group session: The focus of this session is
around building a sense of safety. The activities are based on
creating rituals and clear boundaries. By the end of the
session: YP understand how to check in authentically so we
can respond to their needs, YP empowered to do/not do the
programme.
5-day intensive programme:
●
Day 1
○
Intro and contracting: Participants feel involved,
will have contributed to the group code, & be
contained by the knowledge that this is their
space.
○
Understanding me: Participants start identifying

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
57
their habits and understand that the journey to
change can be challenging, identify the impact
of beliefs on our behaviour, understand others’
perspectives & experience.
○
Introduction to triggers: Participants understand
that when we are triggered it can affect how we
perceive things around us as well as our
reactions
●
Day 2
○
Triggers continued: Participants understand the
power of triggers and the effects acting on
them can have and develop some negotiation
skills.
○
Emotional literacy: Participants will understand
their physiological and emotional responses
when triggered, have a better understanding of
their own and others’ experiences of
dysregulation, understand the relationship
between emotional responses & brain
development, setting the scene for coping
strategies.
○
Embedding emotional literacy: Participants will
be able to differentiate between observable
behaviours and the underlying emotions or
triggers, develop their ability to empathise and
communicate effectively by considering the
underlying emotions in themselves & others.
They will also practise self-awareness.
●
Day 3
○
Exploring challenges: Participants understand
different types of violence, and the distinctions
between them. They will explore less apparent
types of violence and begin to recognise the
common thread of power & control that links
different types of violent behaviour.
○
Challenges continued: Participants will apply
their understanding of different types of
violence to real-life scenarios, practice
empathy by playing different roles in freeze
frames and reflect on their own behaviours and
attitudes to inferiority/superiority and develop
an awareness of how their behaviour can
impact their relationships/contribute to conflict.
○
Challenge cycle: Participants will realise there
is a pattern to violence (it doesn’t just happen),
start to identify their own pattern of behaviour,
recognise they are not alone in their thoughts,
feelings, actions and needs and understand

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
58
how unmet needs contribute to reactions - all
setting the scene for coping skills on day 4.
●
Day 4
○
Understanding impact of behaviour &
developing coping skills: Participants will gain a
deeper understanding of the Challenge Cycle
and its impact on others. They will also practice
and embed effective strategies to manage
challenging situations and emotions building
their toolbox of healthy coping mechanisms.
○
Accepting my mask: Participants will explore
the concept of masks, connect with their
challenged selves, express themselves
artistically, and engage in meaningful dialogue
within a therapeutic and supportive setting.
○
Accept and appreciate me: Participants
connect with their authentic self, recognise
their skills, assets and attributes and consider
what’s important to them to start the healing
process.
●
Day 5
○
Asset building: Participants will build their
self-esteem, self-worth and their capacity to
present positively to others.
○
Future focus: Participants will connect with
their aspirations, develop a clearer
understanding of their past and future,
enhance their communication skills, and gain
insights into personal growth and
decision-making.
○
Appreciation and close: Participants will further
their self-acceptance, develop their resilience
and connect their progress to the overarching
learning outcomes, exercise choice and
agency, and conclude the day with a sense of
reflection and fulfilment.
Post-programme group session: The focus of this session
is reflection and helping participants to recognise their
personal strengths. Appreciation, gratitude and reflective
practice enable participants to realise their progress and to
consider how they can apply what they’ve learnt in their daily
lives
Post-programme one-to-one session: Most significant
change discussion, discussion of post-programme survey
Who: For each category of
intervention provider (such
as psychologist, nursing
Khulisa ensures that all facilitators are therapeutically trained
in art or dramatherapy so that they have the skills and training
required to create ‘safe containment’ for the group and to

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
59
assistant), describe their
expertise, background, and
any specific training given.
continually assess the pace and content of programmes
based on participants’ needs. There is no minimum level of
therapeutic training, but each programme has someone who
is therapeutically trained.
All programme managers and associate facilitators receive a
stringent 3-day training programme to ensure that they are
well-versed in programme content and methodology before
they ever deliver a programme. They will also complete levels
2 and 3 safeguarding training. Each programme manager
observes and supports 8 programmes as a co-facilitator
before they lead a programme of their own.
All facilitators are given access to our written methodology
and scaffolding process along with a programme manual
setting out key activities corresponding to specific parts of the
neurosequential model.
Facilitators receive ongoing support via monthly clinical
supervision, fortnightly/monthly supervision with their line
manager, monthly group supervision and peer to peer training
support every 6 weeks (during half-terms as there will be no
delivery). In addition during half-terms/summer provision,
Face It offers a weekly optional online session for staff to
practise activities together.
Facilitator backgrounds include counselling and youth work,
drama
and
movement
therapy,
education,
education
psychology, and dance movement therapy.
How: Describe the modes of
delivery (such as face to
face or by some other
mechanism such as internet
or telephone) of the
intervention and whether it
was provided individually or
in a group.
Pre-programme one-to-one session: One 60 minute
in-person session
Pre-programme group session: One 2.5 hour in-person
session
5-day intensive programme: One intensive 5-day
programme delivered in person each day from 9:30-15:30,
Monday-Friday
Post-programme group session: One 2.5 hour in-person
session
Post-programme one-to-one session: One 60 minute
in-person session
Where: Describe the type(s)
of location(s) where the
intervention occurred,
including any necessary
infrastructure or relevant
features.
School-based
Khulisa shares a list of programme venue room requirements
with schools as part of our service level agreements.

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
60
When and how much:
Describe the number of
times the intervention was
delivered and over what
period of time including the
number of sessions, their
schedule, and their duration,
intensity, or dose.
The following intervention components are spread over a
period of approximately 6 weeks. This estimates one session
a week, with a break for half term.
Pre-programme one-to-one session: One 60 minute
session
Pre-programme group session: 2.5 hours
5-day intensive programme: One intensive 5-day
programme delivered each day from 9:30-15:30
Monday-Friday
Post-programme group session: 2.5 hours
Post-programme one-to-one session: One 60 minute
session
Tailoring: If the intervention
was planned to be
personalised, titrated or
adapted, then describe
what, why, when and how.
Modification within programme description based on ongoing
facilitator assessment of group dynamics. For example,
facilitators choose an initial icebreaker from a predefined list
of options, depending on the dynamic of the group and
whether they aim to calm or energise participants.
Modification: If the
intervention was modified
during the course of the
study, describe the changes
(what, why, when, and how).
To be assessed as part of the pilot trial.
How well (planned): If
adherence or fidelity was
assessed, describe how and
by whom, and if any
strategies were used to
maintain or improve fidelity,
describe them.
Our programme delivery is often emergent, and dependent on
the
emotional
regulation
and
capacity
of
Khulisa’s
participants. Throughout each day of the programme various
icebreakers, games and energisers are used to assist in the
regulation of emotions. This ensures Khulisa’s Facilitators can
safely contain participants, whilst still providing an opportunity
for learning and education.
Our programme balances the flexibility afforded by this
emergent
process
with a very robust methodology and
scaffolding process based on Bruce Perry’s neurosequential
model (Perry 2006, 2009). To further ensure programme
fidelity and to quality assure our programmes we have
identified and closely monitor core milestones and activities
every Face It programme should contain. This means that as
different as the configuration of programme activities in
response to specific needs of the group might be, we will have
visibility of and an ability to ensure every Face It programme
touches on key milestones that we have identified are central
to effecting change. These are reviewed every six weeks as a
measure of accountability and quality assurance for the team.

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
61
How well (actual): If actual
adherence or fidelity was
assessed, describe the
extent to which the
intervention was delivered
as planned.
To be assessed as part of the pilot trial.

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
62
Annex B: Khulisa’s Theory of Change narrative and diagram
Khulisa delivers intensive social and emotional skills development programmes for young people in
schools, pupil referral units, prisons and communities. Programmes are delivered for up to 12
participants at a time over a 6-8 week period. We aspire to only work in organisations where we can
deliver training and support for professionals alongside direct delivery. This promotes culture change
and an organisation-wide focus on developing and sustaining social and emotional wellbeing. We
believe that schools and prisons have a crucial role to play in promoting wellbeing in everything they
do, for both young people and the staff who support them.
When we use the term ‘social and emotional wellbeing’, we mean “the strength and capacity of our
minds to grow and develop, to be able to overcome difficulties and challenges and to make the most
of our abilities and opportunities” (Young Minds, 2006).
Why focus on Social and Emotional Learning and Wellbeing? Research and evidence clearly shows
that social and emotional learning (SEL) and wellbeing is central to successful life outcomes for all
young people (Moffitt et al., 2011; Sacker et al., 2007; Gutman et al., 2012; Department for
Education, 2011). There is international evidence that SEL reduces mental health issues, prevents
criminal behaviour and promotes not only academic achievement, but lifelong learning, satisfaction
and success (Cefai et al., 2018; PHE, 2017; WHO, 2012). Evidence shows that SEL is successful for
young people from across ethnic, cultural and economic backgrounds and is especially effective for
young people from disadvantaged backgrounds (Ceefai et al., 2018; Clarke et al, 2015). The positive
impacts of quality SEL are proven to persist over time and deliver strong financial returns on
investment (Cefai et al., 2018), with Public Health England reporting a cost benefit ratio of 5:1 (PHE,
2017).
The Context: Despite this evidence, there is inconsistent delivery of SEL in schools, prisons and in the
community. We are facing a wellbeing crisis in the UK, with young people reporting the lowest levels
of wellbeing, second only to Japan (Varkey Foundation in the Guardian, 2017) in an international
survey of 20 countries. Linked to this, more UK children than ever, 1 in 10, are suffering from mental
health issues (NHS Digital, 2017; (Children’s Society, 2008), yet services have diminished. Two thirds
of those referred for specialist care received no treatment at all in 2017 (Spurgeons children’s
charity). At the same time, the number of children excluded from school continues to rise - there are
now the equivalent of more than 35 permanent exclusions every school day in England (DfE, 2018).
Persistent disruptive behaviour remains the most common reason for these exclusions (DfE, 2018)
and we know that in the vast majority of cases, this behaviour is a cry for help and a signal of poor
social and emotional wellbeing. We also know that excluded young people are more likely to end up
in the criminal justice system. Around 60% of the prison population were excluded from school
(IPPR). Pupils excluded by the age of 12 are four times more likely to be jailed as adults (McAra, L. &
McVie, S, 2010). The UK prison system is struggling to cope with increasing levels of violence and

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
63
assault and the proven reoffending rate is unacceptably high for juvenile offenders at over 40%
(Ministry of Justice, 2018).
The majority of young people in custody, or at risk of entering custody, have experienced neglect,
abuse and trauma. They often have a history of adverse childhood experiences, exclusion, and
disempowerment. As many as 90% of people in prison are estimated to have mental health
problems,(Durcan, 2016), and almost half the children in Young Offender Institutions are Looked
After Children (Prison Reform Trust, 2018). Childhood years are vital for developing positive social
and emotional skills and well-being. Traumatic or adverse experiences in the crucial early years have
a damaging effect on the development of these core cognitive and emotional skills, which are
essential for managing thoughts, feelings, and behaviour, as well as interactions with others (WHO,
2012).
Many of the young people who come into contact with the criminal justice system carry a deep sense
of shame and have been negatively labelled and excluded from society. Identity loss, low self-esteem
and a socially devalued sense of self are all markers of poor mental well-being (Thoits in Sharma &
Sharma) and also a central driver of criminal behaviour (Maruna and Farrall, 2004; McNeill and
Weaver, 2010 in Beyond Youth Custody).
Our Participants: We believe that all young people should have access to quality SEL, however we
focus our limited resources on providing this for young people who are most at risk of exclusion,
offending or re-offending. Khulisa works with young people who:
●
Suffer from poor mental health and low levels of social and emotional well-being;
●
Have experienced trauma and / or adverse childhood experiences;
●
May have behaved violently (physically or non-physically) towards themselves or others;
●
Have been - or are at risk of being - excluded, marginalised and labelled;
●
Lack confidence, self-esteem, self-worth and a sense of identity;
All of our young participants have potential and courage. We work with these young people because
we believe that change is possible. We understand the effects of trauma and adverse childhood
experiences and how they present in the body and are expressed through behaviour. We have
identified an unmet need to treat young people who’ve been excluded from school or society with
respect, and to work with them as the central agents of their own journey to positive change.
How do we work?
The Khulisa programme is emergent and participant centered. Khulisa’s therapeutically-trained
facilitators work through the content that is most relevant for the needs of the specific participants in
the room, at a pace that matches their ability to absorb the learning. This means that no Khulisa
programme is exactly the same as another.

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
64
We deliver the programme through a mixture of group activities and individual reflection sessions,
using a range of creative activities including storytelling, mask-making, games and role-play. The
programme is delivered by qualified art and drama therapists who use a therapeutic process to
support the young people through a journey of self-awareness, self-discovery and realisation.
The programme is based on evidence of what works to build key social and emotional skills and is
centered around 5 core approaches:
Dramatherapy: Khulisa’s trained art and drama therapists use creative techniques (e.g. music, art,
role-play, story-telling). These activities are associated with increased self-esteem, self-efficacy,
empowerment, social and emotional competency, emotional intelligence and resilience (Daykin,
2008; Bungay, 2013, Zarobe and Bungay 2017)·
Small Group and 1-1 sessions: Our peer groups explore positive and negative relationships together,
while our ongoing 1-1 sessions help to embed individual learning. This approach is proven to develop
emotional intelligence and social and emotional competence (Clarke, 2015).
Asset-based theory: through appreciation circles and strength-based observation, we help children
realise and build on their strengths, essential in developing confidence and coping skills. This is
proven to be particularly effective in reducing anxiety, improving coping skills and resilience (Collins,
2004; Sin et al., 2009; Neil et al. 2009)·
Restorative Approaches: techniques such as circle dialogs, talking pieces, sociograms etc. empower
young people to improve their social and emotional competency, relationships and emotional
intelligence (Ortega et al., 2016).
The Process of Change
Short-term Changes
A range of evidence and direct delivery experience, supports the importance of the six clusters of
social and emotional skills outlined below for social and emotional well-being and positive
longer-term life outcomes (Young Foundation, 2012, Education Endowment Foundation, 2017).
These competencies are inter-connected and at times overlapping. Although the programme is
designed to develop this broad spectrum of social and emotional skills, improved resilience, coping
skills and emotional self regulation are prioritised as key short-term outcomes.
These skills are the main focus of the programme and are seen as providing the foundation for the
successful development of other social and emotional skills. This relates to Dr Bruce Perry’s ‘The

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
65
Three R’s’ model (Regulate, Relate, Reason) for learning among young people who have experienced
trauma (Dr Bruce Perry, Three Rs Model). Heading straight for the ‘Reasoning’ part of the brain is
unlikely to be successful if the young person is dysregulated and disconnected from others. First we
must help the child to regulate and calm their flight / fight / freeze responses. Second, we must build
trust, relate and connect. We must build resilience and coping skills to lay the foundation for the
young person to engage with more challenging parts of the programme. Only then can we effectively
support the young person to learn, reason, reflect, articulate, remember and become self-assured
enough to fully engage.
Longer-term Changes
Improved social and emotional skills form the basis for improved longer-term social and emotional
wellbeing, and positive life outcomes. These longer-term outcomes are not limited to the absence of
crime, exclusion or negative behaviour. They include meaningful engagement with others and
society, positive and productive life choices and a lifestyle that promotes well-being (Goodfellow et
al, 2015 in BYC). In practice, this will look different for every individual. The focus is on flourishing and
thriving rather than compliance and desistance.
Overall, we look for positive long-term changes in the following outcomes across all programmes:
●
Social and emotional wellbeing – we measure both how a participant feels and how well
they feel they are functioning - this is the key to achieving positive longer-term outcomes.
Depending on the type of intervention delivered, we also monitor the following longer-term
observable behaviours:

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
66
●
In-school / prison behaviour
●
Academic attainment
●
Return to mainstream schooling
●
Re-offending rates
●
Engagement in productive activity
●
Transition to independent living
●
Building strong support networks
Our own data shows that improved and social emotional skills and wellbeing are associated with
positive external changes in behaviour. We believe that this positive behaviour in-turn further
strengthens and reinforces wellbeing, promoting a positive and continued cycle of change over time.
This process of change is complex and non-linear. Working with young people in difficult situations
with complex needs requires intensive intervention and an awareness that relapse is possible as part
of this process.
Khulisa’s Theory of Change Diagram
The below image brings together short and longer-term outcomes, illustrating Khulisa’s overall
high-level Theory of Change:
Enabling Factors:
Listed below are the key enabling factors that we believe need to be in place for the intervention to
be successful:
●
Highly skilled, trauma-informed drama-therapists are adept at working with young people;
●
Positive relationships and a sense of trust between the Khulisa facilitator(s), and with the
other group members;
●
Khulisa facilitators self-disclose, creating an open, safe environment;

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
67
●
Young people feel respected and feel no judgement throughout the programme;
●
Young people feel a sense of enjoyment, fulfilment and / or achievement as part of the
programme;
●
Young people feel that their voice is heard throughout the programme;
●
Carefully selected partner institutions (schools pupil referral units, prisons, community
organisations) share our core values and vision of success;
●
Partner institutions understand the power of monitoring and evaluation and support and
engage with data collection activities;
●
Professionals in partner institutions are willing to fully engage with and adopt the Khulisa
methodology;
●
A prison and education sector that is able to invest in preventative solutions for the
long-term.
Our Assumptions
While there is strong, consistent evidence proving the importance of social and emotional skills and
wellbeing for positive life outcomes, specific evidence around what works and the causal
mechanisms of change is still emerging. For this reason, we recognise that the intervention is based
on a series of assumptions.
Causal link assumptions:
●
Overall social and emotional wellbeing is strongly influenced by the 6 clusters of social and
emotional skills identified in the framework presented above (the EEF SPECTRUM
framework), in particular, resilience and coping;
●
Improvements in social and emotional wellbeing resulting from Khulisa’s intervention are
sustained over the long-term (at least 12 months post intervention);
●
Positive social and emotional wellbeing promotes positive mental health;
●
Improved knowledge, skills and wellbeing of professionals leads to improvements in the
social and emotional skills and wellbeing of the young people they support.
Equally important for the success of the programme are a series of practical implementation-focused
assumptions:
●
Partner institutions are selected purposefully and rigorously to ensure mutual responsibility
for programme success;
●
Partner institutions share the same values and vision;
●
Partner institutions have the resources, commitment, leadership and capacity to translate
trauma-informed theory into practice;
●
Partner institutions fully understand and buy-in to the Khulisa methodology;
●
The availability of a consistent, private, safe and adequate room in which programme
delivery can take place without disruption;
●
The ability of Khulisa facilitators to create a space where participants feel calm and able to
openly express themselves without fear of judgement.

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
68
●
Participants are able to commit to the process and are able to show up for the programme
consistently.
What Makes Khulisa Unique?
Khulisa is providing a service that is currently not available for young people in schools, prisons and
community organisations across the UK. We are the only organisation to provide all of the following
as part of one package:
●
Intensive support to young people most at need, addressing root causes not symptoms;
●
Targeted support for young people with complex needs;
●
A whole school / prison approach, aspiring to create cultural change and embedding our
services in an institution over a significant period of time (usually 3 years) until sustainable
institutional-level change is achieved;
●
Programmes run only by highly qualified art and drama therapists who are skilled enough to
deliver our methodology via a flexible approach whilst guaranteeing safety and security;
●
Methodology built on an strengths-based approach (incorporating the Good Lives Model)
recognising the assets and abilities of the young people we work with, no matter what
situation they are in;
●
A strong commitment to evidence-based approaches with commitment to quality
monitoring and evaluation to use data to influence policy as well as practice.
References
●
Beyond Youth Custody: Supporting the Shift: Framework for the effective resettlement of
young people leaving custody accessed:
http://www.beyondyouthcustody.net/wp-content/uploads/Now-all-I-care-about-is-my-futur
e-Supporting-the-shift-full-research-report.pdf
●
Bungay, H. & Vella-Burrows, T., "The Effects of Participating in Creative Activities on the
Health and Well-Being of Children and Young People: A Rapid Review of the Literature."
Vol.133 (1) (2013) Perspectives in Public Health pp44-52
●
Education Endowment Foundation; (2017) Social, Psychological, Emotional, Concepts of self,
and Resilience outcomes: Understanding and Measurement (SPECTRUM) A brief report of a
conceptual mapping exercise accessed:
https://educationendowmentfoundation.org.uk/public/files/Evaluation/SPECTRUM/EEF_SPE
CTRUM_Guidance_Document_Conceptual_mapping.pdf
●
World Health Organisation (2012), Risks to Mental Health: An Overview of Vulnerabilities and
Risk Factors
http://www.who.int/mental_health/mhgap/risks_to_mental_health_EN_27_08_12.pdf
●
Sharma, S. & Sharma, M. Psychol Stud (2010) 55: 118.
https://doi.org/10.1007/s12646-010-0011-
●
Cefai, C.; Bartolo P. A.; Cavioni. V; Downes, P.; Strengthening Social and Emotional Education
as a core curricular area across the EU. A review of the international evidence, NESET II
report, Luxembourg: Publications Office of the European Union, 2018. doi: 10.2766/664439
●
The Young Foundation (2012), A framework of outcomes for young people accessed:
https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_d
ata/file/175476/Framework_of_Outcomes_for_Young_People.pdf

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
69
●
Young Minds, 2006, accessed:
https://youngminds.org.uk/search-results/?terms=mental+health+and+wellbeing
●
Huppert FAaBN (2004) Well-being: towards an integration of psychology, neurobiology and
social science. Biological Sciences 358 1447.
●
Ryff CD, B. H. Singer, et al. (2004) Positive health: connecting well-being with biology.
Philosophical Transactions of the Royal Society B: Biological Sciences 359: 1383.
●
Keyes C.L.M. (2007) Promoting and protecting mental health as flourishing: A
complementary strategy for improving national mental health. American Psychologist 62: 95.
●
Steptoe A WJ, Marmot M. (2005) Positive affect and health-related neuroendocrine,
cardiovascular, and inflammatory processes. . Proceedings of the National Academy of
Sciences of the United States of America 102: 6508.
●
Chida Y SA (2008) Positive psychological well-being and mortality: A quantitative review of
prospective observational studies. . Psychosomatic Medicine 70: 741.
●
McAra, L. & McVie, S, 2010). Youth Crime and Justice: Key Messages from the Edinburgh
Study of Youth Transitions and Crime. Criminology & Criminal Justice, Vol.10(2), pp.179-209
●
Ministry of Justice, "Her Majesty’s Prison and Probation Service Offender Equalities Annual
Report 2017/18" (2018)
●
Durcan, G. (2016). Mental health and criminal justice: Views from consultations across
England and Wales. Centre for Mental Health
●
Prison Reform Trust, "Prison: The Facts" Bromley Briefings (2018)
●
Zarobe, L and Bungay, H "The role of arts activities in developing resilience and mental
wellbeing in children and young people a rapid review of the literature" Vol.137 (6)
(2017) Perspectives in Public Health pp337-347

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
70
Annex C: Power calculations
Power calculations using a within-schools design can be found below.
Within-schools trial
n
n (per arm)
MDES
160
80
0.45
Power calculation was determined on the basis the following assumptions:
●
Power: 0.8
●
Significance level: 0.05
●
Level of randomisation: individual-level (i.e. individual students will be randomised)
●
Number of trial arms: 2 (i.e. intervention and control)
●
Outcome measure: SDQ
●
Standard deviation: 6.5 in both groups
Stata code:
power twomeans 10, n(160) power(0.5) sd(6.5)

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
71
Annex D: Face It milestones
The following activities have been identified by Khulisa as essential to the core content of the
Face It intervention. As such, while elements of programme content may vary in timing and
specific implementation, a programme must contain all of these activities to be considered to
be a typical Face It intervention.
The following activities comprise the 5-day programme; core activities are highlighted in
bold.
Day 1
Intro and contracting
Recap why we are here
Recap what the programme is and how it will work
Icebreaker
Check-in circle
Code of honour
Check-out
Understanding me
Check-in circle
Game
There’s a hole in my pavement
Sociograms
Game
Check-out
Introduction to triggers
Check-in circle
Game
Recap AM session
Cookie story
Window of tolerance
Emotional literacy wall/mapping

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
72
Game
Check-out
Day 2
Triggers continued
Check-in
Game
Recap
Trigger game
Triggers
Game
Check-out
Emotional literacy
Check-in
Game
Fight, flight, freeze
Body mapping
Intro to brain development
Game
Check-out
Embedding emotional
literacy
Check-in
Game
Iceberg
Grounding activity
Game
Check-out
Day 3
Exploring challenges
Check-in
Game
Recap
Sociograms
Step in the middle

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
73
The 3 levels of violence
Game
Check-out
Challenges continued
Check-in
Game
Role play
Inferior/superior
Game
Check-out
Challenge cycle
Check-in
Game
Challenge cycle
Embedding learning
Game
Check-out
Day 4
Understanding impact
of behaviour and
developing coping
skills
Check-in
Game
Ripple effect
Are you a carrot, egg, or coffee bean
Coping skills activity
Embedding coping skills
Game
Check-out
Accepting my mask
Check-in
Game
Please hear what I am not saying
Mask-making

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
74
Check-out
Accept and appreciate
me
Check-in
Game
Poem
Hat-making
Game
Check-out
Day 5
Asset building
Check-in
Game
Recap
3rd person intro
Game
Check-out
Future focus
Check-in
Game
Future/back
Dragon’s den
Two wolves: A Cherokee Teaching
Game
Check-out
Appreciation and
close
Check-in
Game
Sweet appreciation
Pencil-maker
Game
Check-out

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
75
Annex E: Evaluation team experience
●
Tom McBride is the Director of the Ending Youth Violence Lab and has over 15
years of experience in research and evaluation roles. He is the former Director of
Evidence at the Early Intervention Foundation and Head of Strategic Analysis at the
Department for Education. Tom will have overall responsibility for the delivery and
quality of this work.
●
Jack Martin is an Assistant Director within the Ending Youth Violence Lab and has
over 8 years of experience working at the Early Intervention Foundation and sits on
the Government’s Trials Advice Panel. Jack will oversee the delivery of the work and
support, supervise and quality assure the work of the project team.
●
Alice Worsley is an Associate Policy Advisor based in the Health and Wellbeing
team at BIT. She has worked on projects across a range of policy areas and
methodologies. Alice will project manage and coordinate the project.
●
Patrick Taylor is a Principal Research Advisor and leads BIT's education and youth
evaluation work, supporting the design, improvement and evaluation of complex
interventions in these fields. Patrick will provide support and quality assurance for the
pilot evaluation.
●
Naomi Jones is a highly experienced social research consultant who specialises in
helping organisations to design, commission, deliver and use research better, with
over 18 years applied research experience. Naomi was formerly head of social
attitudes at NatCen, where she led a mixed-method research team and oversaw the
British Social Attitudes Survey. Naomi will lead the qualitative evaluation.
●
Lilli Wagstaff is a quantitative research advisor in the Home Affairs and Security
team at BIT and leads the evaluation and day-to-day delivery of a number of projects
focusing on policy areas including reducing violence and recidivism. Lilli will lead the
quantitative evaluation.
●
Niall Daly is a Research Advisor in the Health and Wellbeing team at BIT,
specialising in trial design, implementation, and quantitative data analysis across a
range of projects within the health space. He will support the quantitative evaluation.
●
Tess Moseley-Roberts is an Associate Policy Advisor in the Home Affairs, Security
and International Development team at BIT. She has worked across peacebuilding,
anti-corruption and education in the international space and a number of UK criminal
justice projects. Tess will support the qualitative evaluation.

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
76
●
Dr Nick Axford is an Associate Professor at the University of Plymouth with a
specialism in evidence-based prevention and early intervention to improve child
well-being, with a particular focus on developing or adapting interventions and
evaluating their effectiveness and implementation. Nick has been a Member of the
Early Intervention Foundation Evidence Panel since 2014 and will be providing
expert advice and challenge on the design and delivery of the evaluation on a
consultancy basis.

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
77
Annex F: Schedule for recruitment, interventions and assessments
Procedures
Data collection timepoints
Screening
Baseline
Treatment phase
End-of-programme
3-month follow-up
Referral form
Screening log
Eligibility
Informed consent
Demographics
Randomisation
Intervention delivery
Compliance
Fidelity measures
Screening
Baseline
Treatment phase
End-of-programme
3-month follow-up
Attendance/ engagement logs
Session summary forms
Outcome measures
Screening
Baseline
Treatment phase
End-of-programme
3-month follow-up

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
78
School attendance
SDQ (parent and young person
self-assessment)
SWEMWBS
Children’s Hope Scale
The Emotional Regulation
Questionnaire
The Self-Report Delinquency Scale
The Problem Behaviour Frequency
Scale (overt and relational
victimisation subscales only)
School Exclusion
Acceptability of content
Control group services as usual
received
Qualitative interviews with young
people
Qualitative interviews with parents

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
79
Qualitative interviews with key school
staff
Qualitative interviews with
practitioners
Withdrawal from programme

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
80
Annex G: BIT data protection policy summary
The General Data Protection Regulation (GDPR) imposes certain obligations upon
Behavioural Insights Limited (BIT), and other companies within the group, as Controllers and
/ or Processors in relation to processing Personal Data.
BIT takes these obligations seriously. BIT is committed to respecting the rights of all
individuals whose personal data it processes:
1.
In relation to data security, BIT has implemented appropriate measures to ensure
the secure storage and handling of Personal Data, including obtaining a Cyber
Essentials Plus certification and developing a comprehensive Data Handling
Protocol.
2.
In relation to data protection and privacy rights, our data processing activities are
conducted according to the principles relating to the processing of Personal Data set
out in the GDPR, including that Personal Data shall be processed lawfully, fairly and
in a transparent manner, and in a manner that ensures the security of the Personal
Data. BIT has policies and procedures in place to ensure compliance with these
principles.
More information on how we handle Personal Data in relation to projects we are working on
is detailed below.
BIT is registered with the UK ICO under the terms of the Data Protection Act 2018. Our
registration number is ZA038649.
Privacy by design
BIT conducts all trials and research projects with a privacy by design approach to protect
and maintain the privacy and security of research participants’ and research subjects’ data.
We work closely with clients, government departments and research partners when
designing interventions to ensure that a privacy by design approach is implemented and
respected.
Our data protection and data security policies and procedures reflect necessary legislative
requirements and set out the standard to which BIT staff should work when dealing with
Personal Data, including:
●
Attendance at mandatory data protection training for all employees;
●
Identifying data requirements from the outset of each project;
●
Minimising use of Personal Data where possible and ensuring we have the right to
handle any Personal Data where successful project delivery is reliant on using it;
●
Putting in place data processing agreements with all clients and suppliers to clarify
data handling arrangements ahead of any data being transferred;

The Ending Youth Violence Lab / Face It Evaluation - Pilot study protocol
81
●
Complying with all relevant data residency requirements and implementing
appropriate technical and organisational measures, to protect data and avoid
unauthorised access, internally and externally;
●
A clear internal reporting process in the event of a data breach, to consider the
nature of the breach and identify any necessary action, including whether the breach
should be reported to the relevant authorities, i.e. the Information Commissioner’s
Office in the UK or the Office of the Australian Information Commissioner;
●
Clear procedures on retention and destruction of Personal Data to avoid keeping
hold of Personal Data longer than necessary for the purposes of each project; and
●
Implementing robust investigation and reporting procedures in relation to any data
breach or security issues that arise both within our own systems and those of our
clients, partners and suppliers.
Data Protection Officer
The BIT group of companies has appointed a Data Protection Officer (DPO) who is the first
point of contact for any issue regarding data protection and data security. The DPO can be
contacted via email at legal@bi.team or by writing to us at:
Legal Team, Behavioural Insights Limited, 58 Victoria Embankment, London, EC4Y 0DS,
United Kingdom.