COMMISSION STAFF WORKING DOCUMENT EVALUATION Accompanying the document REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF THE REGIONS on the ex post evaluation of the 2014-2020 justice programme and interim evaluation of the 2021-2027 justice programme
Date: 2025-06-03
Institutions: Directorate-General for Justice and Consumers, European Commission
Subjects: EU programme, European judicial area, access to the courts, democracy, fundamental rights, independence of the judiciary, judicial cooperation in civil matters in the EU, judicial cooperation in criminal matters in the EU, rule of law, vocational training

EN 

  EN 

   EUROPEAN COMMISSION  Brussels, 3.6.2025  SWD(2025) 134 final  COMMISSION STAFF WORKING DOCUMENT EVALUATION Accompanying the document REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF THE REGIONS  on the ex post evaluation of the 2014-2020 justice programme and interim evaluation of the 2021-2027 justice programme {COM(2025) 267 final}  
 
 
Table of contents 

1. 

Introduction ............................................................................................................... 1 

1.1.  Purpose and scope of the evaluation .......................................................................... 1 

1.2.  Methodology framework ........................................................................................... 2 

2.  What was the expected outcome of the intervention? ............................................... 3 

2.1.  Description of the interventions and their objectives ................................................ 3 

2.2.  Intervention logics ..................................................................................................... 7 

2.3.  Point(s) of comparison............................................................................................. 11 

3.  How has the situation evolved over the evaluation period? .................................... 12 

3.1.  Implementation of the justice programme 2014-2020 ............................................. 12 

3.2.  Implementation of the justice programme 2021-2027 ............................................. 13 

4.  Evaluation findings (analytical part) ....................................................................... 18 

4.1.  To what extent was the intervention successful and why? ...................................... 18 

4.1.1. 

4.1.2. 

4.1.3. 

Effectiveness ................................................................................................... 18 

Efficiency ........................................................................................................ 25 

Coherence ....................................................................................................... 33 

4.2.  How did the EU intervention make a difference and to whom? .............................. 35 

4.3.  Is the intervention still relevant? ............................................................................. 37 

5.  What are the conclusions and lessons learnt? .......................................................... 42 

5.1.  Synergies between the 2014-2020 and the 2021-2027 justice programmes ............ 42 

5.2.  Conclusions and lessons learnt ................................................................................ 43 

Annex I. Procedural Information ...................................................................................... 48 

Annex II. Methodology and Analytical models used ....................................................... 51 

Annex  III.  Evaluation  matrix  and  details  on  answers  to  the  evaluation  questions  (by 
criterion) .................................................................................................................. 62 

Annex IV. Overview of benefits and costs, simplification and burden reduction ............ 81 

Annex V. Stakeholders consultation - Synopsis report .................................................... 95 

Table of abbreviations .................................................................................................... 104 

 
 
 
 
 
 
 
1.  INTRODUCTION 

This staff working document (SWD) presents the results of the ex post evaluation of the 
2014-2020 justice programme1 as well as the interim evaluation of the 2021-2027 justice 
programme2. The evaluation is subject to a report to the European Parliament, the Council, 
the European Economic and Social Committee, and the Committee of the Regions. 

1.1. Purpose and scope of the evaluation 

The  2014-2020  and  2021-2027  justice  programmes  are  two  funding  programmes 
contributing to the development of a European area of justice based on the rule of law. 

The justice programme (2014-2020) contributed to the further development of a European 
Union area of justice based on mutual recognition and mutual trust, by promoting judicial 
cooperation  in  civil  and  criminal  matters  and  helping  train  judges  and  other  legal 
practitioners. Since 2021, the justice programme (2021-2027) continues contributing to the 
further  development  of  a  European  area  of  justice  based  on  the  rule  of  law,  including 
independence and impartiality  of the judiciary, on mutual  recognition,  mutual  trust  and 
judicial cooperation. It aims at strengthening democracy, rule of law, and the protection of 
fundamental rights. 

Considering the links between the two programmes, the Commission carried out a joint 
evaluation made up of two components. The first component is the ex post evaluation of 
the  2014-2020  justice  programme,  assessing  its  longer-term  impact  and  sustainability 
effects, based on the results of the first part of the ex- post evaluation3 completed in 2022. 
The  second  component  is  the  interim  evaluation  of  the  2021-2027  justice  programme, 
assessing its preliminary achievements. 

The first part of the ex post evaluation aimed to report to the European Parliament and the 
Council  as  provided  by  Article 14  of  Regulation  No 1382/2013.  Considering  that  a 
significant number of projects were still ongoing, the first part ex post evaluation assessed 
the preliminary achievements of the 2014-2020 programme, which fed into the design of 
the activities of the current one. 

The  evaluation  assesses  from  a  longitudinal  perspective  to  what  extent  the  respective 
activities and expected outcomes achieved the objectives of the two programmes against 
the criteria of effectiveness, efficiency, coherence,  EU added value, and relevance. The 
evaluation  also  assesses  the  synergies  between  the  two  programmes  and  the  new 
programme architecture’s added value. In a forward-looking perspective, the evaluation 
provides lessons learnt for the ongoing implementation of the 2021-2027 programme as 
well as the following funding cycle. 

The  ex  post  evaluation’s  scope  covers  the  implementation  of  the  2014-2020  justice 
programme  from  1 January  2014  until  31 December  2020  and  all  of  the  programme’s 
activities  that  took  place  in  all  participating  countries4  during  that  time.  The  interim 
evaluation covers the implementation of the 2021-2027 justice programme from 1 January 

Regulation 

No 1382/2013 

1 
(EU) 
ELI: http://data.europa.eu/eli/reg/2013/1382/oj. 
2 Regulation (EU) No 2021/693 of 28 April 2021, OJ L 156, 5.5.2021, pp. 21-38, ELI: http://data.europa.eu/eli/reg/2021/693/oj. 
3 COM/2022/121 final. 
4 All EU Member States (except for Denmark and the UK), Albania and Montenegro. 

17 December 

28.12.2013, 

2013, 

354, 

OJ 

of 

L 

p. 73-83, 

1 

 
 
 
2021  until  31 December  2023,  and  all  of  the  programme’s  activities  taking  place  in  all 
participating countries during that time5. 

1.2. Methodology framework 

The  evaluation  –  supported  by  an  external  study  (hereafter  the  ‘supporting  study’)6  – 
combines  and  triangulates  qualitative  and  quantitative  data  sources  and  methods;  it  is 
underpinned  by  desk  research7.  It  was  carried  out  by  the  European  Commission’s 
Directorate-General for Justice and Consumers (DG JUST). 

The  evaluation:  (i)  takes  stock  of  an  earlier  impact  assessment8  (2018)  and  (ii) 
acknowledges both the interim9 evaluation of the 2014-2020 justice programme (2018) and 
the first part ex post evaluation10 of the 2024-2020 justice programme (2022). The interim 
evaluation of the 2021-2027 justice programme draws on the findings of both the first part 
and the second part of the ex post evaluation of the 2014-2020 programme. 

A  wide  range  of  stakeholders  was  consulted  to  prepare  this  evaluation,  including 
representatives of: (i) EU Member States who are also members of the relevant programme 
committees; (ii) programmes’ applicants and beneficiaries; (iii) agencies; (iv) civil society 
organisations;  and,  (v)  the  general  public11.  Dedicated  methods  and  tools  were  used  to 
conduct  the  consultations:  a  questionnaire-based  online  public  consultation,  interviews, 
targeted  surveys.  They  complemented  data  and  information  collected  through  other 
methods, such as desk research and case studies. 

Four main challenges were identified and mitigated: 

•  Limited data accessibility for the  ex post evaluation: the quantitative analysis for the 
years  2014  and  2015  presents  caveats  related  to  challenges  in  retrieving  data  from 
obsolete corporate tools. Considerable time was dedicated to build an updated dataset 
of  grants  and  procurement  contracts  and  to  clarify  approaches  for  data  analysis. 
Additionally, data analysis was complemented with qualitative sources of information 
mainly drawn from the stakeholder consultation activities. 

•  The  statistical  analysis  on  survey  participants’  associations  with  programme  related 
variables needed to be matched to the data sample: different statistical tests were run, 
and their requirements verified. The test that was finally used was different from the 
initially  envisioned  one  and  required  a  further  adaptation  of  the  scale  measuring 
interviewees’ attitudes12. As a result, the conditions for the selected statistical test could 
be met and the test performed. 

•  Inherent limitations in the text-mining exercise: the use of automated prompts within 
natural  language processing  (NLP)  models  limits the scope  for  parameter  definition; 
this  makes  it  difficult  to  obtain  in-depth  results.  To  mitigate  this  challenge,  various 
manual prompts were tested on a smaller data set. This helped in gauging the prompts’ 
accuracy and select suitable ones. 

5 All EU Member States (expect for Denmark) participate to the programme. In November 2024, participating countries also included 
Albania, Bosnia and Herzegovina, Kosovo and Ukraine. 
6 The term ‘evaluation’ refers to the Commission Staff Working Document, while the term ‘supporting study’ refers to the study carried 
out by external experts to support the evaluation. The supporting study was carried out by a consortium led by Tetra Tech International 
Development (hereinafter ‘the Contractor’) and was coordinated by the Commission’s Directorate-General for Justice and Consumers, 
with the support of an Inter Services Steering Group (ISSG) including a range of Commission Directorates-General. 
7 The overall approach is detailed in Annex II. 
8 SWD(2018) 290 final. 
9 COM(2018) 507 final. 
10 COM/2022/121 final. 
11 Details available in the stakeholders’ consultations synopsis report in Annex V. 
12 The Fisher’s exact test was selected. This required to consolidate the Likert-scale from five categories to three categories. Details 
available in Annex II. 

2 

 
 
•  Limitations with the quantitative analysis: to examine the geographical dimension of 
the  2021-2027  justice  programme,  both  the  analysis  of  variance  and  the  association 
analysis relied on categorising countries into regions. However, the categorisation in 
itself  may  affect  the  results.  Desk  research  was  performed  to  determine  the  most 
appropriate categorisation, and it was finally decided to rely on the 2023 EU Rule of 
Law  report’s  one.  Countries  were  clustered  into  four  regions  considering  their 
similarities and differences and based on the indicators employed in the analysis. This 
approach allowed selecting a categorisation that considers the different types of justice 
systems in each Member State and that is meaningful in the context of the evaluation. 

2.  WHAT WAS THE EXPECTED OUTCOME OF THE INTERVENTION? 

This chapter is organised in three parts. Section 2.1 describes the context and rationale for 
the two programmes, Section 2.2 depicts the intervention logics, and Section 2.3 explains 
the approach to establish the points of comparison. 

2.1. Description of the interventions and their objectives 

The  Treaty  on  the  Functioning  of  the  European  Union  provides  for  the  creation  of  a 
European area of freedom, security and justice based on the mutual recognition of judicial 
decisions  and  mutual  trust  among  Member  States.  This  priority  was  reaffirmed  by  the 
European Council in the Stockholm programme13. 

Regulation (EU) No 1382/2013 established the justice programme for the period from 
1 January  2014  to  31 December  202014  (hereafter  also  referred  to  as  ‘the  previous 
programme’) by merging three previous funding programmes15. The programme’s general 
objective was to contribute to the further development of a European area of justice based 
on mutual recognition and mutual trust, in particular by promoting judicial cooperation in 
civil and criminal matters16. 

The programme had four specific objectives17: 

a)  facilitating  and  supporting  judicial  cooperation  in  civil  and  criminal  matters 

(JCOO); 

b)  supporting  and promoting judicial training, including  language training on legal 
terminology, with a view to fostering a common legal and judicial culture (JTRA); 

c)  facilitating effective access to justice for all, including promoting and supporting 

the rights of victims of crime, while respecting the rights of the defence (JACC); 

d)  supporting initiatives in the field of drugs policy as regards judicial cooperation 
and  crime  prevention  aspects  closely  linked  to  the  general  objective  of  the 
programme, insofar as they were not covered by the internal security fund (ISF) 

13 OJ C 115, 4.5.2010, p. 1. 
14  Regulation  (EU)  No 1382/2013  of  the  European  Parliament  and  of  The  Council  of  17 December  2013  establishing  a  Justice 
Programme for the period 2014 to 2020 (OJ L 354, 28.12.2013, pp. 73-83, ELI: http://data.europa.eu/eli/reg/2013/1382/oj). 
15 (i) The criminal justice programme as set out in Council Decision 2007/126/JHA of 12 February 2007 establishing for the period 
2007-2013, as part of the General Programme on Fundamental Rights and Justice, the Specific Programme ‘Criminal Justice’ (OJ L 58, 
24.2.2007, p. 13).;  (ii)  the civil justice programme  as  set out  in  Decision No 1149/2007/EC  of  the  European  Parliament  and of  the 
Council of 25 September 2007 establishing for the period 2007-2013 the Specific Programme  ‘Civil Justice’ as part of the General 
Programme ‘Fundamental Rights and Justice (OJ L 257, 3.10.2007, p. 16); and (iii) the drug prevention and information programme as 
set out in Decision No 1150/2007/EC of the European Parliament and of the Council of  25 September 2007 establishing for the period 
2007-2013 the Specific Programme  ‘Drug prevention and information’ as part of the General Programme  ‘Fundamental Rights and 
Justice’ (OJ L 257, 3.10.2007, p. 23). 
16 Regulation (EU) No 1382/2013, Article 3. 
17 Regulation (EU) No 1382/2013, Article 4. 

3 

 
 
 
for financial support for police cooperation, preventing and combating crime, and 
crisis management or by the health for growth programme (JDRU). 

Nevertheless,  different  national  standards  in  the  justice  systems  of  the  Member  States 
continued  to  negatively  affect  mutual  trust  and  the  consistent  application  of  EU  law; 
vulnerable groups still suffer from an inadequate access to justice for; and unharmonised 
training levels among judicial staff still hinder effective cooperation and cross-border legal 
processes. Therefore, Regulation (EU) No 2021/693 established the justice programme 
for the duration of the multiannual financial framework  2021-202718 (hereafter also 
referred  to  as  ‘the  current  programme’).  Other  factors  called  for  the  2014-2020 
programme’s  continuation:  a  growing  threat  to  fundamental  rights,  EU  values  and 
democratic principles due to rising extremisms and a diminished space for civil society. 

The  general  objective19  of  the  2021-2027  programme  is  to  contribute  to  the  further 
development  of  a  European  area  of  justice  based  on  the  rule  of  law  including  the 
independence and impartiality of the judiciary, on mutual recognition and mutual trust, and 
on  judicial  cooperation  thereby  also  strengthening  democracy,  the  rule  of  law  and  the 
protection of fundamental rights. 

The  programme  has  three  specific  objectives20  that  are  mostly  in  continuity  with  the 
previous programme: 

a)  to facilitate and support judicial cooperation in civil and criminal matters, and to 
promote  the  rule  of  law  and  the  independence  and  impartiality  of  the  judiciary, 
including  through  supporting  efforts  to  improve  the  effectiveness  of  national 
justice systems and the effective enforcement of decisions21; 

b)  to support and promote judicial training, with a view to fostering a common legal 
and judicial culture as well as a culture based on the rule of law, and to support and 
promote  the  consistent  and  effective  implementation  of  the  Union  legal 
instruments that are relevant in the context of the programme; 

c)  to facilitate effective and non-discriminatory access to justice for all, and effective 
remedy, including by electronic means (e-justice), by promoting efficient civil and 
criminal procedures and by promoting and supporting the rights of all victims of 
crime as well as the procedural rights of suspects and accused persons in criminal 
proceedings. 

For the 2021-2027 programming period, supporting initiatives in the field of drugs policy 
were transferred to other EU funding programmes22 while the objective on effective access 
to  justice  was  amended  to  include  effective  and  non-discriminatory  access  to  justice, 
including by electronic means (e-justice), as well as effective redress. Therefore, the 2021-
2027  justice  programme  has  an  increased  focus  on  the  respect  of  the  rule  of  law,  the 
involvement  of  civil  society  organisations  (CSOs)  and  on  ensuring  an  equal  and  non-
discriminatory access to justice for all. 

The following paragraphs zoom in the programme’s main building blocks over the course 
of the 2014-2020 and 2021-2027 funding periods. 

18 Regulation (EU) 2021/693 of the European Parliament and of the Council of 28 April 2021 establishing the Justice Programme and 
repealing Regulation (EU) No 1382/2013 (OJ L 156, 5.5.2021, pp. 21-38, ELI: http://data.europa.eu/eli/reg/2021/693/oj). 
19 Regulation (EU) No 2021/693, Article 3(1). 
20 Regulation (EU) No 2021/693, Article 3(2). 
21 The same three acronyms (i.e. JCOO, JTRA and JACC) are used to refer to the same three specific objectives of both the previous 
and current programme. 
22 Drugs-related initiatives were transferred to the internal security fund (Regulation (EU) 2021/1149) and the EU4Health programme 
(Regulation (EU) 2021/522).s 

4 

 
 
i. Judicial cooperation in civil and criminal matters 

Judicial cooperation in  civil  and criminal  matters is  a specific objective that  the justice 
programme  addresses  in  both  funding  periods  (2014-2020  and  2021-2027).  Under  this 
objective both programmes focus on ensuring the correct application and enforcement of 
the existing EU acquis in this field thus increasing trust between legal professionals from 
different Member States. 

Both  programmes  also  support  key  EU-level  networks  active  in  field  of  judicial 
cooperation and contribute to the EU participation in The Hague Conference of Private 
International Law2324. The European Judicial Network in civil and commercial matters is 
another important pillar funded by the two programmes. The network provides continued 
support to Member States for the setting up and strengthening of national networks active 
in  the  area  of  judicial  cooperation  in  criminal  matters.  Additionally,  both  programmes 
support EU cooperation with the Council of Europe in advancing data collection in prisons 
through the publication of the SPACE report25 and the EU network of prison monitoring 
bodies. 

ii. Judicial training 

In line with the European judicial training strategies2627, in both funding periods, the justice 
programme  supports  cross-border  judicial  training  for  justice  practitioners,  including 
judges, prosecutors, court staff, bailiffs, lawyers, notaries, prison staff, probation officers, 
mediators, and legal interpreters. EU topics covered during training courses are numerous, 
ranging  from  EU  civil  and  criminal  law,  fundamental  rights  and  the  rule  of  law.  To 
complement training on EU legislation, both programmes fund courses on non-legal issues 
such as judicial ethics to support the development of professional skills. In both funding 
periods,  the  justice  programme  supports  also  the  European  Judicial  Training  Network 
(EJTN). In particular, the Regulation establishing the 2021-2027 programme provides that 
the programme shall support EJTN’s expenditure and that any operating grant to that effect 
shall be awarded without a call for proposal28. The 2021-2027 programme further invests 
in supporting the digitalisation of training, both in terms of content and methodology. 

iii. Access to justice for all 

In both funding periods, access to justice is tackled both from the perspective of promoting 
and supporting victims’ rights and the procedural rights of suspects and accused persons. 
At the same time, both programmes support the creation of more effective justice systems 
also thanks to the use of e-justice. The 2021-2027 justice programme puts more focus on 
ensuring a non-discriminatory access to justice, also through a higher use of digital tools 
and new technologies. Finally, both programmes (2014-2020 and 2021-2027) support key 
EU-level networks active in field of access to justice and the annual collection of data and 
information through the EU Justice Scoreboard29. 

23  The  EU  is  party  to  The  Hague  Conference  on  Private  International  Law  since  2007.  The  purpose  of  this  international 
intergovernmental organisation is to work for the progressive unification of the rules of private international law in the participating 
countries. Most of the conventions developed by The Hague Conference fall within exclusive or partial external competence of the EU 
and are part of the EU law. 
24  2006/719/EC:  Council  Decision  of  5 October  2006  on  the  accession  of  the  Community  to  the  Hague  Conference  on  Private 
International Law OJ L 297, 26.10.2006, pp. 1-14. 
25 Statistiques Pénales Annuelles du Conseil de l’Europe (SPACE). 
26 Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and 
the Committee of the Regions: Building Trust in EU-Wide Justice - A New Dimension to European Judicial Training, COM(2011) 551 
final. 
27 2011-2020 European judicial training strategy and European judicial training strategy for 2021-2024. 
28 Regulation (EU) 2021/693, Article 11(3). 
29 EU Justice Scoreboard. 

5 

 
 
iv. Drugs policy and crime prevention 

This specific objective was only covered by the 2014-2020 programme. It focused on drugs 
and  crime  prevention  policy  via  judicial  cooperation  with  key  activities  fostering  the 
practical  application  of  drugs-related  research.  Funding  promoted  stakeholders’ 
knowledge and capacity to address new psychoactive substances. 

The programmes’ synergies 

The 2014-2020 and the 2021-2027 justice programmes have links, potential synergies, and 
are complementary to other relevant initiatives of the European Union (EU), the Council 
of Europe and the United Nations such as: the EU Justice Agenda for 202030; the European 
Agenda  for  Security31;  the  ‘Juncker  priorities’  for  2015-201932;  the  von  der  Leyen’s 
Commission priorities for 2019-202433; the 2030 Sustainable Development Agenda34; the 
European  Convention  on  Human  Rights35.  These  initiatives  have  similar  objectives, 
particularly regarding issues of effectiveness of judicial remedies, judicial cooperation, and 
drugs  prevention.  At  the  same  time,  both  programmes  support  the  implementation  of 
several  sectorial  EU  strategies  thereby  contributing  to  the  achievement  of  key  policy 
objectives and priorities in the justice field. The core EU strategies supported are the EU 
strategy  on  victims’  rights  (2020-2025)36,  the  European  security  union  strategy  (2020-
2025)37,  the  European  e-justice  strategy  (2024-2028)38,  the  European  judicial  training 
strategies (2011-2020 and 2021-2024)3940, and the EU strategy on the rights of the child 
(2021-2024)41. 

The programmes’ geographic scope 

In the period 2014-2020, the justice programme’s geographical coverage encompassed all 
EU Member States except for Denmark and the United Kingdom42. In addition, Albania 
joined the programme in 2017 and Montenegro in 2018. 

Under the  current  programming  period, the justice programme’s geographical  coverage 
encompasses  all  EU  Member  States,  except  for  Denmark.  By  the  end  of  2024,  the 
following  non-EU  countries  participated  in  the  programme:  Albania,  Bosnia  and 
Herzegovina, Kosovoi and Ukraine. 

The programmes’ implementation mode 

Both the 2014-2020 and 2021-2027 justice programmes are implemented through direct 
management by the European Commission under the lead of the Directorate-General for 
Justice and Consumers (DG JUST). A limited number of initiatives is implemented under 
indirect management, in particular by the Council of Europe. Both programmes use similar 

30 COM/2014/0144 final. 
31 COM/2015/0185 final. 
32  European  Commission,  Directorate-General  for  Communication,  Ten  priorities  for  Europe,  Publications  Office,  2015, 
https://data.europa.eu/doi/10.2775/983859. 
33 A Union that strives for more. My agenda for Europe. Political guidelines for the next European Commission 2019-2024. 
34  Transforming  our  World:  The  2030  Agenda  for  Sustainable  Development,  United  Nations,  2015,  available  at: 
https://sdgs.un.org/publications/transforming-our-world-2030-agenda-sustainable-development-17981. 
35 Council of Europe, European Convention on Human Rights, as amended by Protocols Nos 11, 14 and 15, ETS No 005, 4 November 
1950. 
36 COM/2020/258 final. 
37 COM/2020/605 final. 
38 C/2025/437. 
39 COM/2011/0551 final. 
40 COM/2020/713 final. 
41 COM(2021) 142 final. 
42 The justice programme has legal bases that belong to Part V of  Title III of the Treaty on the Functioning of the European Union 
(TFEU). Therefore, protocols 21 and 22 to the TEU and the TFEU apply, and Denmark and the UK never participated in the Justice 
programme. 

6 

 
 
actions  (i.e.  action  grants,  operating  grants,  indirect  management  and  procurement)  to 
achieve their objectives. 

2.2. Intervention logics 

The 2014-2020 justice programme’s general and specific objectives were derived from a 
set of underlying problems, including: 

a)  legislation and instruments’ limited implementation in judicial cooperation; 

b)  persistent differences in law among Member States; 

c)  a lack of trust in the judicial systems of other Member States; 

d)  difficulties in the exercise of EU citizenship rights; 

e)  a lack of awareness on EU law; 

f)  no improvements in the prevention of drug-related crimes; 

g)  limited capacity to tackle cross-border problems. 

The 2014-2020 justice programme aimed to finance a variety of activities to contribute to 
the development of a European area of justice, including analytical work, mutual learning, 
cooperation, awareness-raising, dissemination, and training, as well as actions supporting 
key  actors  such  as  European  networks,  public  and  private  organisations  (typically  non-
profits),  authorities  at  all  levels,  CSOs,  universities,  research  institutions  within  EU 
Member  States  and  participating  countries,  as  well  as  international  organisations.  The 
programme’s primary target group was the judiciary and judicial staff. Other direct target 
groups covered by the activities of the programme are victims of crime, people suspected 
or accused of crimes and, indirectly, all EU citizens. 

i This designation is without prejudice to positions on status and is in line with UNSCR 1244/1999 and the ICJ Opinion on the 
Kosovo declaration of independence. 

7 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Figure 1 - Intervention logic of the 2014-2020 Justice programme 

8 

 
 
The intervention logic of the justice programme 2021-2027 summarises the causal links 
between  needs,  problems,  objectives,  inputs,  activities,  outputs,  results  and  impacts. 
Specifically, the intervention logic presents more in details the types of activities which 
can be funded under the programme through action grants, indirect management actions, 
procurement contracts and operating grants. These types of action are specified in Article 8 
of Regulation (EU) 2021/693, with the view to contributing towards the achievement of 
the specific objectives of the programme. The list of outputs resulting from these activities 
illustrates how they play a crucial role in achieving the objectives of the programme, as 
well  as  helping  to  ensure  the  sustainability  of  the  programme  beyond  its  termination, 
contributing to the realisation of the outcomes and results. 

9 

 
 
Figure 2 - Intervention logic of the 2021-2027 Justice programme 

10 

 
 
2.3. Point(s) of comparison 

The  ex  post  evaluation  of  the  2014-2020  justice  programme  used  the  2011  impact 
assessment43 as point of comparison for measuring achievements. The impact assessment 
outlined a baseline scenario against which the policy options for the justice programme 
were benchmarked as well as a series of expected outcomes. The baseline analysis showed 
that, if no changes had been made between the 2007-2013 and the 2014-2020 programming 
periods,  the  programme  would  have  continued  to  be  successfully  implemented,  but  it 
would not have reached its full potential. The Commission identified the following issues 
as potentially critical: 

• 

• 

the  lack  of  flexibility  in  the  funding  instruments  would  have  been  an  obstacle, 
given the pace of change and reform in the area of justice; 

the  fragmentation  of  funding,  especially  where  funding  is  needed  to  support 
horizontal  and  cross-cutting  issues,  could  have  reduced  the  capacity  of  the 
predecessor programmes to deliver results; 

•  having multiple funding programmes with different rules and procedures, in the 
context of rising  numbers of applicants,  could  have increased the administrative 
burden, potentially resulting in delays in procedures for the beneficiaries. 

As a result, the three predecessor programmes – i.e. the criminal justice programme, the 
civil  justice  programme  and  the  drug  prevention  and  information  programme  –  were 
merged into the 2014-2020 justice programme. 

In  addition,  the  regulation  establishing  the  2014-2020  justice  programme44  set  out 
performance  indicators  which  served  as  a  basis  for  monitoring  and  evaluating  the 
achievement of the programme’s specific objectives. The baseline is the same as for the 
first part of the ex post evaluation, which included contracts and grant agreements signed 
until  31 December  2020.  The  current  report  complements  the  previous  one  through  the 
inclusion of the contracts and grant agreements signed as of 1 January 2021. Annex II.2 
presents  the  points  of  comparison  related  to  the  indicators  set  out  in  Article 15  of  that 
regulation (see Table 4). 

The interim evaluation of the justice programme 2021-2027 used as point of comparison 
the  baseline  scenario  outlined  by  the  2018  impact  assessment  that  accompanied  the 
proposal for a regulation establishing a justice, rights and values fund with two underlying 
funding  programmes:  the  2014-2020  justice  programme  and  the  rights  and  values 
programme (which later became the citizenship, equality, rights and values programme – 
CERV)45. The interim evaluation also referred to the performance monitoring framework, 
established as  per the requirements  of the  regulation establishing  the 2021-2027 justice 
programme46, and its set of indicators. The achievement of the programme’s general and 
specific objectives is assessed against the indicators listed in Annex II of Regulation (EU) 
2021/693 (see Table 5 in Annex II.2). 

43 SEC(2011) 1364 final. 
44 Regulation (EU) No 1382/2013, Article 15. 
45 SWD/2018/290 final. 
46 Regulation (EU) 2021/693, Article 13 and Annex II. 

11 

 
 
 
 
3.  HOW HAS THE SITUATION EVOLVED OVER THE EVALUATION PERIOD? 

3.1. Implementation of the justice programme 2014-2020 

The 2014-2020 justice programme was allocated47 a budget of EUR 377 604 000, with the 
largest proportion of funding (approx. 37%) dedicated to the JTRA objective. As per the 
programme’s  annual  work  programmes  (AWPs),  the  budget  allocations  per  year  are 
illustrated in the table below. 

Table 1 - Budget per year 

Budget year 

Amount planned (EUR)  Amount planned (including 

amendments) (EUR) 

2014 

2015 

2016 

2017 

2018 

2019 

2020 

Total 

45 812 000 

48 051 000 

50 250 000 

52 631 000 

45 949 000 

43 675 000 

45 603 000 

45 812 000 

48 051 000 

50 250 000 

52 786 000 

45 948 700 

43 675 000 

45 603 000 

332 971 000 

332 125 700 

Source: Elaboration from annual work programmes and following non-substantial modifications 

The programme awarded 568 grants, through a total of 118 calls, and 445 procurement 
contracts48, distributed to specific objectives as following: 

•  188 grants and 203 procurement contracts supporting access to justice; 
•  180 grants and 7 procurement contracts providing judicial training; 
•  168 grants and 200 procurement contracts fostering judicial cooperation; 
•  32 grants and 3 procurement contracts supporting initiatives in drugs policy. 

413  procurement  contracts  supported  the  programme’s  specific  objectives,  with  most 
contracts  dedicated  to  the  JACC  and  JCOO  objectives.  The  remaining  procurement 
activities covered general funding needs related to IT maintenance and IT experts. 

The number of grants awarded per year, as shown in the figure below, decreased during 
the period. The highest success rates – defined as the percentage of projects awarded in 
relation to the number of applications submitted – were recorded in 2016 and 2017, with 
both  years  achieving  a  success  rate  of  59%  respectively.  The  lowest  success  rate  was 
recorded in 2020 (21%). 

Regulation 

47 
(EU) 
ELI: http://data.europa.eu/eli/reg/2013/1382/oj. 
48 For procurement contracts, elaboration of data based on the Commission’s Accrual-based Accounting (ABAC) system (DG JUST, 
2024).  

No 1382/2013 

17 December 

28.12.2013, 

p. 73-83, 

2013, 

354, 

OJ 

of 

L 

12 

 
 
 
 
 
 
 
 
Figure 3 - Number of grants signed per year and success rate (2014-2020) 

355

262

112

87

159

94

162

96

187

76

147

55

225

48

400

200

0

100%

50%

0%

2014

2015

2016

2017

2018

2019

2020

Applications

Awarded

Success rate

Source: Elaboration of data from the interim evaluation (2014-2018) and programme data (eGrants, 2019 – 2020) 

The highest number of projects were funded under the JACC objective (188), followed by 
JTRA (180) and JCOO (168), while the lowest number of grants was awarded under JDRU 
(32). 

Figure 4 -Grants by year and specific objective (2014-2020) 

45

40

35

30

25

20

15

10

5

0

40

37

28

32

33

32

29

30

29

28

32

25

25

22

22

25

16

6

7

5

5

4

16

5

12

10

13

2014

2015

2016

2017

2018

2019

2020

JACC

JCOO

JDRU

JTRA

Source: Elaboration of data from the interim evaluation (2014-2018) and programme data (eGrants, 2019 – 2020) 

Projects funded under the JDRU objective were larger on average, both in terms of the 
average  consortium  size  (15  partners)  and  the  average  amount  of  requested  EU 
contributions  (EUR 478 815)  compared  to  other  projects.  JACC  projects  involved  on 
average  7.5  organisations  with  an  average  requested  budget  of  EUR 350 931.  JCOO 
projects involved an average of 9 consortium partners, with an average funding request of 
EUR 335 385, while JTRA consortia on average included 10 organisations with an average 
requested EU contribution of EUR 393 98949. 

3.2. Implementation of the justice programme 2021-2027 

The 2021-2027 justice programme is on track to meet its targets and milestones linked to 
its general and specific objectives, with notable achievements in several key areas such as 
judicial  training,  judicial  cooperation,  and  the  support  of  CSOs.  Some  areas,  such  as 
awareness-raising  and  information  dissemination,  can  be  further  improved.  Chapter  4 
presents the analysis of performance and findings by evaluation criteria, while Table 5 in 
Annex II.2 provides details on indicators. 

Three  of  the  indicators  used  to  assess  the  objective  1:  facilitate  and  support  judicial 
cooperation are not only on track to hit their target, but they have already exceeded it. 
This is the case for the number of exchanges recorded on the European Criminal Records 

49 Elaboration based on programme data and documents extracted from eGrants (DG JUST, 2024). 

13 

 
 
 
 
Information System (ECRIS), the number of CSOs supported and the cumulative number 
of people reached by mutual learning activities under this specific objective50. Only one 
indicator (i.e. people reached by JCOO awareness raising, information and dissemination 
activities) shows that more progress is still needed to meet the 2024 milestone, particularly 
considering the specificities of the programme’s main target group51. 

Two indicators under objective 2: to support and promote judicial training are on track. 
The annual target for the number of justice professionals trained using funding from the 
programme52 was exceeded in 2022, showing a good trend following the record number of 
justice professionals trained in 2020 during the COVID-19 pandemic, which then led to a 
drop in 2021 when the annual target was not reached53. In addition, cumulatively between 
2021 and 2023, the number of CSOs supported54 under this objective already surpassed the 
milestone foreseen in 2024. The two indicators linked to the ‘number of people reached by 
awareness  raising,  information  and  dissemination  activities’  and  ‘number  of  people 
reached by mutual learning and exchange of good practices activities’ are not on track to 
meet their targets55. 

For the objective 3: to facilitate effective and non-discriminatory access to justice for 
all, the following two indicators have significantly exceeded their targets56: ‘number of hits 
on the e-justice portal / pages addressing the need for information on cross-border civil and 
criminal cases’ and ‘people reached by mutual learning and exchange of good practices 
activities’. The latter reached the highest number of people across the three objectives57 
with 2 262 057 people, which significantly exceeds the targets set for both 2024 and 2027. 

Concerning  the  two  indicators  linked  to  the  ‘number  of  people  reached  by  awareness 
raising, information and dissemination activities’ and ‘number of CSOs reached by support 
and capacity building activities’, the data collected for this evaluation provides a partial 
picture58. Due to its temporal scope, the analysis did not include data related to the 2023 
access to justice call￼, which often attracts many CSOs, and it is now every two years. 
The results achieved so  far show a positive trend59 toward the achievement of the 2024 
milestone  in  relation  to  the  number  of  CSOs  reached  by  support  and  capacity  building 
activities. 

In fact, overall, there appears to be a notable improvement in the total number of CSOs 
being  reached.  The  allocation  of  funding  showed  that  non-profit  entities  received  a 
significant portion of the grants financed by the justice programme,  second after public 
bodies. Ensuring an increased support for CSOs active in the justice field is an important 
objective of the current programme and the data show that the programme is going in the 
right direction60. 

50 See Table 5 in Annex II.2 for data sources and values. 
51 Based on data provided in beneficiaries’ application, see Table 5 in Annex II.2. The programme primarily targets the judiciary and 
national authorities, which focus less on awareness-raising and dissemination activities. 
52 Ibid. 
53 Being available at the end of 2024 following the preparation of the annual report on European judicial training, data for 2023 was not 
available for the analysis of this evaluation. 
54 Based on programme data extracted from eGrants (DG JUST, 2024). 
55 See Table 5 in Annex II.2 for data sources and values.  
56 Ibid. 
57 Ibid. 
58 Call for proposals for action grants to support transnational projects in the fields of e-Justice, victims’ rights and procedural rights 
(JUST-2023-JACC-EJUSTICE). 
59 See Table 5 in Annex II.2. 
60 See recital 17 and Article 8 (g) of Regulation (EU) 2021/693 establishing the justice programme. 

14 

 
 
 
 
Figure 5 - Number of beneficiaries participating in grants per organisation type 

Source: Elaboration of programme data (eGrants, 2021-2023) 

Indicators  also  pointed  to  results  in  terms  of  changes  in  the  behaviour,  perception  and 
practice of participants. 

Figure 6 - Change in participants practice, behaviour and perception 

Source: Elaboration of data from the EU Survey on Justice, Rights and Values (2021-2023) 

The data below presents the state of play of the physical implementation of the programme 
over the period 2021-2023. 

167 grant agreements were signed of which 44% supported the JCOO specific objective. 
The overall success rate of the programme was circa 38%. 

Figure 7 - Awarded grants by specific objective 

Source: Elaboration of programme data (eGrants, 2021-2023) 

15 

 
 
 
 
In terms of consortium size, grants awarded under JACC and JCOO have been relatively 
similar involving on average 3.80 and 3.12 organisations respectively. On the other hand, 
the grants awarded under JTRA have been led by larger consortia comprising of an average 
of 6.12 organisations61. 

The average value of the requested EU contribution varied between grants62 awarded under 
the three specific objectives, with: 

- 

JCOO  and  JTRA  grants  amounting  to  an  average  of  EUR 413 258  and 
EUR 462 303 respectively63. 

-  Grants  under  the  JACC  objective  have  been  larger  with  the  average  amount  of 

requested EU contribution amounting to EUR 567 53564. 

In total, in the period 2021-2023, the 2021-2027 justice programme is expected to reach 
around 2 million people through its activities. These figures are estimates provided by the 
applicants at the application stage, which need to be confirmed after the finalisation of their 
projects. 

The type of activities most frequently granted was mutual learning and training, followed 
by awareness-raising. 

Figure 8 - Grants per type of activities implemented 

Source: Elaboration of programme data (Part C, 2021-2023) 

Instead,  data  on  the  distribution  of  procurement  contracts  (260  signed  until  December 
2023)  showed  that  they  were  used  mainly  to  support  the  JACC  and  JCOO  objectives 
through ICT and analytical and monitoring activities. 

61 Elaboration based on programme data and documents extracted from eGrants (DG JUST, 2024). 
62 Considering only action grants. 
63 Elaboration based on programme data extracted from eGrants (DG JUST, 2024). 
64 Ibid. 

16 

 
 
 
 
 
 
 
Figure 9 - Activities undertaken in procurement contracts 

Source: Elaboration of programme data (DG JUST, 2021-2023) 

Overall,  beneficiaries  of  the  justice  programme  came  from  28  countries,  covering  26 
Member  States  and  2  non-EU  countries  (Albania  and  Ukraine).  Like  for  the  previous 
programme,  a  high  number  of  grants  was  received  by  southern  and  eastern  countries65. 
Evidence gathered for this evaluation indicated that the grant distribution has not changed 
between the two funding periods66. 

Figure 10 - Dispersion of grants across countries (coordinators and participants)67 

Source: Elaboration of programme data (eGrants, 2021-2023) 

65 Countries are clustered by regions: Northern Europe (Denmark, Estonia, Finland, Ireland, Latvia, Lithuania, and Sweden), Western 
Europe  (Austria,  Belgium,  France,  Germany,  Luxembourg  and  The  Netherlands),  southern  Europe  (Cyprus,  Greece,  Italy,  Malta, 
Portugal  and  Spain)  and  eastern  Europe  (Bulgaria,  Croatia,  Czechia,  Hungary,  Poland,  Romania,  Slovakia  and  Slovenia).  Details 
available in Annex II.1. 
66 Elaboration based on programme data extracted from eGrants (DG JUST, 2024). 
67 Project partners are referred to as participants. 

17 

 
 
 
 
4.  EVALUATION FINDINGS (ANALYTICAL PART) 

This  chapter  is  structured  in  three  parts:  Section  4.1  assesses  the  two  programmes’ 
performance through their effectiveness, efficiency68 and coherence; Section 4.2 identifies 
their EU added value; and Section 4.3 analyses their relevance. 

4.1. To what extent was the intervention successful and why? 

4.1.1.  Effectiveness 

Justice programme 2014-2020 

The first part of the ex post evaluation concluded that the 2014-2020 justice programme 
was effective as it was overall successful in reaching its objectives. This finding is based 
on metrics assessed through a set of indicators, which are based on the programme’s impact 
assessment69 and further outlined in the programme’s basic act70. 

To  meaningfully  assess  the  previous  programme’s  effectiveness  and  illustrate  the 
programme’s  impact,  the  evidence  is  presented  by  specific  objective.  This  section  also 
touches on relevant horizontal factors and challenges. 

i. Judicial training 

Evidence  from  the  ex  post  evaluation  emphasised  the  programme’s  effectiveness  in 
advancing judicial training. 

The  number  and  percentage  of  EU  judiciary  and  judicial  staff  participating  in  training 
activities funded by the programme was set as an indicator of the specific objective related 
to judicial training. This indicator provided an absolute and relative measure to assess how 
the  programme  reached  the  judiciary  and  judicial  staff,  with  targets  of  16 000  people 
working in the judiciary and judicial staff and 13% of judiciary’s members and judicial 
staff trained71 by 2020. 

The  threshold  of  16 000  trainees  was  already  reached  in  2017.  By  contrast,  the  13% 
threshold was only reached in 202072. This lag of three years indicates the expansion of the 
justice programme’s target group. Even though the judiciary consists of a rather specific 
group of actors, the scope of judicial staff is more dynamic. Another factor that helps to 
explain this lag is the evolving legal context. Beneficiaries reported in interviews that they 
struggled to keep training guides up to date. The justice programme’s continued funding 
in this area was therefore effective as it helped legal practitioners to engage in continued 
training. Follow-up trainings with updated content were highlighted as essential to promote 
judicial training’s sustainability together with peer-to-peer workshops. Sustainability for 
judicial training lies in their trickle-down effects. Several beneficiaries illustrated this with 
reference to social workers active in judicial systems: according to interview feedback, the 
trickle-down  effect  manifested  in  knowledge  transfer  from  social  workers  to  people 
working  in  CSOs.  There  is  further  evidence  from  interviews  that  underpin  the  justice 
programme’s  effectiveness  in  providing  judicial  training.  When  prompted  to  indicate 
which activities contribute the most to projects funded by the programme, three quarters 
of answers from beneficiaries and programme committee members indicated that training 
activities  contribute  the  most.  Interviews  with  beneficiaries,  text  mining  and  surveys 
corroborated this result. Nearly all surveyed beneficiaries (96%) indicated that training led 
to the acquisition of new skills and knowledge. 

68 An overview of costs and benefits as well as potential for simplification is presented in Annex IV. 
69 SEC(2011) 1364 final. 
70 Regulation (EU) No 1382/2013, Article 15. 
71 Ratio between the number of supported participants and the total number of legal practitioners. 
72 Based on first part ex post evaluation and programme data extracted from eGrants (DG JUST, 2024). 

18 

 
 
Overall, the justice programme contributed to the cumulative number of legal practitioners 
receiving training on EU law, including  training funded by other EU programmes.  The 
programme contributed to the target number of 800 000 legal practitioners trained, which 
was  achieved  ahead  of  schedule:  already  in  2016,  such  number  amounted  to  820 000 
people73. 

ii. Judicial cooperation 

The  analysis  conducted  for  the  ex  post  evaluation  yielded  evidence  confirming  the 
programme’s effectiveness in promoting judicial cooperation. The 2014-2020 programme 
met  a  significant  number  of  targets,  even  though  not  all  those  targets  related  to  the 
promotion  of  judicial  cooperation.  Although  achievements  in  this  area  cannot  be  fully 
attributed  to  the  justice  programme,  the  qualitative  data  provided  robust  evidence  to 
confirm its effectiveness. 

The relevant specific indicators74 are: 

•  The surrender procedure’s average time under the European arrest  warrant. 

This indicator measures the time in days to execute the procedure. 
•  The total number of information exchanges via the ECRIS platform. 

The target to reduce the average time taken to surrender a wanted person to a requesting 
country under the European arrest warrant to 10 days by 2020 was not met by a margin of 
11 days. The data showed that the average time increased from 16.7 days in 2019 to 21.26 
days in 202075. As confirmed by the Commission working document on European arrest 
warrant  statistics,  Member  States  do  not  always  comply  with  the  obligation  to  execute 
arrest warrants in the indicated time frame76. Across Member States there is a tendency to 
increase  the  time  for  the  surrender  procedure.  This  tendency  from  Member  State  that 
contributes to delays together with the COVID pandemic’s effects are likely explanations 
for the increased time needed to conclude the surrender procedure. 

By contrast, the programme met its target on the total number of information exchanges 
via the ECRIS platform. The final target of 3 500 000 exchanges in 2020 was exceeded by 
over 500 000 with a total of 4 136 249 exchanges77. 

Qualitative evidence from the ex post evaluation showed that both mutual trust between 
members of the judiciary in different EU countries and consistent implementation of EU 
law  are  key  to  the  effectiveness  of  2014-2020  the  programme.  Investments  in  judicial 
cooperation were considered to promote the effective and comprehensive implementation 
of EU law78. Programme committee members identified judicial cooperation as a driver of 
change. According to them, effective collaboration between partners across EU Member 
States was the main reason for them to adapt their way of working. Projects promoting 
court-based  support  of  judicial  practitioners  and  probation  practices  have  proven  to  be 
particularly suited to facilitate learning. 

Finally,  as  emerged  from  the  consultation  activities,  the  programme  was  effective  in 
disseminating best practices to address crucial issues such as gender-based violence and 
mental health in criminal justice systems. 

73 Ibid. 
74 See Table 4 in Annex II.2. 
75  European  e-Justice  Portal,  EAW.  Available  at:  https://e-justice.europa.eu/topics/court-procedures/criminal-cases/judicial-
cooperation/european-arrest-warrant_en?clang=en. 
76  European  Commission  (2024),  Statistics  on  the  practical  operation  of  the  European  arrest  warrant  –  2022.  Available  at: 
https://commission.europa.eu/publications/replies-questionnaire-quantitative-information-practical-operation-european-arrest-
warrant_en. 
77 See Table 4 in Annex II.2. 
78 Interview feedback from beneficiaries. 

19 

 
 
iii. Access to justice 

For the 2014-2020 justice programme, the number of hits on the e-justice portal quantified 
the programme’s promotion of access to justice79. The indicator set a dynamic target of a 
yearly  20%  increase  starting  from  the  baseline  of  441 000  hits.  In  2020,  the  total  hits 
amounted to 4 619 548, thus the programme exceeded its goal by far80. As confirmed by 
the analysis, the e-justice portal was perceived by programme committee members, EU 
networks and agencies as an important tool to make information available. 

Interview feedback stressed that the 2014-2020 programme effectively raised awareness 
on the important role of professionals who support victims of crime. These professionals 
acted as trusted companions for victims of crime during court proceedings. 

iv. Drugs policy 

The  programme  did  not  meet  its  target  to  assess  a  set  number  of  new  psychoactive 
substances: as of 2015, the number of new substances assessed decreased. Between 2016 
and 2020 the number of substances assessed was around 50. In 2020, the programme set 
out to assess 95 new substances, but only assessed 4681. According to the European Union 
Drugs Agency, the reason for the observed decline in the number of substances assessed 
is  uncertain82.  Potential  explanations  for  the  decline  are  increased  controls  and 
interventions to reduce the sale and consumption of these substances. 

Horizontal factors promoting the programme’s effectiveness 

Interviews  conducted  for  the  ex  post  evaluation  found  that  the  2014-2020  programme 
helped to develop models that are applicable across various jurisdictions. The evaluation 
found83 that the programme’s design allowed for the necessary flexibility to run projects 
that covered multiple countries: projects were effective in disseminating knowledge and 
best practices across participating countries; networks and stakeholder meetings supported 
by the programme were a key driver for knowledge sharing. Although geographic balance 
was not an objective of the programme per se as opposed to funding the maximum number 
of  quality  projects,  evidence  collected  for  this  evaluation  showed  a  skewed  funding 
distribution84  with  organisations  based  in  Italy,  Belgium  and  Spain  featuring  more 
frequently within funded projects. 

Another  key  benefit  of  networks  supported  by  the  programme  was  their  sustainability. 
Programme committee members and beneficiaries alike underpinned that networks created 
through funding remained operating after the projects’ end. According to them, long lasting 
partnerships also increased mutual learning and thereby the operational capacity of funded 
organisations. 

Starting  in  the  2014-2020  funding  period,  the  justice  programme  was  effective  in 
promoting digitalisation. Although this was not a programme objective, when confronted 
with the COVID-19 pandemic, the programme ultimately accelerated the digitalisation of 
the  activities  it  funded.  The  programme  accelerated  digitalisation  by  funding  different 
types of activities, including Information Technology (IT) related activities that fell under 
its remit. The constraints imposed by the pandemic raised the need to find alternative ways 

European 

EMCDDA, 

79 See Table 4 in Annex II.2. 
80 Ibid. 
81 
at:  
https://www.euda.europa.eu/publications/edr/trends-developments/2020_en.  EMCDDA,  European  Drug  Report  2021:  Trends  and 
Developments. Available at:  https://www.euda.europa.eu/publications/edr/trends-developments/2021_en.    
82  EUDA  (2024),  EU  Drug  Market:  New  psychoactive  substances  –  Distribution  and  supply  in  Europe.  Available  at: 
https://www.euda.europa.eu/sites/default/files/pdf/31821_en.pdf?527243. 
83 Based on triangulation of data from consultation activities and desk research. 
84 Based on programme data extracted from eGrants (DG JUST, 2024). 

Developments. 

Available 

Trends 

Report 

2020: 

Drug 

and 

20 

 
 
of conducting trainings and cooperating within project consortia. Along with the shift to 
more digital tools came obstacles to using these tools throughout project implementation. 
Evidence from projects85 shows that the uptake of digital tools has been uneven. Differing 
levels  of  digital  literacy  of  participants  as  well  as  difference  in  digital  infrastructure  in 
Member States posed obstacles to legal practitioners. 

Challenges 

Interviews highlighted that the 2014-2020 programme could have provided more support 
to citizens in accessing services offered by CSOs. Specific examples included legal aid, 
psychosocial counselling, family conflicts, and housing assistance. 

Another challenge was the limited involvement of judges as target groups of projects. This 
could  be  explained  mainly  by  the  limited  time  judges  usually  have  at  their  disposal  to 
engage in activities on top of their duties at courts. 

Justice programme 2021-2027 

The interim evaluation of the 2021-2027 justice programme showed that the programme 
has been effective so far in reaching it objectives, surpassing its goals in judicial training 
and cooperation, despite  the negative external  factors such as  the COVID-19 pandemic 
and the political situation in some Member States. The programme has been particularly 
successful in strengthening connections and training among legal professionals, with over 
81% of projects encompassing these elements among their main activities86. Overall, the 
findings  gathered  for  this  evaluation  pointed  towards  a  positive  trajectory  for  the 
programme, with most stakeholders expressing confidence in its success and its alignment 
with the overarching goals of the EU’s justice agenda. 

Compared to its predecessor, the programme incorporated better programme management 
and administrative procedures as well as better monitoring mechanisms and data collection 
tools (e.g. the EU survey on Justice, Rights and Values)87 to track the effectiveness of the 
funded projects. The performance monitoring framework was developed to take on board 
suggestions highlighted in the 2018 impact assessment and in the previous programme’s . 

The  consultation  activities  carried  out  for  the  evaluation  highlighted  that  beneficiaries 
believed  that  their  projects  contributed  to  programme  objectives  more  than  under  the 
previous  programme.  While  the  same  areas  (i.e.  training,  improving  EU  law 
implementation and raising public awareness) remained significant, beneficiaries reported 
even greater contributions towards improving access to CSO services and enhancing cross-
border system interoperability. This indicates progress in areas that had previously been 
regarded as more challenging. Overall, through its comprehensive approach and targeted 
initiatives, the current programme has fostered a significant shift in the knowledge, skills 
and  behaviours  of  its  target  groups,  suggesting  a  strong  potential  for  long-term  effects. 
Surveys  conducted  with  the  beneficiaries  indicated  that  the  programme  has  raised 
awareness  among  participants  in  particular  on  the  rule  of  law  and  the  EU  Charter  of 
Fundamental  Rights,  with  a  marked  difference  in  knowledge  levels  compared  to  the 
general public (see figures hereafter). 

85 Project CrimiLAW, ID 882042. 
86 Based on programme data and documents extracted from eGrants (DG JUST, 2024). 
87 See Annex II.1. 

21 

 
 
 
 
 
Figure 11 - Awareness of the rule of law 

Sources: Elaboration of data from the EU Survey on Justice, Rights and Values (programme participants) and Special 
Eurobarometer 554, 2024 (public) 

Figure 12 - Focus on the ‘Rule of law’ in the projects (2014-2020 vs 2021-2027) 

Source: Elaboration of results of text-mining analysis based on project documents (eGrants) 

Figure 13 - Awareness of the EU Charter of Fundamental Rights 

Sources: Elaboration of data from the EU Survey on Justice, Rights and Values (programme participants) and Special 
Eurobarometer 554, 2024 (public) 

22 

 
 
 
 
 
This suggests that, despite there is still scope for improvement in reaching out the expected 
number of people, those reached by the programme activities increase their knowledge and 
understanding of both EU law and the rule of law. 

The 2021-2027 programme’s contribution to upholding the rule of law and enhancing the 
protection  of  fundamental  rights  remains  evident.  Rule  of  law  aspects  as  addressed  by 
projects have continued to grow in importance, building on a foundation established in the 
previous  programming  period.  The  2021-2027  justice  programme  has  advanced  and 
broadened  the  rule  of  law  aspects  during  the  current  funding  period  by  enhancing 
adherence  to  legal  standards  and  compliance  with  EU  Directives.  The  programme  has 
evolved  to  incorporate  more  advanced  digital  tools  with  a  slight  increase  in  projects 
focusing  on  capacity  building,  judicial  cooperation  and  independence.  The  findings88 
suggest that the 2021-2027 programme is improving the protection of fundamental rights 
by promoting effective and non-discriminatory access to justice for all with more emphasis 
on  the  use  of  digital  tools  for  broader  accessibility.  The  programme’s  efforts  to  ensure 
inclusivity, combat discrimination, and support vulnerable groups have also been enhanced 
with 32% of projects89 addressing the needs of migrants, refugees, LGBTI individuals, and 
persons with disabilities as well as incorporating anti-discrimination training. Children’s 
rights have also been emphasised, with 13% of projects focusing on protecting children’s 
rights in legal proceedings90, ensuring adherence to EU standards for child-friendly legal 
procedures. 

Stakeholders  and  beneficiaries  expressed  confidence  in  the  sustainability  of  the 
programme  outcomes,  with  a  high  percentage  of  beneficiaries  (94%)  believing  that  the 
impact of their projects will extend beyond their conclusion. The programme’s influence 
is already visible through the connections and networks established, indicating a strong 
likelihood of long-term effects. Professional development and networking as well as the 
development  of  methodological  tools,  platforms  and  training  materials  have  been 
highlighted by stakeholders as areas of long-term impact going beyond the projects’ end 
date91.  Beneficiaries  have  pointed  to  systems  that  were  created  under  the  previous 
programming  period  which  remain  operational  and  continue  to  be  utilised  by  legal 
professionals, indicating the programme’s sustained utility. 

The availability of materials in multiple languages has also contributed to an even greater 
impact, ensuring that the benefits of the programme are accessible to a wider audience and 
have  a  lasting  presence  in  the  justice  sector.  A  significant  majority  of  beneficiaries92 
believed their projects will have the potential to shape future legislation (such as in the 
field of victims’ rights), improve judicial practices and set new EU standards for judicial 
cooperation  and  access  to  justice.  At  the  same  time,  some  beneficiaries  also  expressed 
concerns about their dependency on EU funding for the continuation of their initiatives, 
which could limit long-term impact. The short duration of some projects may also limit 
their contribution to the examination of emerging topics and the ability to respond to recent 
changes, such as the adoption of new legislation (for example, the adoption of the Artificial 
Intelligence Act (AI Act)93. 

The  communication  activities  funded  through  the  programme  have  demonstrated  their 
significant  capacity  for  informing  and  engaging  stakeholders,  as  evidenced  by  the 

88 Based on triangulation of data from consultation activities and desk research. 
89 The analysis was based on a sample of 62 representative projects which were analysed using text-mining techniques. 
90 Ibid. 
91 81% of projects funded in the period 2021-2023 aimed to enhance the skills and connections of legal professionals. 
92 Based on targeted consultation activities, such as beneficiary survey. 
93 Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial 
intelligence. 

23 

 
 
substantial number of individuals reached through various initiatives94. Moreover, the e-
justice portal’s pages on criminal matters have seen a 76% increase in visits between 2021 
and 2022, with 2.25 million hits in 202295, reflecting a growing interest in the justice system 
across EU Member States. 

Concerning the programme’s visibility and the channels through which stakeholders learn 
about it, the EU Funding & Tenders portal remains the primary source of information to 
know  more  about  the  2021-2027  justice  programme  and  its  funding  opportunities.  The 
justice programme does not have national contact points in Member States. Based on the 
evidence  gathered  through  stakeholder  consultations,  it  appeared  that  communication 
activities to share awareness on the programme deriving from national, regional or local 
governments as a source of awareness have decreased from 8.7% in the previous funding 
period to 3.7% in the current funding period. But, in general, 71.6% of both successful and 
unsuccessful applicants perceived the justice programme as well-known among potential 
applicants,  mirroring  results  from  the  previous  funding  period.  Nevertheless,  evidence 
gathered for this evaluation through stakeholder consultations highlighted a disparity in 
awareness  among  different  stakeholder  groups.  Higher  education  and  public  authorities 
reported higher awareness of the programme compared to CSOs. This suggests that while 
the  programme’s  communication  activities  are  effective,  they  may  not  be  reaching  all 
sectors equally. This can be explained by the technical nature of the programme, which 
mainly focuses on the specialised needs of the judiciary and judicial staff. In addition, there 
may be a need to enhance communication efforts in some EU regions to further increase 
the overall effectiveness of the programme and the diversity of partnerships across Member 
States. In fact, although geographical balance is not an objective of the programme per se, 
the data96 showed that, until the end of 2023, organisations based in some Member States 
applied and received funds more frequently, as observed also in the previous programme. 

Following the COVID-19 pandemic, the 2021-2027 programme has put more emphasis on 
digitalisation. Beneficiaries have increasingly made use of digital tools in their activities, 
thereby  increasing  their  effectiveness  (e.g.  increased  accessibility  and  reach  of  their 
projects).  At  the  same  time,  the  programme  has  also  increasingly  focused  on  gender 
mainstreaming. This focus includes the implementation of a tracking methodology, more 
explicit  requirements  in  call  documents  and  evaluation  criteria,  as  well  as  a  dedicated 
analysis  of  the  gender  dimension  of  project  proposals.  The  requirement  for  a  more 
dedicated  focus  on  integrating  gender  mainstreaming  in  the  projects  co-funded  by  the 
justice programme improved the programme’s effectiveness as interventions were better 
aligned  with  target  groups’  needs.  Over  80%  of  the  projects  funded  have  incorporated 
elements of gender equality and  were able to effectively mainstream gender equality in 
their applications97. The evidence gathered for this evaluation mainly through stakeholder 
consultations highlighted, however, that some applicants would still benefit from further 
guidance, such as more good practice examples tailored to the specific objectives of the 
justice programme. 

Overall,  the  2021-2027  justice  programme  has  increased  efforts  to  ensure  inclusivity, 
combat  discrimination  and  support  vulnerable  groups,  indicating  a  forward-looking 
approach  to  creating  fair  and  equitable  justice  system  across  the  EU98.  Ensuring  a  non-

94 The programme’s monitoring data indicates that 201 888 people were successfully reached through awareness-raising activities, while 
379 mutual learning activities have engaged a remarkable 2 288 580 individuals. 
95 See Table 5 in Annex II.2. 
96 See Figure 10. Based on programme data extracted from eGrants (DG JUST, 2024).  
97 Based on programme data analysis and text mining of project documents. 
98 Between 2021-2023, around 32% projects address the needs of migrants, refugees, LGBTI individuals, and persons with disabilities; 
13% of projects focus on protecting children’s rights in legal proceedings. 

24 

 
 
discriminatory  access  to  justice  for  all  is  a  novelty  of  the  current  programme  which  is 
already  showing  good  progress  in  this  regard  as  well  as  in  supporting  gender 
mainstreaming and in mainstreaming non-discrimination in all its activities as required by 
its legal basis. 

Figure 14 - Focus on Inclusivity and Awareness-Raising in Funding Periods (2014-2020 vs 
2021-2027) 

0%

20%

40%

60%

80% 100%

Inclusive practices and equal access

Awareness raising and sensitisation

Support for vulnerable groups

69%
69%

54%
56%

54%
56%

2014-2020

2021-2027

Source: Elaboration of results of text-mining analysis based on project documents (eGrants) 

4.1.2.  Efficiency 

Justice programme 2014-2020 

The  ex  post  evaluation  assessed  the  2014-2020  justice  programme’s  efficiency  on  the 
macro and micro level. Conclusions on the programme’s cost efficiency are based on data 
collected  on  the  programme  and  project  level.  Overall,  the  evaluation  found  that  the 
programme has been cost-effective99 in attaining its objectives. 

Insights on the project level 

Feedback  from  beneficiaries  largely  confirmed  that  the  benefits  of  participating  in  the 
programme outweighed the participation cost. 

Figure 15 - In your opinion, what was the cost-benefit ratio of participation in the justice 
programme for your organisation? 

Source: Beneficiary survey (2014-2020 programme) 

99 Measuring the value provided for the investment made. 

25 

 
 
 
 
 
The  formation  of  new  partnerships  emerged  as  frequently  mentioned  and  leading  to 
benefits such as (i) mutual trust between judicial actors; (ii) enhanced implementation and 
application  of  Union  law  and  (iii)  increased  awareness  and  promotion  of  fundamental 
rights. Although  75% of  respondents  drew a positive balance, there  were  also  common 
concerns. 

The surveys carried out for this evaluation revealed that the application process and linked 
administrative processes posed a challenge for applicants to submitting their project ideas. 
Half of the survey respondents identified the application process as highly demanding on 
their resources. The process was particularly demanding for first-time applicants that had 
to acquaint themselves with the application rules and procedures. By contrast, subsequent 
project  submissions  were  perceived  as  less  challenging.  This  finding  indicates  that  the 
applicant’s experience in drafting applications was a relevant factor and the process itself 
was not the only reason contributing to the perceived burden. Administrative burden was 
as well frequently mentioned concerning reporting requirements. Many of the consulted 
beneficiaries  indicated  that  the  highest  staff  costs  were  attributed  to  administrative 
activities. 

Notwithstanding these challenges, stakeholders also valued the Commission actions aimed 
at simplifying the administrative process. For the 2014-2020 programme, almost half of 
surveyed  stakeholder,  46%,  stated  that  the  most  significant  simplification  was  the 
application process’ digitalisation with the EU Funding & Tenders Portal’s introduction. 

Further interview evidence underlined the investments’ efficiency in  IT systems, which 
provide  long-term  value  to  their  users  and  were  therefore  an  efficient  tool  for  the 
programme to achieve its objectives. This qualitative feedback was further evidenced with 
quantitative metrics on the programme level. 

Insights on the programme level 

The regulation set a budget of EUR 377 604 000 for the period 2014-2020, with yearly 
amounts  set  in  the work programmes.  Based on  data  collected for this evaluation,  over 
90% of the budget planned for grants and procurement (i.e. EUR 332 125 700) had been 
committed (i.e. more than EUR 304 000 000). 

At  the  level  of  specific  objectives,  funding  implemented  under  specific  objectives  on 
access to justice and judicial training was slightly above the average ‘consumption rate’ 
(the ‘consumption rate’ or ‘utilisation rate’ is the percentage of planned funding allocated 
to a programme that has actually been  awarded to activities under that programme). By 
contrast,  funds  implemented  under  judicial  cooperation  and  drugs  policy  had  below 
average  consumption  rates.  The  figure  overleaf  illustrates  the  consumption  rate  of 
available budgets by specific objective. 

26 

 
 
 
 
 
 
 
 
 
 
Figure 16 - Consumption rate of available budgets per specific objective 

Source: Elaboration of programme data (DG JUST) 

Juxtaposing the relative funding consumption rate against the planned budgets yields a 
more nuanced picture. 

Figure 17 - Planned budgets per specific objective 

Source: Elaboration of programme data (DG JUST) 

As  outlined  in  the  preceding  section  on  effectiveness,  the  programme  was  effective  in 
achieving its objectives by engaging almost all its planned funds. On the one hand, this 
underpinned  the  funding’s  efficiency.  On  the  other  hand,  the  unused  budget,  albeit  not 
comparatively large, could have been used to further promote the programme’s objectives. 
Interviews carried out for this evaluation confirmed that especially first-time beneficiaries 
underestimated the resources they needed to implement their project, thus leading to lower 
funding requests than what planned at programme level. This finding is in line with the 
evidence from the first part of the ex post evaluation that identified the underestimation of 
project costs as a key reason for not utilising the programme’s entire budget. 

The  evaluation  scrutinised  2014-2020  the  programme’s  investments  in  two  large  and 
important IT systems and in training activities more closely. The IT systems were selected 
as  they  were  associated  with  the  programme’s  effectiveness  in  promoting  judicial 
cooperation and access to justice. The evaluation also analysed the cost-effectiveness of 
training activities, which provide an insight into the programme’s investments in judicial 
training. The evaluation did not further analyse data related to the specific objective on 
drugs prevention as the programme ceased to cover the specific objective. 

27 

 
 
 
For all three investments analyses, the evaluation created a cost-effectiveness ratio, which 
put  the  programme’s  total  annual  cost  in  relation  to  the  output  under  a  programme 
objective. Although the ratios are only an approximation of relative cost-effectiveness as 
annual  budgets  varied,  they  still  provide  a  longitudinal  insight  on  the  programme’s 
efficiency. 

First,  and  related  to  judicial  training,  the  evaluation  analysed  the  training  sessions’ 
efficiency  funded  by  the  programme.  Thus,  the  total  annual  programme  budget  was 
weighted by the total number of people trained in training courses. Figure 18 (section a) 
illustrates the development of the cost- effectiveness ratio and (section b) the ratio’s annual 
change. 

Figure 18 - Cost- effectiveness ratios of the number of the judiciary members trained (2014-
2020) 

Source: Elaboration of programme data (DG JUST) 

As of the programme’s inception in 2014, the annual ratio exhibited a decreasing trend 
until 2018. In 2019, the ratio peaked to an all-time high. Interestingly, the ratio dropped 
afterwards to its lowest level in 2020. It is striking that the number of people trained was 
at its lowest with 10 799 in 2019 and that it was at its highest in 2020. The rapid change in 
the number of people trained and the change in associated costs can be largely explained 
by the impact of the COVID-19 pandemic. Triggered by the onset of the pandemic in 2020, 
digitalisation of training courses increased. Previously reported100 data confirmed that the 
stark increase in the number of people trained between 2019 and 2020 could be attributed 
to online attendance. By funding remotely organised trainings, the programme improved 
its cost-effectiveness. The figures should, however, be contextualised as the work of the 
European Judicial Training Network (EJTN), focusing on in-person exchanges, illustrates. 
Networking and in-person exchanges remain important features contributing to the quality 
of judicial training courses. 

Another  factor  adding  to  the  2014-2020  programme’s  cost-  effectiveness  were  EJTN’s 
activities at European level that created economies of scale. The nexus provided by the 
EJTN  established  an  enabling  environment  for  large-scale  judicial  training  sessions. 
Benefiting from its wide outreach, the EJTN promoted training opportunities impactfully 
among its members. As a result, the programme could offer training sessions at a lower 
cost. 

100 Programme performance statement, 2022, available at Working documents 2023 - European Commission. 

28 

 
 
 
The  cost-effectiveness  of  digital  infrastructure  is  further  demonstrated  by  the  two  IT 
systems that were analysed. 

The European Criminal Records Information System (ECRIS) is an IT system operated by 
Member States’ central authorities to exchange information on criminals101. The number 
of  exchanges  in  the  ECRIS  was  one  of  the  indicators  to  measure  how  the  programme 
promoted judicial cooperation; it is therefore a good estimator to gauge cost-effectiveness. 
Analogue to judicial training, a ratio put the programme’s total annual cost in relation to 
the total number of exchanges via ECRIS in a year. The cost of the exchanges decreased 
continuously until 2019. The number of exchanges via ECRIS increased over the years and 
thus  contributed  to  the  good  cost-output  ratio.  The  pandemic  also  had  an  effect  on  the 
number  of  exchanges  via  ECRIS,  albeit  to  a  very  small  degree.  Figure  19  (section  a) 
illustrates the development of the cost-effectiveness ratio and (section b) the ratio’s annual 
change. 

Figure 19 - Cost-effectiveness ratios of ECRIS exchanges (2014-2020) 

Source: Elaboration of ECRIS data (DG JUST) 

According  to  a  Eurojust  report102,  the  slight  increase  of  costs  in  2020  can  be  mainly 
attributed  to  reduced  activity  of  national  competent  authorities.  Among  other  factors, 
teleworking and a lack of skilled staff were the underlying factors. However, after a short 
transition period, activities returned to normal. As a result, the reduced activity increased 
only slightly the cost of operating ECRIS. From a longitudinal perspective, the marginal 
cost (i.e. the cost for one additional exchange) dropped significantly from EUR 36.65 in 
2014 to EUR 11.03 in 2020. The strong decrease in marginal cost illustrates the usefulness 
of ECRIS and by extension its increasing cost-effectiveness. 

The second IT system for which a cost-effectiveness ratio was established is the e-justice 
portal. The portal provides online access to information to citizens when trying to obtain 
information on legal matters, procedures or legal practitioners to aid them. The hits on the 
e-justice portal were used as a proxy in the 2014-2020 programme’s impact assessment to 
measure how the programme provided access to justice. Building on this metric, the cost-
effectiveness ratio for access to justice put the total annual programme cost in relation to 

101 Council Decision 2009/316/JHA. 
102 Council of the European Union, (2020). The impact of COVID-19 on judicial cooperation in criminal matters - Executive summary 
of information compiled by Eurojust and EJN. Available at: https://db.eurocrim.org/db/en/doc/3528.pdf. 
. 

29 

 
 
 
the  total  number  of  hits  on  the  e-justice  portal.  Figure  20  (section  a)  illustrates  the 
development of the cost-effectiveness ratio and (section b) the ratio’s annual change. 

Figure 20 - Cost-effectiveness ratios for hits on the e-justice portal (2014-2020) 

Source: Elaboration of e-justice portal’s data (DG JUST) 

In a similar way as the investments in the ECRIS, the cost for hits on the e-justice portal 
decreased  over  the  period  in  which  the  programme  was  implemented.  The  2024-2020 
programme  invested  in  expanding  the  e-justice  portal  and  maintaining  it  and  these 
investments  are  likely  to  have  increased  general  awareness  of  the  portal.  This  view  is 
further supported by the increased number of hits on the portal. Thus, the increased use of 
the e-justice portal showed the cost-effectiveness of building a larger digital structure. 

In  conclusion,  the  three  ratios  illustrate  the  programme’s  long-term  cost-effectiveness 
through increases in the use of established IT systems and training approaches. 

Justice programme 2021-2027 

Evidence  gathered  for  this  evaluation  showed  that,  so  far,  the  2021-2027  justice 
programme has been overall cost-effective. Stakeholders’ feedback also corroborates this 
view,  as  most  respondents  confirmed  that  the  benefits  outweigh  the  costs.  As  for  the 
previous programme, collaboration and long-term partnerships led to the main benefits for 
beneficiaries  and  their  target  groups.  Such  benefits  materialised  in  increased  awareness 
and knowledge of the issues addressed by the programme as well as increased skills and 
enhanced cooperation. 

The programme management and administrative procedures improved over the three initial 
years  compared  to  the  previous  programme.  Challenges  remain  for  beneficiaries  and 
applicants regarding the completion of administrative tasks associated in particular with 
the application process and reporting, which remain costly and time-consuming. 

Nevertheless,  many  stakeholders  also  acknowledged  the  improvements  introduced  with 
the 2021-2027 programme103. 

Overall,  according  to  the  stakeholders  consulted  for  this  evaluation,  the  simplification 
measures introduced in the 2021-2027 programming period have been positive and largely 

103 Based on interviews with beneficiary carried out as part of this evaluation, over 90% respondents assessed the application process 
as very or somewhat challenging: however, the share of respondents who assessed it as very challenging decreased from 50% to 40% 
between the  two programmes.  About  80% assessed the  reporting  as very  or  somewhat  challenging; also, in  this case  the share of 
respondents who assessed it as very challenging decreased from 25% to 14% between the two programmes. The financial reporting 
process was highlighted for its complexity. Nonetheless, almost half of the beneficiaries (42%) believed that the administrative process 
improved between the two programming periods. 

30 

 
 
 
reduced  the  administrative  burden.  Changes  introduced  were  seen  by  beneficiaries  as 
having a positive impact even if it is too early to draw definitive conclusions. This includes 
the  use  of  lump  sum  contributions,  the  simplification  of  the  reporting  process  and  the 
introduction of the e-Grants system. Stakeholders reported that the changes have put more 
emphasis on content and less on administrative procedures, which has given beneficiaries 
more flexibility with regards to the implementation of activities. The introduction of lump 
sums should also help simplify and increase the quality of final reporting, which is still 
challenging for some beneficiaries. 

On the other hand, the introduction at Commission corporate level of the standardised unit 
costs for travel and accommodation was assessed less positively since unit costs often fail 
to reflect actual market rates. Additionally, when it comes to financial reporting, it remains 
difficult to maintain two separate bookkeeping records (i.e. one based on actual costs and 
another one  on standardised  unit costs).  This  adds a layer of complexity,  especially  for 
public authorities. 

The feedback received through targeted consultation activities also indicates that there is 
efficiency and clarity in how information is communicated: beneficiaries overall agreed 
that they had heard about funding opportunities in good time (84%) and that the conditions 
for receiving support were clear and transparent (88%). 

Furthermore, thanks to a more structured approach for assessing the performance of funded 
activities, the 2021-2027 programme is better equipped than the 2014/to ensure that funds 
are used efficiently. This is a significant improvement compared to the previous funding 
period.  The  monitoring  framework  of  the  2021-2027  programme  allows  for  regular 
adjustments, ensuring that the programme remains responsive to beneficiaries’ needs and 
emerging challenges. 

Collaboration  with  existing  networks,  such  as  the  EJTN,  has  continued  to  show  the 
potential for improving the cost-effectiveness of the programme. As highlighted by the ex 
post evaluation of the 2014-2020 programme, these partnerships enable beneficiaries to 
organise  large-scale  events  with  extensive  outreach  at  reduced  costs,  thus  achieving 
economies of scale. 

In  terms  of  external  factors  influencing  the  efficiency  of  the  programme,  stakeholders 
consulted for this evaluation particularly highlighted economic and political factors (e.g. 
inflationary  pressures  caused  by  Russia’s  war  of  aggression  against  Ukraine).  Instead, 
challenges linked to COVID-19 pandemic were no longer considered an issue beyond early 
2022 since the programme has been quick and flexible in adopting mitigating measures. In 
the longer term, COVID-19 has even increased the efficiency of projects funded under the 
justice  programmes  by  accelerating  the  uptake  of  digitalisation.  Approximately  29%  of 
projects funded over the period under analysis redirected funds to support the transition to 
digital platforms and to cover increased IT support costs. 

Also, for the interim evaluation of the 2021-2027 programme, the evaluation scrutinised 
the programme’s investments in training activities for members of the judiciary and in two 
important IT systems (ECRIS and the e-justice portal). 

In  terms  of  the  number  of  judicial  staff  trained  under  the  current  justice  programme, 
findings  indicate  that,  overall,  cost-effectiveness  improved  between  2021  and  2022.  In 
2021, a sharp increase in the cost-effectiveness ratio was observed. The reasons for this 
could be varied, including potential changes in programme execution, increased costs, or 
external factors. Then, the subsequent decrease in 2022 brought the cost-effectiveness ratio 
back down to a level very similar to that of 2020, essentially recovering from the spike in 
2021. 

31 

 
Figure 21 (section a) illustrates the development of the cost-effectiveness ratio and (section 
b) the ratio’s annual change. 

Figure 21 - cost-effectiveness ratio of the number of members of the judiciary trained (2021-
2027) 

Source: Elaboration of programme data (DG JUST). Note: 2023 data not yet available at the time of carrying out the 
analysis. 

In  terms  of  exchanges  of  information  in  ECRIS,  data  showed  increases  in  cost-
effectiveness  over  the  period  2021-2023.  The  most  significant  improvement  occurred 
between 2022 and 2023 (see Figure 22). 

The combined data from 2014 to  2023 showed a predominant  trend of increasing  cost-
effectiveness given the steady decrease in the cost per exchange of information on ECRIS. 
The two years of increase (2020 and 2021) represent brief interruptions in an otherwise 
consistent improvement over the ten-year period. 

Figure 22 - cost-effectiveness ratio of ECRIS exchanges (2021-2027) 

Source: Elaboration of ECRIS data (DG JUST) 

Finally, looking at the number of hits on the e-justice portal, despite a brief increase in the 
cost-effectiveness ratio in 2021, the 2021-2027 programme demonstrated a strong ability 
to improve its cost-effectiveness for this indicator over subsequent years, culminating in a 
significant reduction in costs per ‘hit’ by 2023. This suggests that the programme became 
increasingly efficient in delivering its intended results over time, with the cost per ‘hit’ on 
the e-justice portal decreasing substantially. 

32 

 
 
 
Figure 23 - cost-effectiveness ratio of hits on the e-justice portal (2021-2027) 

Source: Elaboration of e-justice portal’s data (DG JUST) 

4.1.3.  Coherence 

Justice programme 2014-2020 

The ex post evaluation showed that the 2014-2020 programme reached overall its goals in 
a coherent way. The programme succeeded particularly in promoting fundamental rights 
and  ensuring  effective  remedies,  promoting  judicial  training  and  cooperation  without 
duplicating other EU funding programmes. Survey results indicated that the programme 
has been unique in offering funding that addressed stakeholders’ specific needs. Only 7% 
of  surveyed  beneficiaries  confirmed  that  other  EU  funding  programmes  could  have 
provided  them  with  funding.  By  contrast,  70%  of  surveyed  beneficiaries  said  that  they 
would  not  have  been  able  to  secure  other  EU  funding  in  the  justice  programme’s 
hypothetical absence. Figure 24 illustrates the stakeholders’ views. 

Figure 24 - If your project had not received funding from the justice programme, would your 
organisation have been able to secure funding from other EU financial instruments for the 
project’s implementation? 

Source: Beneficiary survey (2014-2020) 

33 

 
 
 
 
At the programming level, the evaluation of the EU drugs strategy104 confirmed the 2014-
2020 justice programme’s complementarity on EU drugs policy. Interventions funded by 
the justice programme specifically focused on judicial cooperation and crime prevention. 
Thereby, the justice programme specifically focused on areas that were not covered by the 
ISF.  The  complementarity  between  the  2014-2020  justice  programme  and  the  ISF  was 
further confirmed by the fund’s interim evaluation105. 

Feedback  from  programme  committee  members  supported  the  positive  assessment  of 
synergies  between  the  justice  programme  and  the  ISF.  They  confirm  that  moving  the 
programme’s  objective  on  drugs  policy  to  the  ISF  was  the  right  choice.  Although 
interventions funded through the 2014-2020 justice programme did not overlap with ISF 
initiatives, programme committee members indicated that they fitted much better within 
the ISF’s remit. 

Further  evidence  from  interviews  with  programme  committee  members  identified  two 
salient areas in which the 2014-2020 justice programme successfully complemented other 
national or EU initiatives. 

The  first  area  is  digitalisation.  According  to  programme  committee  members,  projects 
focusing on digitalisation of justice effectively added to national digitalisation initiatives. 
Moreover,  the  evaluation  identified  synergies  in  this  area  with  the  Connecting  Europe 
Facility (CEF)37: funding for digital infrastructure projects under CEF was aligned with 
projects  advancing  judicial  training  online  funded  under  the  justice  programme.  The 
second area is fundamental rights. Synergies were most salient with the  rights, equality 
and citizenship programme. Both programmes promoted protection of fundamental rights 
and access to justice from their respective angles and synergies were obtained by close 
coordination  between  Commission  services  responsible  for  the  two  programmes  which 
helped in aligning the approaches to evaluate proposals. For instance, a thematic synergy 
existed  in  the  area  of  victim  rights,  where  the  2014-2020  justice  programme  was 
complemented  by  the  REC  programme  with  its  support  for  victims  of  gender-based 
violence.  Beneficiaries  reported  that  training  activities  funded  by  the  REC  programme 
complemented judicial training courses funded by the justice programme. 

Justice programme 2021-2027 

The actions funded under the 2021-2027 programme and their objectives were shown to 
be coherent with the EU policies in the justice field. The EU priorities in the field of civil 
and criminal justice are similar to those in the previous programming period, but there is 
now  a  stronger  focus  on  e-justice,  digitalisation  of  judicial  systems  and  cross-border 
information  exchange tools.  The  majority of  programme  committee  members consulted 
for the evaluation, who generally belong to Ministries of Justice of the Member States, 
believed that so far, the 2021-2027 programme aligns well also with national policies. 

Thanks also to its focuses on three specific objectives, the justice programme is coherent 
and  has  created  synergies  with  other  funding  programmes,  such  as  the  (CERV) 
programme,  in  particular  in  relation  to  support  provided  to  victims  of  crime  and  other 
vulnerable groups. Furthermore, since the current programme plays an important role in 
supporting Member States’ efforts for their interconnection to existing EU instruments and 
IT systems, there is complementarity  and synergies with programmes dealing with EU-

104 Final assessment of the EU drugs strategy 2013, available at: https://op.europa.eu/en/publication-detail/-/publication/daf5ddf5-cd52-
11ea-adf7-01aa75ed71a1. 
105 European Commission SWD (2018) 341 final interim evaluation of the internal security fund - police. 

34 

 
 
 
level  IT  systems  and  the  digitalisation  of  national  judicial  systems  (such  as  the  Digital 
Europe  Programme106,  the  Technical  Support  Instrument107,  and  the  Recovery  and 
Resilience Facility)108. 

Only a small percentage of beneficiaries consulted for this evaluation believed they could 
secure  alternative  EU  funding  if  the  justice  programme  were  unavailable  (public 
authorities  and  higher  education  institutions  being  the  most  optimistic  about  this 
possibility,  while  CSOs  are  very  unlikely  to  find  alternative  sources  of  funding  if  the 
programme were to be discontinued). 

The programme is also coherent with EU international commitments and objectives, such 
as the Sustainable Development Goals109, specifically with SDG 16 on ‘promoting peaceful 
and inclusive societies for sustainable development, provide access to justice for all and 
build effective, accountable and inclusive institutions at all levels’. 

4.2. How did the EU intervention make a difference and to whom? 

Justice programme 2014-2020 

For  the  2014-2020  justice  programme,  beneficiaries’  interview  feedback  indicated  that 
they would not have been able to implement  their projects in the absence of the justice 
programme.  This  finding is  echoed by the online survey’s result.  Only 9% of surveyed 
beneficiaries were confident that if the justice programme had not existed, Member States 
would have been an alternative viable funding source for them110. Public authorities such 
as courts, ministries and higher education institutions were among the stakeholders that 
reported most frequently that they would have been able to secure national funding. By 
contrast, CSOs were less confident to find alternative funding sources. Close to half of the 
surveyed  CSOs  indicated  that  national  funding would  not  have  been  available  to  them. 
Figure 25 depicts the responses’ summary. 

Figure 25 - If there were no justice programme, would you be able to obtain financing for 
your project from national instruments/programmes? 

Source: Beneficiary survey (2014-2020) 

106 The Digital Europe Programme (Regulation (EU) 2021/694). 
107 Technical Support Instrument (TSI) (Regulation (EU) 2021/240). 
108 The Recovery and Resilience Facility (Regulation (EU) 2021/241). 
109 https://sdgs.un.org/goals. 
110 This feedback is based on a sample population of 56 beneficiaries. 

35 

 
 
 
Underlying reasons for the reliance on EU funding via the 2014-2020 justice programme 
included the specificity of national funding instruments. Interviews clarified that reference 
to national funding related to specific budgets earmarked under targeted national funding 
programmes. 

By  contrast,  only  some  beneficiaries  believed  that  their  projects  would  have  been 
implemented also without the 2014-2020 justice programme’s funding, albeit with a lower 
scope. The interviewed beneficiaries pointed specifically to the following limitations: 

•  The  projects’  geographical  scope  would  have  been  smaller  based  on  national 

funding. 

•  The project’s thematic scope would have been limited. According to beneficiaries, 
national funding is tailored to specific groups such as victims of a specific kind of 
crime. The justice programme was perceived more flexible. For example, it was 
broader than national funding by providing financial support to projects addressing 
victims of all types of crime. 

•  The organisation’s capacity to implement projects of a larger scope would not have 

been sufficient. 

Feedback  from  programme  committee  members  contextualised  these  limitations  by 
identifying two areas in which the 2024-2020 justice programme helped level the playing 
field among Member States. 

The first is the added value from training sessions co-funded by the 2014-2020 programme, 
which were perceived to be very valuable and impactful for programme stakeholders. 

The second is the 2014-2020 programme’s contribution to digitalising the justice systems 
and  making  them  interoperable.  Programme  committee  members  underlined  that  the 
justice programme was an important tool for them to advance digitalisation. Especially for 
one programme committee member the need to purchase court IT equipment was striking 
and  trumped  the  need  to  promote  interoperability.  Digitalisation  proved  particularly 
important to overcome issues raised by the COVID-19 pandemic. 

In  conclusion,  the  2014-2020  justice  programme  was  essential  in  providing  financial 
support to its stakeholders, who would not have been able to achieve the same outcomes 
had the programme not existed. 

Justice programme 2021-2027 

The  2021-2027  justice  programme  continues  to  provide  a  unique  added  value  which  is 
difficult  to  replicate  at  national  level.  The  programme  adds  value  over  and  above  that 
created  by  Member  States  acting  alone.  Beneficiaries  consulted  for  this  evaluation  still 
believe  that,  while  alternative  funding  sources  exist  (e.g.  private  entities,  regional  and 
national  research  projects  or  other  EU-level  opportunities),  the  support  offered  by  the 
justice programme is unparalleled in its scope and impact. So far, the programme enabled 
organisations to undertake extensive activities that would otherwise have taken years to 
accomplish or would have been significantly scaled down. Moreover, evidence gathered 
for  this  evaluation  through  both  stakeholder  consultation  activities  and  desk  research 
showed that, in most cases, no national funding was available to provide the same kind of 
support to EU justice policies such as cross-border digitalisation or inherently transnational 
activities. Beneficiaries acknowledged the programme’s role in accelerating processes and 
facilitating cross-border projects, which would be limited in scope and scale without its 
support. The justice programme therefore fills in a crucial funding gap, especially in areas 
requiring international cooperation and victims’ support. 

36 

 
The 2021-2027 justice programme is also particularly valued for fostering the creation and 
awareness  of  EU-level  networks  and  in  enabling  smaller  Member  States  to  undertake 
projects that, otherwise, would be beyond their capacity. 

Cross-border collaboration increased from 69%  of all funded projects  in the 2014-2020 
period  to  78%  in  the  2021-2023  period111,  showing  a  continued  focus  on  enhancing 
cooperation among  different  legal  professionals  across EU  Member  States.  Institutional 
participation also grew in 2014-2020, with 69% of projects involving universities, national 
training academies and judicial authorities against 78% in 2021-2023112. 

This suggests that stopping the programme would likely result in significantly reducing 
the scope of justice-related activities, particularly those involving cross-border initiatives. 

The  arguments  with  regards  the  subsidiarity  principle  put  forward  as  part  of  the  2018 
impact assessment are still valid. The objectives set out in the impact assessment are still 
better addressed at  EU level  due to  the persistent  nature of the issues relating to  cross-
border cooperation in the area of justice, the paradigm shift needed for a true change in the 
legal cultures of Member States to cooperate with each other’s judicial systems, and the 
need for coherent application of EU law by national courts. 

The  added  value  of  the  2021-2027  justice  programme  also  lies  in  its  more  targeted 
approach,  enhanced  focus  on  digitalisation,  and  improved  monitoring  systems.  These 
improvements render the current programme even more effective in meeting the evolving 
justice needs for justice of the EU and its citizens. 

4.3. Is the intervention still relevant? 

Justice programme 2014-2020 

The evaluation confirmed the relevance of the 2014-2020 justice programme as already 
identified in the programme’s interim and first part ex post evaluation113114. 

The  beneficiaries  survey  results  showed  that  86%  of  respondents  agreed  that  the 
programme met (very or rather) well their needs. By contrast, only 7% (four respondents) 
indicated that the 2014-2020 justice programme did not meet their needs well. 

Figure 26 - How well did the objectives of the justice programme correspond to the needs of 
organisations like yours? (beneficiaries) 

Source: Beneficiary survey (2014-2020) 

A similar picture emerged from the results of the unsuccessful applicants’ survey. 

111 Based on results of text mining analysis. 
112 Ibid. 
113 EUR-Lex - 52018SC0356 - EN - EUR-Lex. 
114 eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52022DC0121. 

37 

 
 
 
 
 
 
Figure 27 - How well did the objectives of the justice programme correspond to the needs of 
organisations like yours? (unsuccessful applicants) 

Source: Unsuccessful applicants survey (2014-2020) 

On average, unsuccessful applicants were less positive than beneficiaries in considering 
the programme adapted to their needs. However, the majority of unsuccessful applicants 
(69%) still believed that that their needs were met rather well or well. 

The correlation analysis scrutinised the perceptions of success and unsuccessful applicants 
based on their geographic location and organisational type. It did not find any statistically 
significant  link  between  the  opinions  of  success  and  unsuccessful  applicants  on  the 
programme’s relevance115. Thus, successful and unsuccessful  applicants  alike supported 
the programme’s relevance in surveys. 

The data showed116 that the 2014-2020 justice programme funded several projects that had 
‘sequels’.  These  follow-up  projects  pointed  to  the  programme’s  relevance  to  its 
stakeholders  over  time.  A  good  third,  34%  of  all  beneficiaries,  participated  either  in  a 
follow-up  project  or  in  a  different  project.  Hence,  between  2014-2020,  the  justice 
programme managed to steadily keep a significant part of its stakeholders engaged. 

A feature that helped the 2014-2020 justice programme remain relevant over the course of 
its implementation was its ability to adjust its priorities and address emerging needs. The 
justice programme merged the formerly separate civil justice programme, criminal justice 
programme and the drug prevention and information programme. As a result, its objectives 
made  it  flexible  and  able  to  cover  emerging  needs,  especially  in  the  area  of  judicial 
cooperation. The programme’s effective re-programming made it possible to launch new 
call for proposals to address rising radicalisation of prison inmates and cooperate with the 
Council of Europe to gather prison statistics. 

The  programme  further  proved  its  flexibility  throughout  the  COVID-19  pandemic. 
Beneficiaries consulted for this evaluation stated that the programme provided them the 
needed  administrative  flexibility  to  implement  their  projects  despite  the  limitations 
imposed by the pandemic. Reporting extensions, reallocations of funds and shifting from 
in-person to online and hybrid project activities helped to implement projects throughout 
the pandemic, and thereby to maintain the programme’s relevance. 

Beneficiaries were also  asked to indicate how the 2014-2020 justice programme helped 
address needs at national level. 

115 See Fisher test results in Annex II for details. 
116 Based on results of text mining analysis. 

38 

 
 
 
 
 
 
 
Figure 28 - How well do/did the objectives of the justice programme correspond to the needs 
of the justice sector in your country? 

Source: Beneficiary survey (2014-2020) 

Based on feedback illustrated by the figure, 84% of surveyed beneficiaries believed that 
the 2014-2020 justice programme met their needs, showing that the programme addressed 
problems that were relevant in Member States and participating countries. 

Notwithstanding  the  programme’s  stakeholders’  positive  feedback  and  high  level  of 
engagement, stakeholders would have appreciated additional support from the programme 
to disseminate projects’ results. 

The first part ex post evaluation part shed light on a potential improvement of the 2014-
2020 programme’s relevance. It identified  the need to broaden the justice programme’s 
target group. In particular, it singled out CSOs working on victims’ rights as a group that 
should  have  been  actively  supported.  By  contrast,  the  2021-2027  justice  programme’s 
interim  evaluation  did  not  find  strong  evidence  for  this  need.  Only  gathered  scattered 
feedback  from  individuals  in  interviews  suggested  a  need  to  better  include  correctional 
services and prison staff in the target groups. These mixed findings indicate that the 2021-
2027 justice programme should closely monitor its relevance to CSOs working on victims’ 
rights to consolidate the findings from the ex post evaluation. 

Justice programme 2021-2027 

Some of the needs and challenges identified in the 2021-2027 programme’s intervention 
logic still exist and need to be addressed. This is despite some of the recent progress in this 
area. For instance, insights from the EU Justice Scoreboard117 and data collected via recent 
Eurobarometer  surveys118  highlighted  progress  in  Member  States’  efforts  to  deliver 
effective national justice systems, but these insights also suggest that there is a continued 
need  for  improvements119.  Several  challenges  persist  such  as  variations  in  judicial 
efficiency, uneven implementation of judicial reforms and barriers in citizens' access to 
justice. These current and evolving needs prove the continued relevance across the EU of 
the programme’ objectives of promoting judicial cooperation, judicial training and access 
to justice. 

As it was the case under the 2014-2020 programme, the evidence collected for this analysis 
confirmed  that  the  2021-2027  justice  programme  is  meeting  the  needs  of  its  key 
stakeholders,  i.e.  beneficiaries  and  potential  beneficiaries.  The  data  clearly  demonstrate 

117  See  EU  Justice  Scoreboard  2024/  ‘The  EU  Justice  Scoreboard  2024’,  Communication  from  the  Commission  to  the  European 
Parliament, the Council, the European Central Bank, the European Economic and Social Committee and the Committee of the Regions 
COM(2024) 950. 
118 Flash Eurobarometer 540 – Perceived independence of the national justice systems in the EU among the general public, February 
2024. For this Flash Eurobarometer, a representative sample of EU citizens, aged 15 and over, in each of the 27 EU Member States, 
Albania,  Montenegro,  North  Macedonia  and  Serbia  was  interviewed.  Between  14  and  27 February  2024,  29 484  interviews  were 
conducted over the telephone. 
119  This  includes  the  need  for  reduction  of  the burden of  court  fees and  the availability  of  legal aid,  promotion  of  voluntary  use  of 
alternative dispute resolution methods (ADR), support the participation of persons with disabilities as professionals in the justice system 
and facilitating access to justice by electronic means, as the uptake of digitalisation in national justice systems across EU Member States 
remains uneven. In addition, cross border crimes continue to rise since 2020, suggesting cross border judicial cooperation will only 
become more important. 

39 

 
 
 
that the programme, not only aligns with their expectations, but also plays a crucial role in 
addressing the needs of the justice sector at national level (see figures hereafter). 

Figure 29 - How well do/did the objectives of the justice programme correspond to the needs 
of organisations like yours? 

Source: Beneficiary survey (2021-2027) 

Figure 30 - How well do/did the objectives of the justice programme correspond to the needs 
of the justice sector in your country? (n=50) 

Source: Beneficiary survey (2021-2027) 

At the same time, 96% of beneficiaries believed that the outputs of their project correspond 
to the needs of the relevant target groups within the justice sector and society more broadly. 

Figure  31  -  Based  on  your  experience,  to  what  extent  do  the  outputs  of  your  project 
correspond to the needs of the relevant target groups within the justice sector/society? 

Source: Beneficiary survey (2021-2027) 

All this underlines the continued relevance of the 2021-2027 justice programme’s specific 
objectives.  The  architecture  of  the  current  programme  brought  significant  added  value 
compared  with  the  2014-2020  programme  in  this  regard,  particularly  through  its  more 
streamlined  and  focused  design.  In  fact,  the  removal  of  drugs  policy  led  to  a  sharper 
programme focus through its three specific objectives, which are key for the justice area, 
hence  allowing  for  greater  impact.  Moreover,  the  broad  scope  of  the  three  specific 
objectives  allows  for  flexibility  and  ensures  that  the  programme  can  respond  to  new 
priorities, such as the increased importance of digital transformation in the justice sector. 

In  fact,  another  key  element  of  the  2021-2027  programme’s  ability  to  maintain  its 
relevance  to  its  beneficiaries  and  target  groups  in  recent  years  has  been  its  capacity  to 
harness digital technologies. This is particularly important given the uneven progress and 
sometimes fragmented approach seen in Member States’ efforts to digitise their national 
justice systems. Although the digitalisation of national justice systems in the EU is not one 
of the specific objectives of the justice programmes, digitalisation of justice is a horizontal 
component that informs all three specific objectives in a cross-cutting manner and that is 
enabling more accessible, efficient and interconnected judicial processes across the EU. 

The digitalisation of justice systems has the potential to create justice ecosystems that are 
more  resilient,  adaptable  and  aligned  with  the  EU  broader  digital  transformation  goals, 

40 

 
 
 
 
ultimately strengthening public trust in the judicial system and supporting a more cohesive 
European area of justice. Stakeholders and beneficiaries noted a number of areas in which 
funding for digitalisation of justice would be relevant, both in terms of thematic areas (AI, 
copyright infringement, cyberbullying, and transparency in the digital space) and types of 
activities.  This  includes  developing  digital  tools  (e.g.  for  the  purpose  of  improving 
communication,  access  to  information,  comparable  data  collection),  supporting  the 
practical implementation of digital tools, as well as improving interoperability, and raising 
awareness on opportunities and benefits of digitalisation in the justice sector. 

There  remains  scope  to  ensure  a  more  strategic  and  focused  approach  to  address  the 
challenges linked to an uneven level of digitalisation of justice in the Member States and 
all opportunities presented by the rapidly evolving digital landscape. 

41 

 
5.  WHAT ARE THE CONCLUSIONS AND LESSONS LEARNT? 

This chapter is organised in three parts. Following a brief overview focusing on synergies 
between  the  two  programmes,  conclusions  are  presented  by  programme  and  structured 
according to the evaluation criteria of effectiveness, efficiency, coherence, EU added value 
and relevance;  they refer to  findings  presented under the previous chapter. The  chapter 
closes with operational findings and lessons learnt for the 2021-2027 justice programme. 

5.1. Synergies between the 2014-2020 and the 2021-2027 justice programmes 

Overall, the continuity of the 2014-2020 and the 2021-2027 justice programmes’ general 
and specific objectives fosters tangible synergies. Both programmes have successfully 
contributed  (so  far)  to  supporting  mutual  recognition  of  judicial  decisions  across 
Member  States  and  improving  cross-border  cooperation,  which  help  creating  long-
term  impacts  in  further  developing a cohesive European area of justice built  on mutual 
trust and cooperation between Member States. The programmes’ joint focus on judicial 
training  creates  synergies.,  The 2014-2020 programme  fostered  a shared legal  culture 
across  Member  States  through  training  thousands  of  judges,  lawyers,  and  court  staff  in 
both national and EU law. The 2021-2027 programme builds on these effects to further 
develop and unify the legal landscape across the EU. 

Ensuring  access  to  justice,  particularly  for  vulnerable  groups,  has  been  a 
fundamental  objective  of  both  programmes.  In  the  2014-2020  funding  period, 
substantial efforts were made to reduce barriers to access to justice, resulting in improved 
frameworks and increased protections for marginalised individuals. These advancements 
inform the 2021-2027 programme’s approach, ensuring that established lessons continue 
to drive forward a more equitable and inclusive EU legal system. 

The digitalisation of justice systems is also an area in which the two programmes are 
mutually complementary. The 2014-2020 period saw the further improvement of digital 
tools like the European e-justice portal, which continues in the current programme. This 
facilitates easier access to legal information and cross-border cooperation. The 2021-2027 
programme  builds  on  this  foundation,  continuing  to  expand  digital  initiatives  and 
modernising judicial cooperation mechanisms. This continuity supports the EU’s broader 
digital transformation goals, making the justice system more efficient and accessible. 

Both  programmes  have  contributed  to  the  correct,  coherent  and  consistent 
application  of  EU  law.  By  facilitating  exchanges  of  best  practices,  the  2014-2020 
programme  fostered  collaboration  among  Member  States,  resulting  in  a  more  uniform 
application  of  EU  laws  by  judicial  practitioners.  This  ongoing  synergy  enhances  the 
integrity  of  EU  legislation  and  promotes  legal  coherence  across  borders,  an  approach 
furthered in the current programme. 

Finally,  the  support  for  European  networks  and  civil  society  in  the  field  of  justice 
continues across both programmes. The provision of funding for networks such as the 
European  Judicial  Training  Network  allows  for  sustained  collaboration  and  capacity 
building across the EU, ensuring that projects have long-term impact and that cooperation 
between  Member  States  is  strengthened.  This  ongoing  support  helps  ensure  that  legal 
professionals  and  organisations  can  engage  in  sustained,  high-quality  collaboration  on 
justice matters across the EU. 

42 

 
 
5.2. Conclusions and lessons learnt 

Justice programme 2014-2020 

Effectiveness 

The ex post evaluation confirmed that 2014-2020  programme’s implementation  yielded 
the  expected  results.  The  programme  was  especially  effective  in  promoting  mutual 
trust and raising awareness of EU law via its training activities. Stakeholders’ feedback 
stressed the network’s sustainability as they continue to support the programme’s goals 
after  the  EU  funding  ended.  Thus,  the  programme  effectively  established  long-term 
partnerships that kept the programme’s momentum. 

Although digitalisation’s advancement was not one of the programme’s explicit objectives, 
digitalisation  played  a  key  role  for  the  programme’s  effectiveness.  Faced  with  the 
COVID-19 pandemic’s impact,  the programme ensured it supported its stakeholders by 
implementing  activities  digitally.  Moving  project  implementation  to  the  digital  space 
revealed potential for closer cooperation between Member States and improved  thereby 
information exchange and interoperability. 

Efficiency 

The  ex  post  evaluation  confirmed  the  conclusion  that  the  programme’s  benefits 
outweighed its costs. Direct funding beneficiaries and target groups who benefited by their 
activities  obtained  immediate  and  tangible  benefits  from  the  projects.  The  EJTN 
functioned as a multiplier in the area of judicial training. By leveraging its outreach, it 
enabled  beneficiaries  to  organise  large-scale  trainings  at  a  low  cost.  The  EJTN  created 
economies of scale by facilitating and ensuring quality of training sessions at European 
level. 

Streamlining administrative processes with the introduction of the EU Funding & Tenders 
portal and e-Grants system for the 2014-2020 justice programme made the application and 
grant  management  process  easier  for  applicants.  Although  the  portal  was  well  received 
among  applicants,  there  is  still  scope  for  improving  its  user  friendliness.  Beneficiaries’ 
interview feedback pointed clearly to the need to make grant applications lighter and ease 
administrative burden. 

The pandemic promoted the uptake of digital technologies for project implementation. The 
increased use of digital tools helped the programme reach a wider audience through 
their funded activities. It thus led to economies of scale and reached out better to its target 
groups at a comparatively lower cost. 

Coherence 

The programme’s objectives were coherent with EU policies and priorities over the 
2014-2020 implementation period. The programme was aligned with EU strategic agendas 
such  as  the  EU  Agenda  for  Justice  2020  and  national  policies.  Throughout  its 
implementation,  the  programme  created  synergies  with  other  EU  funding 
programmes. The most salient synergies were confirmed with the REC programme. Close 
cooperation  between  Commission  services  responsible  for  the  REC  and  the  justice 
programmes improved project evaluation and the training courses’ impact. 

In hindsight, moving the objective related to drugs policy from the justice programme 
to the ISF was the right decision. Although with its focus on judicial cooperation it did 
not overlap with other specific objectives, the objective related to drugs policy synergised 
much better with the international security fund’s objectives. 

43 

 
EU added value 

The justice programme provided funding for activities that yielded an impact which 
Member States would not have achieved on their own. Interview feedback concluded 
that  individual  projects  could  have  been  funded  by  Member  States  on  the  same  topics, 
however, with a lesser scope and impact. The justice programme’s EU added value was 
indeed articulated by its scope and size. Without the programme’s funding, stakeholders 
would have required much more time and efforts to build and develop their capacities. The 
analysis  identified  three  areas  in  which  the  programme  helped  promote  Union-wide 
capacity building. The programme succeeded in contributing to a more level playing 
field in the areas of judicial training, interoperability of IT systems and digitalisation. 
Judicial staff and national judiciaries largely benefited from training offers and investments 
in digital capacities. 

Relevance 

The  programme  was  highly  relevant  to  its  beneficiaries  and  target  groups’  needs. 
Interview  feedback  from  applicants  and  beneficiaries  alike  confirmed  the  programme 
responded  well  to  its  stakeholders’  needs.  This  conclusion  was  further  stressed  by  the 
projects with sequels. The programme supported its stakeholders over a longer period of 
time to work towards obtaining its objectives. 

Merging the formerly separate civil justice programme, criminal justice programme 
and  drugs  prevention  and  information  programme  into  the  justice  programme 
yielded  a  long-term  impact.  It  provided  the  justice  programme  with  a  broad,  albeit 
flexible  remit.  Thereby,  the  programme  remained  relevant  to  its  stakeholders  over  the 
course of its implementation. 

This flexibility was a strong asset when the programme had to deal with the COVID-19 
pandemic’s  impact.  The  programme’s  design  and  scope  enabled  it  to  adjust  its 
programming and address emerging needs. 

Justice programme 2021-2027 

Effectiveness 

The  2021-2027  justice  programme  is  making  notable  progress  towards  its  main 
objective of creating a European area of justice rooted in the Union’s core values, such as 
the rule of law, and mutual trust. The programme’s contribution to upholding the rule 
of  law  and  enhancing  the  protection  of  fundamental  rights  remains  particularly 
evident.  Significant  progress  has  been  seen  also  in  promoting  judicial  training  and 
cooperation. Instead, areas like awareness-raising and information dissemination may need 
more focus to achieve the set targets. 

The performance monitoring framework has significantly addressed and improved the 
gaps  identified  during  the  previous  programme  and  provides  now  a  solid  basis  for 
monitoring the programme performance. Nonetheless, certain improvements could still 
be  considered.  For  instance,  in  the  process  for  collecting  stakeholders’  perceptions  and 
experiences;  moreover,  the  added  value  and  visibility  of  the  project’s  outputs  could  be 
enhanced by linking the existing networks (such as the EJTN, the e-justice portal) with 
successful projects on the EU Funding & Tenders Portal. 

Most  beneficiaries  of  the  2021-2027  justice  programme  believe  that  their  projects 
contributed to programme’s objectives more than under the 2014-2020 programme 
and that the results of their projects will be lasting. At the same time, some beneficiaries 

44 

 
have  also  expressed  concerns  about  their  dependency  on  EU  funding  for  the 
continuation of their initiatives. 

The programme’s communication activities have demonstrated a significant capacity 
to  inform  and  engage  stakeholders  and  the  programme  itself  appears  as  well-known 
among potential applicants. At the same time, the programme awareness is somewhat 
limited  particularly  among  stakeholders  from  the  non-profit  sector  and  it  varies 
between  Member  States,  even  though  this  is  explained  by  the  programme’s  technical 
nature and its focus on the needs of the judiciary and judicial staff. Programme committee 
members  play  a  crucial  role  in  supporting  national  stakeholders  in  identifying  which 
funding instruments are most relevant to them and, in general, in raising awareness about 
the different funding opportunities. In Member States where they are responsible for more 
than one funding instrument, an increased cooperation could be explored to engage more 
national stakeholders. 

Overall,  the  current  programme  has  put  an  increased  focus  on  digitalisation, 
inclusivity, combating discrimination, supporting vulnerable groups, and on gender 
mainstreaming. The programme’s increased attention on gender mainstreaming made its 
funding more effective by better aligning the funding’s design with beneficiaries’ needs. 
This last point could be further improved by providing additional guidance tailored to the 
specific objectives of the programme. 

Efficiency 

According to the majority of stakeholders, the benefits of the programme outweigh its 
costs. 

Taking on board the conclusions of the first part of the ex post evaluation, the 2021-2027 
justice  programme  implemented  several  simplification  measures  to  streamline  its 
processes  and  administrative  procedures  thereby  enhancing  its  efficiency.  The  new 
measures show promising results even though it is too early to assess their full impact. 

Therefore,  it  is  important  that  the  Commission  maintains  the  current  support  and 
guidance and enhances the visibility of existing materials before developing new ones. 
Providing additional targeted support may be also considered. Beneficiaries appreciated, 
in particular, that the new measures allowed for more emphasis on the content by ensuring 
also more flexibility. However, some of them continue to experience challenges with the 
application procedure and the reporting requirements. Even if more time is needed to assess 
the  effects  of  the  simplification  measures  introduced  (in  particular,  the  lump  sums), 
additional guidance targeting the identified challenges could be provided. 

Thanks to its performance framework, the current programme is better equipped than 
the  2014-2020  programme  to  ensure  that  funds  are  utilised  efficiently.  This  is  a 
significant improvement compared with the previous funding period. 

As in the previous programme, collaboration with existing networks, such as the EJTN, 
has  continued  to  show  the  potential  for  improving  the  cost-effectiveness  of  the 
programme.  These  networks  enable  beneficiaries  to  organise  large-scale  events  with 
extensive outreach at reduce costs, thereby achieving economies of scale. Consequently, 
these types of collaborations should be maintained and opportunities to collaborate with 
other networks could be further explored. 

Also, the analysis of the programme’s investments linked to the performance indicators for 
its three specific objectives showed that the trend of improved cost-effectiveness is still 
valid. 

45 

 
As during the previous programme, COVID-19 has created some disruptions but mostly 
during the first year of the programme. Issues related to the pandemic were mitigated 
by  2022,  thereby  demonstrating  the  potential  of  digitalisation  to  further  improve  the 
efficiency of the projects funded. 

Coherence 

The actions funded under the current programme are coherent with the EU policies 
in the justice field. Compared with the previous funding period, the current programme 
sees an increased focus on digitalisation. The actions funded are also coherent with policies 
at national level and with other EU international commitments and objectives, such as the 
Sustainable Development Goals. 

The  justice  programme  ensures  a  high  level  of  coherence  amongst  the  different 
available EU funding instruments, in particular with the CERV programme. Enhanced 
coherence is also due to the programme’s focus on just three specific objectives. 

The  current  programme  is  playing  an  important  role  in  helping  Member  States  in 
developing national tools to create the interfaces required between EU-wide systems 
and national ones. In this regard, most extended synergies identified were between the 
justice programme on the one hand and the Digital Europe programme and Recovery and 
Resilience Fund investments in the field of digitalisation on the other. Funding for digital 
projects could benefit from a clear mapping of needs at EU, national  and local level in 
terms of IT infrastructure and training to ensure that the full potential of existing and future 
investments is reached. 

Only  a  small  percentage  of  beneficiaries  said  that  they  believed  they  could  secure 
alternative EU funding if the justice programme  were unavailable.  CSOs, in  particular, 
said they were very unlikely to find alternative sources of funding if the programme 
were to be discontinued. 

EU added value 

By  addressing  EU-level  issues  and  needs,  the  justice  programme,  provides  unique 
added value that would be difficult to replicate at national level. The programme plays 
an important  role in  accelerating processes and facilitating cross-border projects, which 
would be limited in scope and scale without its support or might even not exist according 
to some beneficiaries. The justice programme is also particularly valued for fostering the 
creation and awareness of EU-level networks and enabling smaller Member States to 
undertake projects they otherwise could not. 

Stopping  the  programme  would  likely  result  in  a  significant  reduction  in  the  scope  of 
justice-related activities in the EU, particularly those involving cross-border initiatives as 
the programme is filling a crucial funding gap. The added value of the 2021-2027 justice 
programme also lies in its more targeted approach,  greater focus on digitalisation, 
and  use  of  improved  monitoring  system.  These  improvements  allow  the  current 
programme to be effective in meeting the evolving justice needs of the EU and its citizens. 

Relevance 

The current justice programme, also thanks to its more streamlined and focused design, 
is maintaining a high degree of relevance, a conclusion in continuity with the findings 
from the 2018 interim and the first part ex post evaluations of the previous programme. 
The  programme’s  specific  objectives  are  still  highly  relevant  to  the  needs  of  its 
beneficiaries and target groups in the wider justice sector. 

46 

 
The programme could play a key role in bridging the gaps still existing in the next funding 
period by further enhancing its EU added value. 

The  programme’s  broad  objectives  have  been  crucial  in  also  ensuring  that  the 
programme remains sufficiently flexible to adapt to new merging needs. 

In  particular,  the  justice  programme  was  able  to  adapt  to  the  pace  of  the  progress 
generated  by  digital  technologies.  The  COVID-19  pandemic  accelerated  the  shift  to 
digital  technologies  and  underscored  the  programme’s  ability  to  maintain  its  relevance 
through its flexibility in an evolving judicial and societal landscape. 

However, while recognising the progress made so far, there is a continued need for 
the programme to fund projects that strengthen Member States’ capacities in new 
digital technologies and encourage innovation. The possibility to include ‘digitalisation 
of justice’ as an additional specific objective of the programme in the next  Multiannual 
Financial Framework (MFF), rather than treating it as a horizontal one, could be explored 
to ensure a more strategic and consistent approach to all the related issues and challenges. 

Operational findings and lessons learnt 

Based on the conclusions of the interim evaluation, some areas for improvement or follow-
up  could  be  addressed  through  the  current  implementation  of  the  2021-2027  justice 
programme. 

During the first three years of implementation, the 2021-2027 programme showed notable 
progress  towards  its  main  objective  of  creating  an  EU  justice  area.  However,  to  better 
reflect the programme’s impact and enhance its added value and visibility, some actions 
could be considered. For instance, a more systemised process for collecting stakeholders’ 
perceptions and experiences and ways to link the existing networks (such as the EJTN, the 
e-Justice portal) with successful projects on the EU Funding & Tenders portal. 

While  the  programme  has  already  reached  a  large  number  of  people,  the  programme’s 
awareness remains limited among certain types of stakeholders, and it varies also between 
Member  States.  To  address  this,  an  increased  cooperation  with  programme  committee 
members could be explored to engage more national stakeholders. 

At the same time, and also to keep increasing the cost-effectiveness of the programme, the 
collaboration with existing EU-level networks could be further explored. 

Progress  made  on  gender  mainstreaming  could  be  consolidated  through  additional 
guidance for applicants and beneficiaries as well as good practice examples tailored to the 
specific objectives  of the justice programme.  Additional guidance to  support applicants 
and beneficiaries, who still experience some challenges with the application procedure and 
the reporting requirements, could also be provided. 

Finally,  there  remains  scope  to  further  capitalise  on  opportunities  brought  on  by 
digitalisation in the justice field. The possibility to include ‘digitalisation of justice’ as an 
additional  specific  objective  of  the  programme  in  the  next  funding  period  could  be 
explored to ensure a more strategic and consistent approach to all the related issues and 
challenges. 

47 

 
 
 
 
ANNEX I. PROCEDURAL INFORMATION 

Lead DG, DECIDE Planning/CWP 

This evaluation was included in the Commission’s Agenda Planning System (PLAN/2023/1855) and carried out in compliance with the Article 14(4) of 
the justice programme Regulation 1382/2013120 and Article 14(2) of the justice programme Regulation 2021/693121. In line with the Better Regulation 
Guidelines122, it assesses the extent to which the two programmes were effective, efficient and coherent, provided EU added value and remained relevant 
to tackle present needs. 

The evaluation was carried out by the Directorate-General for Justice and Consumers (DG JUST). 

DG JUST  carried out  the ex  post evaluation of the 2014-2020 justice programme based on a two-step approach to  both  fulfil the requirements of the 
regulation and carry a meaningful, cost-effective evaluation exercise. The first part of the ex post evaluation was completed in March 2022123; it provided 
an overview of the funding distribution and assessed the preliminary achievements of the programme, which fed into the design of the activities of the 
current one. The second part instead focused on assessing long-term impact and sustainability. The structure and timeframe of the evaluation were agreed 
with  the  Secretariat-General  and  Legal  Service.  Agreement  was  re-confirmed  by  the  dedicated  Inter-Service  Support  Group  supporting  the  evaluation 
during its meeting124 of 23 October 2024. 

Organisation and timing 

DG JUST prepared the evaluation roadmap, the stakeholder consultation strategy and the technical specifications for the supporting study contract. The 
documents were submitted for feedback and approval to the dedicated Inter-Service Steering Group (ISSG). 

The ISSG was established in June 2023 following the invitation sent on 25 May 2023 to the following DGs: BUDG, HOME, COMM, COMP, SANTE, 
RTD,  DIGIT,  EAC,  EMPL,  ESTAT,  REGIO,  REFORM,  OLAF,  DGT,  CNECT,  INTPA,  NEAR,  the  Secretariat-General,  the  Legal  Service  and  the 
European External Action Service. The group met five times (see ‘Evidence, sources and quality’). 

120 https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32013R1382 
121 https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32021R0693 
122 https://commission.europa.eu/law/law-making-process/planning-and-proposing-law/better-regulation/better-regulation-guidelines-and-toolbox_en 
123 COM/2022/121 final 
124 Ares(2024)7725582 

48 

 
 
 
DG JUST conducted the supporting study with external experts between 4 December 2023 – 17 November 2024. The ISSG was consulted on the supporting 
study report (inception, interim, final and synopsis reports) during the dedicated meetings and through a dedicated online collaborative space. The relevant 
evaluation documents (evaluation roadmap, call for evidence, open public consultation) were published on the Europa ‘Have your Say’ dedicated page for 
stakeholders’ feedback and consultation respectively between 22 August 2023 and 19 September 2023 for the call for evidence, and between 3 April 2024 
and 26 June 2024 for the public consultation. 

DG JUST concluded the evaluation in 2025 with issuing a Commission Report and a stakeholders consultations synopsis report. 

Exceptions to the Better Regulation Guidelines 

The Commission’s Better Regulation Guidelines were followed to carry out the evaluation without deviations. 

Evidence, source and quality 

Table 2 summarises the successive steps of the evaluation. An external supporting study was carried out by a consortium led by Tetra Tech International 
Development with a contract duration of 50 weeks. The study applied a mix of evaluation methods including desk research, online public  consultation, 
surveys, and interviews with stakeholders, EU officials and Member States’ representatives. The ISSG concluded that the study was conducted in line with 
the technical specifications and the agreed inception report. The study used relevant qualitative and quantitative sources and methods, although data had 
some specific caveats (discussed in Chapter 1 and Annex II). The analysis and conclusions are sound, and the methodological framework and its limitations 
are clearly outlined. 

Table 2: Evaluation timeline 

Steps/tasks 
Preparation (March 2023 – November 2023) 
Draft the evaluation roadmap and the consultation strategy 
Set up the inter-service steering group (ISSG) and draft the supporting study 
technical specifications (ToR) 
ISSG meeting to discuss the ToR and the consultation strategy 
Tendering procedure 
Roadmap publication  
Call for Evidence 

Evaluation supporting study (December 2023 – November 2024) 
Signature of the contract 

Timing 

March – May 2023 
May – June 2023 

21 June 2023 
July – November 2023 
August 2023 
22 August 2023 – 
19 September 2023 

4 December 2023 

49 

 
Kick-off meeting 
Inception meeting (with ISSG) 
Online public consultation 
Interim meeting (with ISSG) 
Validation workshop  
Final report meeting (with ISSG) 
Finalisation of the supporting study 
Report and Staff Working Document (November 2024 – May 2025) 
Draft Report, SWD and Synopsis report 
ISSG meeting on Report and SWD 
Prepare and launch the Inter-Services Consultation package 
Revision and presentation to College of final Report  
Transmission of the Commission Report to the European Parliament, the Council, 
the European Economic and Social Committee and the Committee of the Regions 
(Interinstitutional Database, Europa, etc.) 

12 January 2024 
15 February 2024 
3 April 2024 – 26 June 2024 
10 July 2024 
24 September 2024 
23 October 2024 
October – November 2024 

November – December 2024 
5 February 2025 
February – March 2025 
April 2025 
May 2025 

Source: DG JUST 

Consultation of the Regulatory Scrutiny Board (RSB) 

The evaluation was not scrutinised by the RSB. 

50 

 
ANNEX II. METHODOLOGY AND ANALYTICAL MODELS USED 

This annex provides an overview – complementary to Section 1 – of the methodology adopted to carry out this evaluation. It details data collection methods 
and tools used, as well as limitations to the reliability of the analysis carried out. 

II. 1 Methods and tools 

This Staff Working Document builds on data collected and findings obtained mainly through the supporting study carried out by the contractors. The study 
provided an evidence-based assessment of the relevance, effectiveness, efficiency, coherence and EU added value of the two programmes. An intervention 
logic was designed or reviewed for evaluating each programme. The analytical approach was rooted in theory-based evaluation, complemented by a mixed-
methods strategy that incorporated both qualitative and quantitative data. The approach was tailored to maximise the benefits of conducting two concurrent 
evaluations  of  separate  programmes  at  different  stages.  Three  case  studies  were  also  conducted.  Figure  32  presents  and  overview  of  the  study’s 
methodology. 

Figure 32. Overview of study methodology 

51 

 
 
 
The following sections provide an overview of the methods used in the supporting study. 

Study desk-based research (secondary data collection) 

The  preliminary  data  mapping  helped  identifying  a  wide  range  of  available,  relevant  documents  at  policy,  programme  and  project  level  for  the  two 
programmes. The policy review was based on policy-specific reports as well as political and policy documents to support the assessment of coherence and 
complementarity of the programmes with EU policies or other funding programmes with similar objectives. The review also helped to show interlinks 
between the programmes and their wider policy context. At programme level, programming documents for the two programmes were reviewed to support 
the analysis of questions under effectiveness, efficiency, and relevance, as well as to contribute to collecting and aggregating data on specific indicators 
listed for each programme. Furthermore, the study elaborated data from  the special programme’s Eurobarometers125. For project-level documents, an 
automated text mining approach was used to extract relevant data from proposals and periodic reports. The desk review also included project-level data 
collection, including: grant data, procurement data, Part C data126, results of the EU Survey on Justice, Rights and Values127. 

Text mining 

The text mining aimed to assess the effectiveness of the programmes by analysing a set of qualitative indicators, as follows: (a) change in participants’ 
perception, behaviour and practice; rule of law aspects reflected in project applications; (b) enhanced focus on anti-discrimination in the access to justice 
reflected in project applications; (c) extent to which gender equality is mainstreamed in projects; (d) impacts of the programme on gender equality; (e) 
participants’ overall assessment of project activities; (f) target groups of the participants; (g) examples of programme design being adapted to the page of 
progress generated by digital technologies; (h) impact of COVID-19 on project implementation; (i) dissemination of project’s best practices to other member 
States; (j) unexpected or unintended effects (positive or negative) of the programme. 

The analysis enabled collecting additional qualitative input needed for answering the relevant evaluation questions. The text mining was performed on 
project documents (n= 1 180) for a sample of 62 projects. The project sample was selected with the aim of ensuring representativeness of the selected 
projects, based on the following sampling criteria: funding period (i.e. 42% of projects funded between 2014-2020 and 58% of projects funded since 2021), 
type of beneficiary, type of support received, and other criteria such as the Member State of the beneficiary, specific objective of the projects, type of 
activities funded, and size of funding. To review the documents, the supporting study team used the EYQ chat tool128, which employs advanced natural 
language processing algorithms to identify and group relevant patterns within large volumes of text. Prompts were set-up and refined to extract relevant 

125 Eurobarometer 514 and Eurobarometer 552. The series of surveys aims to give a snapshot of EU citizens’ perception of the values promoted by the EU funding programmes related to rights and values (the Citizens, Equality, 
Rights and Values programme and the justice programme) and their awareness of the different instruments used to promote and protect rights and values. The Eurobarometer survey is one of the tools set up by the performance 
monitoring framework of the 2021-2027 programme. 
126 Between 2021 and the beginning of 2024, the Commission collected data on the (estimated) number of people reached by activities organised by the funded projects through the ‘Part C form’ annexed to the application form 
submitted by applicants. 
127 Mandatory event survey for beneficiaries to distribute among event participants, officially kicked-off in September 2022. The survey is one of the tools set up by the performance monitoring framework of the 2021-2027 programme. 
128 EYQ Chat is an advanced textual analysis and text-mining tool designed to efficiently process and extract relevant information from large volumes of documents. It is an in-house tool of one of the consortium partners that carried 
out the supporting study. It provides adherence to data protection and GDPR compliance. 

52 

 
 
data for the selected indicators. However, when creating prompts the following limitations were encountered: prompts used to analyse some indicators (i.e. 
those related to gender equality, or those related to the impact of COVID-19 and digital technologies) yielded similar results. Although the related analysis 
focused on different aspects, some results were repeated across different indicators, particularly in areas such as gender equality and anti-discrimination. 
To address this limitation, the analysis differentiated the results, with the anti-discrimination indicator focusing on other forms of discrimination. 

Study fieldwork activities (primary data collection) 

This task comprised the stakeholder consultation activities conducted for the evaluation. A broad range of stakeholders were reached through the various 
consultation  activities,  covering  a  wide  geographical  scope,  showing  views  of  all  stakeholder  groups  directly  affected  by  or  having  an  interest  in  the 
programme. The consultation activities are summarised hereafter and analysed in the stakeholders’ consultation synopsis report (Annex V).  

The study included four surveys – a public consultation, a targeted survey of beneficiaries, a targeted survey of unsuccessful applicants, and a survey of 
Programme Committee Members. 

Online public consultation 

The  online  public  consultation  aimed  to  strengthen  the  evidence  base  for  the  evaluation  of  the  two  programmes  by  gathering  the  views  of  relevant 
stakeholders on the programmes’ effectiveness, efficiency, relevance, coherence, and EU added value. It was launched on 3 April 2024 and was open for 
12 weeks (closing on 26 June 2024). The questionnaire – including closed and open questions – was published on the dedicated “Have your say” webpage 
of the European Commission. Respondents were also invited to submit position papers at the end of the survey. A total of 8 responses were received. The 
factual summary report can be consulted here. 

Survey of beneficiaries 

The survey of beneficiaries was shared with all project coordinators for the two programming periods. It was hosted on the Qualtrics platform. The survey 
was launched on 29 March and closed on 24 May 2024. It was shared with 274 beneficiaries in total and received 109 responses. Respondents came from 
23 Member States129. 

Survey of unsuccessful applicants 

The  survey  of  unsuccessful  applicants  was  launched  on  29  March  and  closed  on  24 May  2024  for  those  who  had  unsuccessfully  applied  for  both 
programmes130.  It  was  hosted  on  the  Qualtrics  platform.  The  survey  was  shared  with  454  unsuccessful  applicants  in  total  and  received  79  responses. 

129 No responses from beneficiaries in Ireland, Malta, and Poland. 
130 As of 1 March 2019, to comply with GDPR requirements. 

53 

 
 
Respondents came from 18 Member States131 and Albania. The response rate and the number of respondents was considered sufficient for having produced 
valid results, particularly considering the challenge of engaging unsuccessful applicants, for whom the time investment in responding to a survey about 
programmes that they had not been a beneficiary of could reasonably be assumed to be inefficient. 

Survey of Programme Committee Members 

The survey of Programme Committee Members was launched on 29 March and closed on 24 May 2024. It was hosted on the Qualtrics platform. Since 
Programme Committee Members had already provided feedback on the 2014-2020 justice programme during the first part of the ex post evaluation, this 
survey focused on the current edition of the programme. The survey was shared with Committee Members in all Member States (n=78 contacts in total) 
and received 27 responses from respondents in 21 Member States132. 

The study also carried out 40 interviews with beneficiaries (n=35) and Programme Committee Members (n=5). 

Interviews with beneficiaries 

The supporting study conducted 35 interviews with beneficiaries across the two programmes. The sample aimed for a proportionate approach to balance 
both programmes, types of grants, types of beneficiaries per programme, specific objectives. The interviews were carried out using a semi-structured guide 
that facilitated discussions on a range of topics, including the effectiveness, efficiency, relevance and added value of the programmes, as well as the impact 
of COVID-19 and digitalisation. Interviews were conducted between 11 April and 31 July 2024. 

Interviews with Programme Committee Members 

The supporting study conducted 5 interviews with Programme Committee Members (i.e. those who accepted to be interviewed, representing Austria (n=2), 
France, Finland, and Italy). Interviews were conducted between 11 April and 31 July 2024. They followed a semi-structured guide, covering topics such as 
programme effectiveness, efficiency, relevance, added value, the impact of COVID-19, and digitalisation. The discussion also included topics such as 
gender mainstreaming and simplification. 

Case studies 

The three thematic case studies explored and provided additional in-depth qualitative and quantitative evidence illustrating topical issues. While each drew 
on findings from the main data collection activities conducted in the supporting study, additional data collection was conducted for case study 1 and case 
study 3. Table 3 provides an overview of the case studies’ themes and methodology. 

131 No responses from applicants in Cyprus, Estonia, Finland, Ireland, Latvia, Luxembourg, Poland, and Slovakia. 
132 Austria, Belgium, Bulgaria, Croatia, Czechia, Estonia, Finland, France, Greece, Hungary, Italy, Lithuania, Luxembourg, Malta, Netherlands, Poland, Portugal, Romania, Slovenia, Slovakia, and Spain. 

54 

 
 
Case study 
Case  study  1:  Evaluation of gender  mainstreaming  at 
the level of the programme and its activities 

Case  study  2:  Effects  generated  by  the  simplification 
efforts 

Case  study  3:  Adaptability  and  suitability  of  the 
programme for the (possibly) combined impact of the 
COVID-19 pandemic and new digital technologies 

Table 3. Overview of thematic case studies 

Methodology 
Desk-based  review  of  the  main  documents  at  programme  level  (e.g.  programme  regulations,  annual  work  programmes, 
programme statements, previous evaluations) and project level (e.g. proposals, periodic reports and grants agreement) for a 
sample (n=12) of all projects considered for the consultations of the overarching evaluation. Interviews with beneficiaries, 
Programme Committee Members and the European Commission (n=15 interviews in total). The case study also considered 
the results of the analysis carried out on the same topic in the context of the first part of the ex- post evaluation of the 2014-
2020 programme.  
Desk-based  review  of  the  main  documents  at  programme  level  (e.g.  programme  regulations,  annual  work  programmes, 
programme statements, previous evaluations). Analysis of  the findings from the surveys of beneficiaries and unsuccessful 
applicants. Interviews with beneficiaries from the sample of projects, Programme Committee  Members and the European 
Commission. 
Desk-based  review  of  EU  policy  documents  (e.g.  the  EU  2030  Digital  Compass,  Communication  on  the digitalisation  of 
justice, the e‑Justice strategy and associated action plans) as well as specific documents related to the justice field (e.g. impact 
assessment, evaluations and studies on digitalisation uptake within the EU; Eurobarometer reports; Justice Scoreboard; Rule 
of Law report; etc.). Desk-based review of the main documents at programme level (e.g. programme regulations, annual work 
programmes, programme statements, previous evaluations). Desk-based review of project documents (e.g. proposals, periodic 
reports and grants agreement of a selected sample of projects). Analysis of the findings from the surveys of beneficiaries and 
the text mining exercise as well as the finding from the previous evaluation of the justice programme (2014-2020). In-depth 
interviews with key stakeholders, including beneficiaries (n=9) from the sample of projects as well as EU stakeholders from 
EU Institutions and Agencies, EU networks and organisations, and Programme Committee Members (n=7). These interviews 
provided insights into the programme and the views of both EU institutions and Member States, as well as on the spread of 
digitalisation in justice systems, the impact of COVID-19, and potential synergies between EU-funded initiatives. 

Data analysis and triangulation 

Data analysis included both qualitative and quantitative methods based on the data collected in relation to each evaluation question. Several types of data 
analysis activities were performed, including: descriptive, association and inference analysis. 

Quantitative analysis 

Quantitative analysis aimed to identify specific and quantifiable outcomes of the programmes as well as any relevant trends. It was performed per each 
programme separately and grouped into different types: descriptive analysis of quantitative data, testing correlation or association between variables of 
interest, variance analysis, and regression analysis. 

55 

 
 
One aspect of the quantitative analysis consisted of testing association133 between variables measured through the survey with beneficiaries and unsuccessful 
applicants. For this purpose, a chi-square independence test134 was considered; if the assumption on the adequate sample size was violated, Fisher’s exact 
test135 was used instead. In the context of this analysis, the assumption was not satisfied in all cases, which is why Fisher’s exact test was finally used to 
conduct the association analysis. All statistically significant associations were further investigated with the Cramer’s V test136 which provided more insight 
about the strength of the association. 

The regression analysis  was based on ordinal logistic regression.  Instead, the analysis of variance used methods such as one-way analysis of variance 
(ANOVA)137, the Welch’s test138 or the Kruskal-Wallis test139 amongst others. 

The analysis also explored how outliers or certain groupings may affect its results. Outliers can be excluded from an analysis because they could have a 
significant impact on the results, potentially leading to misleading conclusions; throughout the analysis, outliers were excluded considering the organisation 
type of the survey respondents. Grouping was used to construct country regions, which were tested as a predictor throughout the entire analysis. Since the 
sample size did  not  allow stratification per country, the analysis relied on grouping  the countries according to  different  criteria.  The  supporting  study 
grounded the classification of Member States into four regions in the framework established by the 2023 EU Rule of Law report, which considers Member 
States  based  on  their  adherence  to  rule  of  law  principles,  including  judicial  independence,  anti-corruption  measures,  media  freedom,  and  governance 
effectiveness.  By  utilising  this  report  as  a  reference,  the  countries  were  grouped  according  to  shared  challenges  and  characteristics  that  reflect  their 
respective contexts in relation to the characteristics of their justice systems, rule of law issues and other related characteristics. Based on the parameters 
defined in the 2023 EU Rule of Law report, the geographical regions were defined as: 

•  Northern Europe: Denmark, Estonia, Finland, Ireland, Latvia, Lithuania, and Sweden. 
•  Western Europe: Austria, Belgium, France, Germany, Luxembourg and The Netherlands 
•  Southern Europe: Cyprus, Greece, Italy, Malta, Portugal and Spain. 
•  Eastern Europe: Bulgaria, Croatia, Czechia, Hungary, Poland, Romania, Slovakia and Slovenia. Albania was added to this group, as non-EU country 

that participated in the programme. 

133 The association analysis tested the hypothesis that certain outcomes, perceptions and behaviours may vary between different types of participants and between participants from different countries. In other words, the analysis 
assessed if organisation type and country are associated with any variation in stakeholder’s perceptions about the programmes’ effectiveness, efficiency and relevance. 
134 A statistical method used to determine if a significant association between categorical variables exists. It compares the observed frequencies in each category to the frequencies that would be expected if there were no association 
between the variables. 
135 A statistical test used to determine if there are non-random associations between two categorical variables in a contingency table, typically of size 2x2. It is particularly useful when sample sizes are small, or when the expected 
frequencies in the contingency table are too low for the chi-square test to be reliable. 
136 A test that measures how strongly two categorical fields are associated. 
137 A statistical method used to compare the means of three or more independent groups to determine if there is a statistically significant difference between them. 
138 When the homogeneity of variances assumption is not met, especially with unequal sample sizes, Welch’s Test is a good approach for performing an ANOVA analysis. 
139 The test is particularly useful when the assumptions of one-way ANOVA (such as normality and homogeneity of variances) are not met. 

56 

 
 
Different grouping was also tested for outcome variables. This involved Likert scale140 data obtained from the survey. 

Qualitative analysis 

The qualitative analysis relied on findings from the case studies, text mining as well as opinions gathered through interviews and the three surveys. 

To  analyse  the  interview  data,  a  combination  of  qualitative  research  methods  was  employed,  including  thematic  analysis,  content  analysis,  cross-case 
analysis, and the constant comparative method. These approaches allowed for a thorough and structured interpretation of responses that aligned with the 
evaluation’s goals and criteria. 

To support the  analysis  of the  efficiency of the  programme, the supporting study performed  a cost-effectiveness  analysis  (CEA), which compares the 
relative costs of the outcomes of two or more courses of action. The CEA allowed to assess and compare the cost-effectiveness ratios (CERs) of the two 
justice  programmes,  according  to  three  selected  indicators,  both  individually  across  the  yearly  outcomes  and  in  relation  to  each  other.  The  CER  was 
calculated using the formula Cost of intervention A / Outcome of intervention A.  The three indicators were selected based on the objective they were 
associated with and comparability across both programmes, namely: (a) number of exchanges of information on the European Criminal Records Information 
System (ECRIS), (b) number of members of the judiciary and judicial staff trained by the justice programme, and (c) number of hits on the e-justice portal. 
Each indicator was evaluated against the total cost of the programme annually. 

The results stemming from  the various data collection activities and analysis were triangulated to develop  robust answers to  the evaluation  questions. 
Several triangulation methods were used, including:  triangulation of different types of data, data collection tools, types of stakeholders as well as data 
sources. Triangulation was carried out for each evaluation question separately. 

Finally, the supporting study team organised a validation workshop with DG JUST aiming at discussing the draft conclusions and lessons learned. 

II.2 Points of comparison 

The points of comparison presented in the tables hereafter were established based on core performance indicators established by the respective regulations, 
previous evaluations and impact assessments as well as performance monitoring frameworks. Explanations are included in Section 2.3. 

140 A likert scale, or rating system, is a measurement method used in research to evaluate attitudes, opinions and perceptions. 

57 

 
 
 
 
Table 4. Justice programme 2014-2020: objectives, indicators and points of comparison 

Indicator 

Point of comparison 
(baseline) 

Point of comparison 
(target 2020) 

Achievement by end of programme 

Specific objectives (indicators set out in Article 2 of the programme Regulation) 

The number and percentage of persons in a target group 
reached by awareness-raising activities funded by the 
programme141 

Number and percentage of members of the judiciary and 
judicial staff that participated in training activities, staff 
exchanges, study visits, workshops and seminars funded 
by the programme 

The improvement in the level of knowledge of Union 
law and policies in the groups participating in activities 
funded by the programme compared to the entire target 
group 

0 

0 

0 

NA 

16 000 (percentage target 
= 0.13, i.e. the ratio 
between the number of 
supported participants and 
the total number of legal 
practitioners) 
NA 

The number of cases, activities and outputs of cross-
border cooperation, including cooperation by means of 
information technology tools and procedures established 
at Union level 
Participants’ assessment of the activities in which they 
participated and of their (expected) sustainability 

0 

0 

141 Data available for the period 2016-2020, as per findings of the first part ex-post evaluation. 
142 Special Eurobarometer survey conducted in 2021 and 2024. 

NA 

NA 

58 

10 200 000 EU citizens 
90 202 judges and judicial staff 
29 000 lawyers 
14 000 other professionals 
109 796 (percentage = 0.18) 

Over 60% of Europeans reports142 being 
aware of EU laws, with the most recognised 
being the right of EU citizens to live freely 
within the EU (82%), Equal treatment in 
employment (78%), and the General Data 
Protection Regulation (GDPR) (77%). A 
minimum of three out of every ten 
Europeans was familiar with EU initiatives 
and tools including the Code of conduct for 
countering illegal hate speech (56%), the 
European Disability Card online (48%) the 
European Citizens’ Initiative (41%), the 
European e-justice portal (31%), and the 
EU Justice Scoreboard (30%).  
1000 cross-border tools and mechanisms 

Most beneficiaries (94%) considered the 
effects of their projects as long-lasting and 

 
 
 
sustainable, with examples particularly 
evident in the creation of tools, 
partnerships, capacity building through 
training and increased awareness. 
Involvement in project activities positively 
influenced participants’ perceptions, 
knowledge, and to a lesser extent, 
behaviour. A significant majority (88%) 
reported a shift in their perception of the 
subjects they engaged with, while around 
half intended to apply their new knowledge 
in daily life and work. 
100% 

According to the feedback from 
beneficiaries, the majority of projects (83%) 
under the justice programme reached their 
objectives. 
191 instruments and tools 

Overall, the programme provided value at 
the EU level, beyond what could have been 
achieved by Member States acting alone. 
Where projects could have been funded in 
the absence of the justice programme, they 
would have been implemented on a smaller 
scale (geographically or in terms of their 
content). 
The programme was overall efficient, and 
the cost-effectiveness increased throughout 
its duration. Beneficiaries found the 
benefits of the programme outweighed the 
costs. Stakeholders mostly considered 
benefits resulting from the networking 
opportunities for which the effects 

The geographical coverage of the activities funded by 
the programme 
Additional indicators set out by the programme Regulation 

0% 

The perceived impact of the programme on access to 
justice based on qualitative and quantitative data 
collected at European level 

The number and quality of instruments and tools 
developed through actions funded by the programme143 
The European added value of the programme, including 
an evaluation of the programme’s activities in the light 
of similar initiatives which have been developed at 
national or European level without support from Union 
funding, and their (expected) results and the advantages 
and/or disadvantages of Union funding compared to 
national funding for the type of activity in question 

NA 

0 

NA 

100% of the participating 
countries 

NA 

NA 

NA 

The level of funding in relation to the outcomes 
achieved (efficiency) 

NA 

NA 

143 Ibidem. 

59 

 
 
materialised well beyond the original 
project timeframes. 
The programme has generally improved its 
administrative process between the 2014-
2020 and 2021-2027 programming periods. 
The simplification efforts were positively 
welcome by beneficiaries. Beneficiaries still 
report very heterogeneous effects of these 
measures on their workload and 
administrative burden, indicating how 
different organisations (not necessarily of 
the same size or type) have very different 
experiences in dealing with the changes to 
the administrative process (in particular 
those related to the introduction of unit 
costs). 

371 000 (2014), 494 753 (2015), 638 000 
(2016), 820 199 (2017), 1 023 919 (2018), 
318 000 (2020) 

14-20 days 

10 days 

21.25 days (2020) 

300 000 

630 000 

+20% per year 

3 500 000 

4 136 249 (2020) 

68 

95 

1 136 849 (2014); 1 751 180 (2015); 1 884 600 
(2016), 2 690,574 (2017); 2 962 558 (2018); 
4 343 547 (2019); 4 619 548 (2020) 
46 (2020) 

The possible administrative, organisational and/or 
structural obstacles to the smoother, more effective and 
efficient implementation of the programme (scope for 
simplification) 

NA 

NA 

Additional indicators reported in programme performance statements 

239 000 

700 000 

Cumulative number of legal professionals receiving 
training (not only through the programme) on EU law or 
law of another Member State, including Civil Justice, 
Criminal Justice and Fundamental Rights 
Average time of the surrender procedure (time between 
the arrest and the decision on the surrender of the person 
sought) under the European Arrest Warrant in cases 
where the person consents to the surrender (SO1) 
Number of exchanges of information in the European 
Criminal Records Information System (ECRIS) (SO1)144 
Number of hits on the e-justice portal (SO3)145 

Number of new psychoactive substances assessed 
(including through testing, if necessary) to enable the 
EU or the Member States to take appropriate action to 
protect consumers, depending on the type and level of 
risk that they may pose when consumed by humans 
(SO4) 

144 Based on data from the European Criminal Records Information System (ECRIS) platform (DG JUST, 2024). 
145 Based on elaboration of data extracted from the e-justice portal (DG JUST, 2024). 

60 

 
 
Table 5. Justice programme 2021-2027: objectives, core performance indicators and points of comparison 

Indicator 

Point of comparison 
(baseline) 

Point of comparison (target 2027) 

Achievement 2021-2023 

Facilitate and support judicial cooperation (JCOO) 

Number of exchanges via the European Criminal 
Records Information System (ECRIS)146 

4 136 249 (2020) 

People reached147 by awareness raising, information and 
dissemination activities 
People reached by mutual learning and exchange of 
good practices activities 
Civil society organisations reached by support and 
capacity building activities 
Support and promote judicial training (JTRA) 

0 

0 

0 

Milestones: 4 400 000 (2023), 4 800 
000 (2025) 
Target: 5 200 000 (2027) 
Milestone: 164 250 (2024) 
Target: 285 748 (2027) 
Milestone: 2 683 (2024)  
Target: 4 667 (2027) 
Milestone: 60 (2024) 
Target: 105 (2027) 

4 047 463 (2021) 
4 700 000 (2022) 
4 900 000 (2023) 
100 072 

21 253 

66 

0 

Number of members of the judiciary and judicial staff 
who participated in training activities 
People reached by awareness raising, information and 
dissemination activities 
People reached by mutual learning and exchange of 
good practices activities 
Civil society organisations reached by support and 
capacity building activities 
Facilitate effective and non-discriminatory access to justice for all (JACC) 

0 

0 

0 

Civil society organisations reached by support and 
capacity building activities 
Number of hits on the e-justice portal / pages addressing 
the need for information on cross-border civil and 
criminal cases149 
People reached by mutual learning and exchange of 
good practices activities 
People reached by awareness raising, information and 
dissemination activities 

0 

1 430 000 (2020) 

0 

0 

Target: 15 000 annually 

Milestone: 836 (2024) 
Target: 1451 (2027) 
Milestone: 11 945 (2024) 
Target: 20 728 (2027)  
Milestone: 88 (2024) 
Target: 154 (2027) 

Milestone: 156 (2024) 
Target: 253 (2027) 
Milestone: 2 100 000 (2024) 
Target: 2 800 000 (2027) 

Milestone: 2 627 (2024) 
Target: 4 574 (2027) 
Milestone: 160 881 (2024) 
Target: 280 075 (2027) 

12 700 (2021) 
24 000 (2022) 
301 

5 270 

91 

106148 

1 280 000 (2021) 
2 250 000 (2022) 

2 262 057 

101 515 

146 Based on data from the European Criminal Records Information System (ECRIS) platform (DG JUST, 2024). 
147 Values for all indicators on people reached are based on data from eGrants, namely Part C, as presented in beneficiaries’ applications. Values cover all projects awarded by 15 January 2024. The numbers were derived using the 
data point ‘Activity Type’ and relevant gender data points (male / female / non-binary). 
148 Data covers grants signed by December 2023, therefore CSOs reached by the JUST-2023-JACC-EJUSTICEcall are not included. 
149 Based on elaboration of data extracted from the e-justice portal (DG JUST, 2024). 

61 

 
 
ANNEX III. EVALUATION MATRIX AND DETAILS ON ANSWERS TO THE EVALUATION QUESTIONS (BY CRITERION) 

The evaluation design relied on two evaluation matrices built along the five Better Regulation Guidelines’ evaluation criteria that guided the evaluation 
process. Based on the programmes’ intervention logics and the supporting study technical specifications, the matrices feature the key research questions, 
indicators and related judgement criteria. The data sources are included in the tables. 

Justice programme 2014-2020 

EFFECTIVENESS 

EQ1 To what extent did the programme achieve its general and specific objectives?  

Judgement criteria 

Indicators 

Sources 

•  The  final  quantitative  data  for 
effectiveness  indicators  meet  the 
targets 

•  Stakeholders  confirm 

•  Stakeholders  confirm 

the 

that 
programme met its objectives 
that 

the 
programme  effects  are  sustainable 
and  can  identify  examples  of  its 
long-lasting effects 

Quantitative indicators 

Desk research 

•  Success rate  (number of projects funded out of total  proposals 

•  The report of the first part  ex post evaluation of the 

submitted) 

justice programme 2014-2020 

•  Percentage  of  projects  that  are  a  continuation  of  previously 

•  Programme data 

funded projects 

•  Number  of  long-lasting  judicial  cooperation  extended  beyond 

the project duration 

•  Number  of  projects  for  which  activities  continued  beyond  the 

project duration 

•  Cumulative number of legal practitioners who received training 

•  Number of exchanges of information in the European Criminal 

Records Information System (ECRIS) 

•  Number and percentage of members of the judiciary and judicial 
staff that participated in training activities, staff exchanges, study 
visits, workshops and seminars 

•  Number of hits on the e-Justice portal 

•  Number of new psychoactive substances assessed to enable the 
EU  or  the  Member  States  to take  appropriate  action  to  protect 

•  Annual reports on European Judicial Training 

•  The European Criminal Records Information System 

•  Annual  monitoring 

the 
reports 
implementation  of  the  Annual  Work  Programmes 
(AWP) 

(reports  on 

•  Database for the Justice programme 2014-2020150 
• 

Justice programme - Performance151 

•  The European Monitoring Centre for Drugs and Drug 

Addiction (EMCDDA)152 

•  National reports to Council EAW annual statistics153 
•  European  Commission  databases  (Programme 
data, Database for the Justice programme 2014-2020) 

150 European Commission Justice, Justice Programme 2014-2020. Available at: https://ec.europa.eu/justice/grants1/closed-calls/index_en.htm. 
151 European Commission, Justice Programme – Performance. Available at: Justice programme - Performance (europa.eu). 
152 EUDA, New psychoactive substances- the current situation in Europe. Available at: https://www.emcdda.europa.eu/publications/european-drug-report/2023/new-psychoactive-substances_en#. 
153 Eurojust, Annual Report 2022 – European Arrest Warrant. Available at: https://www.eurojust.europa.eu/annual-report-2022/judicial-cooperation-instruments/european-arrest-warrant. 

62 

 
 
 
 
consumers, depending on the type and level of risk that they may 
pose when consumed by humans 

•  Average  time  of  the  surrender  procedure  under  the  European 
Arrest  Warrant  (in  cases  where  the  person  consents  to  the 
surrender) 

•  Number of funded projects per year (action grants and operating 

grants) 

•  Number of procurement contracts signed per year and per type 

of project 

•  Number of calls for proposals per specific objective 

Qualitative indicators 

•  Stakeholders’ assessment of the performance of the programme 

•  Stakeholders’ assessment on programme meeting its objectives 

•  Literature  and  reports  (e.g.  the  report  of  the  first 
part  ex post  evaluation  of  the  Justice  programme 
the 
2014-2020,  Annual  monitoring  reports  on 
implementation  of  the  Annual  Work  Programmes, 
national  reports  to  Council  EAW  annual  statistics, 
etc.) 

Survey with Programme Committee Members 
In-depth interviews with the beneficiaries 

EQ2. To what extent are the effects of the programme long-lasting and sustainable? 

Judgement criteria 

Indicators 

Sources 

•  Stakeholders 

that 
programme’s  effects  have  been 
sustainable 

confirm 

•  Stakeholders  confirm  that  the  tools 
used  to  share  the  results  of  the 
programme are sustainable 

•  Stakeholders  provide  examples  of 
long-lasting 
the 
programme,  such  as  partnerships, 
continuous activities 

effects 

of 

Qualitative indicators: 

Desk research 

•  Stakeholders’ assessment of the sustainability of the Programme 
•  Stakeholders’ assessment of the tools sharing the results of the 

•  Eurobarometer data (2021 edition and 2024 edition) 

and EU Survey on Justice, Rights and Value 

programme 

•  Examples of long-lasting effects of the programme 

In-depth interviews with the beneficiaries 
In-depth interviews with the Programme Committee 
Members 
Survey with the beneficiaries 
Survey with the Programme Committee Members 

EQ3. If any, what were the unexpected or unintended effects (positive or negative) of the programme? What were the implications? 

Judgement criteria 

•  There are examples of and trends in 
unexpected  or  unintended  effects 
(positive  or  negative)  of 
the 
programme 

Indicators 

Sources 

Qualitative indicator 

Desk research: 

•  Examples of unexpected or unintended effects of the programme 

•  Project reports 

and their impact and the frequency of these examples 

Text mining 
In-depth interviews with beneficiaries and Programme 
Committee Members 
Surveys with beneficiaries 

63 

 
 
EQ4. How evenly are the effects observed so far distributed across the different target groups and participating countries? What are the likely reasons of variation in the 
programme’s effects across target groups and Member States? 

Judgement criteria 

Indicators 

Sources 

Surveys with Programme Committee Members 

•  The  final  quantitative  data  confirm 
that the effects of the programme are 
evenly 
the 
different target groups 

distributed 

across 

•  The  final  quantitative  data  confirm 
that the effects of the programme are 
across 
distributed 
evenly 
participating countries 
•  Stakeholders  confirm 

the 
distribution of programme’s effects 
has been evenly distributed 

that 

Quantitative indicators 

Desk research 

1.  Number  of  applications  and  grants  (action  and  operating) 

•  The report of the first part  ex post evaluation of the 

awarded by country and by year 

justice programme 2014-2020 

2.  Number  of  applications  and  grants  (action  and  operating) 

•  Monitoring data from DG JUST 

awarded by year per target group 

Qualitative indicator 

•  Frequency of each target group mentioned in project application 

per specific programme objective 

•  Stakeholders’  assessment  on  the  distribution  of  programme’s 
effects to different target groups and participating countries 

•  Annual  monitoring 

the 
reports 
implementation  of  the  Annual  Work  Programmes 
(AWP) 

(reports  on 

•  Database for the justice programme 2014-2020 

•  European  Commission  databases  (Programme 
data, Database for the justice programme 2014-2020) 

•  Literature  and  reports  (e.g.  the  report  of  the  first 
part  ex post  evaluation  of  the  justice  programme 
the 
2014-2020,  Annual  monitoring  reports  on 
implementation of the Annual Work Programmes) 

Survey with the Programme Committee Members 
Survey with the beneficiaries 
Text mining: different target groups per specific objectives 
reflected in projects reports 

EFFICIENCY 

EQ5. Have the effects of the programme been achieved at initially expected costs or were these costs different? What could explain the differences (if any)? 

Judgement criteria 

Indicators 

Sources 

•  The final financial data confirm that 
the  programme  effects  have  been 
achieved 
expected 
(planned) costs 

initially 

at 

•  There are factors identified that can 
explain the differences (if any)  

Quantitative indicators 

Desk research: 

•  Actual and planned costs of an of awareness raising, information 

and dissemination activities 

•  Actual and planned costs of a mutual learning and exchange of 

good practices activities 

•  Actual and planned costs of a training activities 

•  European  Commission  databases  (Programme 
data, Database for the justice programme 2014-2020) 

•  Literature  and  reports  (e.g.  the  report  of  the  first 
part  ex post  evaluation  of  the  Justice  programme 
the 
2014-2020,  Annual  monitoring  reports  on 
implementation  of  the  Annual  Work  Programmes, 

64 

 
 
•  Actual and planned costs of 1 person/day/trainings 

•  Actual and planned costs of 1 person trained 

•  Actual and planned budget of procurement contracts 

national  reports  to  Council  EAW  annual  statistics 
etc.) 

•  Programme original budget/budgetary assumptions 

•  Actual and planned budget per year and per specific objective 

In-depth interviews with DG JUST staff 

EQ6. Was the implementation of the programme, including administrative costs, efficient or could the programme be implemented in a more efficient way? 

Judgement criteria 

Indicators 

Sources 

•  Stakeholders  confirm 

the 
the  programme  are 

that 

benefits  of 
higher than the costs incurred 
•  Stakeholders  identify  factors  that 
influenced  cost  efficiency  of  the 
programme 

•  Stakeholders  confirm 

that 

the 
simplification  measures,  including 
streamlining  the  application  rules 
and  procedures, 
increased  cost-
benefit ratio 

•  Evidence 

suggests 

that 

programme 
suffered 
underspending on projects 

the 
from 

Quantitative indicators 

Desk research 

•  The level of understanding on projects 

•  The report of the first part  ex post evaluation of the 

Qualitative indicators 

1. 

Identified  examples  of  irregularities  or  delays  in  programme 
implementation 

2.  Stakeholder feedback on cost-benefit ratio of the programme 
3.  Stakeholder  feedback  on  cost-benefit  ratio  of  similar  funding 

schemes 

4.  Stakeholders  feedback  on  factors 

influencing  efficiency, 

including feedback on simplification measures 

justice programme 2014-2020 

•  Project data (planned budgets against spendings) 

•  Programme ex post evaluation (first part) 
Survey with the Programme Committee Members 
Survey with the beneficiaries 
In-depth interviews with DG JUST staff 
In-depth interviews with selected beneficiaries 

•  There  were  irregularities  or  delays 
implementation, 

in 
programme 
e.g.in publishing calls 
•  Stakeholders  confirm 

that  cost-
benefit  ratio  of  the  programme  is 
higher than in case of other similar 
funding schemes 

EQ7. How did the introduced simplification measures influence the cost-benefit ratio? 

Judgement criteria 

Indicators 

Sources 

Qualitative indicators 

Case study 2: 

•  Stakeholders  feedback  on  factors 

influencing  efficiency, 

•  The effects generated by the simplification efforts 

including feedback on simplification measures 

65 

 
 
 
 
RELEVANCE 

EQ8.1 How well did the objectives of the programme correspond to the needs of the different stakeholders? 

Judgement criteria 

•  Stakeholders confirm that the design 
of  the  programme  was  adequate  to 
meet their needs 

•  Extent to which there is flexibility in 
the programme to address emerging 
needs 

•  Stakeholders  report  the  need  to 
adapt  the  programme  objectives  or 
design to better address their needs 

Quantitative indicators 

Desk research 

Indicators 

Sources 

•  European  Commission  databases  (Programme 
data, list of calls of the justice programme 2014-2020) 

•  Literature  and  reports  (e.g.  the  report  of  the  first 
part  ex post  evaluation  of  the  justice  programme 
2014-2020,  Annual  monitoring  reports  on 
the 
implementation of the Annual Work Programmes) 

Survey with Programme Committee Members 
Survey with beneficiaries and applicants 
In-depth interviews with selected beneficiaries 
In-depth interviews with other key stakeholders 

•  Percentage of beneficiaries that are recurrent beneficiaries and/or 

in recurring consortia 

•  Distribution of funding by type of beneficiary 

•  Number of projects that are not in line with the core needs of the 
beneficiary out of the total number of funded projects (business 
opportunity) 

•  Number of final beneficiaries that confirmed that programme’s 

objectives correspond to their needs 

Qualitative indicator 

•  Stakeholders feedback on the adequacy of the programme design 

to meet their needs 

•  Examples of adapting the programme design to emerging needs 

and changing situation 

•  Stakeholders  reporting  the  need  to  amend  the  programme 

objectives or design to better address their needs 

•  Evidence  confirming  that  the  recommendations  regarding 
stakeholder needs from the interim evaluation were reflected in 
the programme implementation 

EQ 8.2 How relevant was the programme to its final beneficiaries? 

Judgement criteria 

•  Stakeholders confirm that the design 
of  the  programme  was  adequate  to 
meet the needs of the target groups 
•  Extent to which there is flexibility in 
the programme to address emerging 
needs 

•  Stakeholders  report  the  need  to 
adapt  the  programme  objectives  or 

Indicators 

Sources 

Quantitative indicators 

Desk research 

•  Number of final beneficiaries that confirmed that programme’s 

objectives correspond to their needs 

Qualitative indicator 

•  Stakeholders feedback on the adequacy of the programme design 

to meet the needs of the targeted stakeholders 

•  European  Commission  databases  (Programme 
data, Database for the justice programme 2014-2020) 

•  Literature  and  reports  (e.g.  the  report  of  the  first 
part  ex post  evaluation  of  the  justice  programme 
2014-2020,  Annual  monitoring  reports  on 
the 
implementation of the Annual Work Programmes) 

66 

 
 
 
design to better address the needs of 
its target groups 

•  Examples of adapting the programme design to emerging needs 

and changing situation 

•  Eurobarometer  data  (2021  edition  and  2024 
edition) and EU Survey on Justice, Rights and Value 

Survey with Programme Committee Members 
Survey with beneficiaries and applicants 
In-depth interviews with selected beneficiaries 
In-depth interviews with other key stakeholders 

EQ 8.3 What, if any, are the key lessons that can be learnt? 

Judgement criteria 

Indicators 

Sources 

•  Extent 

to 

the 
previous 
recommendations 
evaluations were used to amend the 
design of the programme 

which 
of 

Qualitative indicator 

Desk research 

•  Evidence  confirming  that  the  recommendations  regarding 
stakeholder needs from the interim evaluation were reflected in 
the programme implementation 

•  European  Commission  databases  (Programme 
data, Database for the justice programme 2014-2020) 
•  Literature  and  reports  (e.g.  the  report  of  the  first 
part  ex post  evaluation  of  the  justice  programme 
2014-2020,  Annual  monitoring  reports  on 
the 
implementation of the Annual Work Programmes) 

Text mining 
In-depth interviews with the beneficiaries 
In-depth interviews with the Programme Committee 
Members 
Survey with the beneficiaries and applicants 
Survey with the Programme Committee Members 

COHERENCE 

EQ9. To what extent was the programme coherent with wider EU policies and priorities? 

Judgement criteria 

Indicators 

Sources 

•  Extent to which the objectives of the 
programme were in line with wider 
EU  policies  and  priorities,  and 
international commitments, such as 
the 
Sustainable 
2030 
Development Agenda 

UN 

•  Extent  to  which  there  are  overlaps 
with  other  types  of  EC  and  other 

Quantitative indicators 

Desk research: 

•  Number of projects that were in line with wider EU policies and 
priorities, and international commitments, such as the UN 2030 
Sustainable Development Agenda 

•  The report of the first part  ex post evaluation of the 

justice programme 2014-2020 

•  Monitoring data from DG JUST 

•  Number of projects that overlapped with other types of EC and 
similar 

programmes 

support 

that 

other 
projects/objectives 

funding 

•  Annual  monitoring 

the 
reports 
implementation  of  the  Annual  Work  Programmes 
(AWP) 

(reports  on 

67 

 
 
 
funding  programmes  that  support 
similar projects/objectives 

•  Extent  to  which  the  beneficiaries 
managed 
synergies 
to  achieve 
between  their  projects  and  other 
funding programmes  

Qualitative indicators 

•  Database for the justice programme 2014-2020 

•  Links between the programme objectives and wider EU policies 

•  Programme interim evaluation 

and priorities, and international commitments 

•  Examples of overlaps with other types of EC and other funding 

programmes that support similar projects/objectives 

•  Programme ex post evaluation (first part) 

Additional desk research 

•  Reports of other EU instruments 
In-depth interviews with DG JUST staff 

EQ10. To what extent are there synergies and/or complementarities between the programme and other EU instruments? 

Judgement criteria 

•  Extent  to  which  the  beneficiaries 
managed 
synergies 
to  achieve 
between  their  projects  and  other 
funding programmes 

Indicators 

Sources 

Quantitative indicators 

Desk research: 

•  Number (share) of projects that develop synergies with projects 

•  The report of the first part  ex post evaluation of the 

financed by other EU sources 

Qualitative indicators 

justice programme 2014-2020 

•  Monitoring data from DG JUST 

•  Duplications  of  tasks  and  responsibilities  between  the  Justice 

programme and wider EU policies and priorities 

•  Level  of  coordination  between  the  programme  and  wider  EU 

policies and priorities 

•  Annual  monitoring 

the 
reports 
implementation  of  the  Annual  Work  Programmes 
(AWP) 

(reports  on 

•  Database for the justice programme 2014-2020 

•  Programme interim evaluation 

•  Programme ex post evaluation (first part) 

Additional desk research 

•  Reports of other EU instruments 
In-depth interviews with DG JUST staff 

EU ADDED VALUE 

EQ11. To what extent could the impact of the programme have been achieved without it by the Member States acting alone? 

Judgement criteria 

Indicators 

Sources 

•  Extent  to  which  it  is  possible  to 
define  the  elements  that  represent 
the  added  value  of  the  programme 
compared  to  what  could  have  been 
achieved  by  the  Member  States 
acting alone 

Quantitative indicator 

•  Number of projects creating first time transnational partnerships 

Qualitative indicators 

•  Stakeholders’ assessment of participating countries’ capacity to 

achieve similar effects without the programme 

Survey with Programme Committee Members 
In-depth with selected beneficiaries 
In-depth with other key stakeholders 

68 

 
 
 
•  Extent 

that 

to  which 

similar 
implementation 

stakeholders 
effects, 
confirm 
of 
including 
transnational  projects,  would  have 
been achieved by the Member States 
acting alone 

•  Extent 

to  which 

stakeholders 
identify  negative  consequences  of 
potential limitation or withdrawal of 
the programme 

•  Stakeholders’  assessment  of  beneficiaries  being  able 

to 
implement their projects or similar actions without the funding 
provided by the Programme 

•  Examples of existing alternative funding schemes 

•  Examples  of  achievements  of  the  Programme  that  would  not 

have been possible without the EU-level intervention 

•  Stakeholders’  assessment  of  the  consequences  of  potential 

limitation or withdrawal of the Programme 

EQ12. To what extent did the programme help establish a level playing field between the Member States? 

Judgement criteria 

Indicators 

Sources 

•  Extent  to  which  it  is  possible  to 
define  a  positive  impact  of  the 
programme on establishing a level-
playing field 

Quantitative indicators 

Desk research 

•  Number of victim support organisations with national coverage 

•  Number  of  applications  and  grants  (action  and  operating) 

awarded by country and by year 

•  Number of collaborations and transnational networks that were 

developed under the programme 

Qualitative indicators 

•  Stakeholders  confirm  that  the  programme  helped  establish  a 

level playing field 

•  Stakeholders  identify  specific  outcomes  that  illustrate  the 
positive impact of the programme on establishing a level-playing 
field 

•  European  Commission  databases  (Programme 
data, Database for the Justice programme 2014-2020) 

•  Literature  and  reports  (e.g.  the  report  of  the  first 
part  ex post  evaluation  of  the  Justice  programme 
the 
2014-2020,  Annual  monitoring  reports  on 
implementation  of  the  Annual  Work  Programmes, 
national  reports  to  Council  EAW  annual  statistics, 
etc.) 

Survey with the Programme Committee Members 
In-depth interview with DG JUST staff 
In-depth interview with other key stakeholders 
In-depth interviews PCMs 
In-depth interviews with beneficiaries 

EQ13. Which were the most significant transnational/cross-border aspects the programme tackled? How could these be quantified? 

Judgement criteria 

Indicators 

Sources 

Quantitative indicators 

Desk research 

•  Number of projects with good practice transfer from one country 

to another 

•  Number of exchanges of information in the European Criminal 

Records Information System 

•  European  Commission  databases  (Programme 
data, Database for the justice programme 2014-2020) 

•  Literature  and  reports  (e.g.  the  report  of  the  first 
part  ex post  evaluation  of  the  justice  programme 

69 

 
 
 
•  Number of exchanges of professionals 

2014-2020,  Annual  monitoring  reports  on 
the 
implementation  of  the  Annual  Work  Programmes, 
national  reports  to  Council  EAW  annual  statistics, 
etc.) 

Survey with Programme Committee Members 
In-depth interview with DG JUST 
In-depth interviews PCMs 
In-depth interviews with beneficiaries 

70 

 
 
Justice programme 2021-2027 

EFFECTIVENESS 

EQ1. To what extent is the programme on track to meeting its objectives, including the general and specific objectives? How does it compare with the previous 
programme? 

Judgement criteria 
•  Quantitative  data  for  effectiveness  indicators 

Quantitative indicators 

Desk research: 

Indicators 

Sources 

meet the mid-term targets 

•  Success rate (number of projects funded out of 

•  European Commission data (Programme data) 

•  Literature  and  reports 

programmes,  monitoring  reports,  national  reports 
Council EAW annual statistics154 etc.) 

(e.g.  Multi  annual  work 
to 

•  Annual reports on European Judicial Training 

•  The European Criminal Records Information System 

•  Database for the Justice programme 2021-2027155 
• 

Justice programme - Performance156 

Survey with Programme Committee Members (PCMs) 
In-depth interviews with the beneficiaries 

•  Stakeholders  confirm  that  the  programme  is 

total proposals submitted) 

meeting its objectives 

•  Number  of  members  of  the  judiciary  and 
judicial  staff  who  participated  in  training 
activities 

•  Number of people reached by mutual learning 
and exchange of good practices activities 

•  Number  of  exchanges  via  the  European 
Information  System 

Criminal  Records 
(ECRIS) 

•  Number of hits on the e-Justice portal / pages 
addressing the need for information on cross-
border civil and criminal cases 

•  Number  of  hits/number  of  visitors  to  the  e-
learning  trainings  in  the  European  Training 
Platform 

154 Ibid. 
155 Available at: https://ec.europa.eu/info/funding-tenders/opportunities/portal/screen/programmes/just2027. 
156 European Commission, Justice Programme – Performance. Available at: Justice programme - Performance (europa.eu). 

71 

 
 
 
•  Number  of hits  on  the  guides  and  factsheets 

on the e-Justice portal 

•  The average time of the surrender procedure 
(time between the arrest and the decision on 
the surrender of the person sought) under the 
European  Arrest  Warrant  in  cases  where  the 
person consents to the surrender 

•  Number  of  visits  to  the  pages  dealing  with 
criminal matters on the e-Justice portal 

•  Number  of  funded  projects  per  year  (action 

grants and operating grants) 

•  Number of procurement contracts signed per 

year and per type of project 

•  Number  of  calls  for  proposals  per  specific 

objective 

•  Annual  number  of 

legal  practitioners 
participating in training on EU law in the EU 

Qualitative indicators 

•  Stakeholders’ assessment of the performance 

of the current programme 

•  Stakeholders’  assessment  of  the  level  of 
breaches of time limits in the procedures 
•  Stakeholders’  assessment  of  the  number  of 

reported crimes by victims 

•  Stakeholders’  assessment  of  the  number  of 

reported crimes by victims 

EQ2. How likely is it that the actions implemented so far will generate long-term effects? 

Judgement criteria 

Indicators 

Sources 

72 

 
•  Stakeholders  confirm  that  the  programme 
effects  are  sustainable  and  can  identify  any 
preliminary examples of its long-lasting effects 
the  participants 
perception/behaviours/practice  have  changed 
due  to  participation  in  actions  funded  by  the 
programme 

is  evidence 

•  There 

that 

•  Stakeholders  confirm  that  the  tools  used  to 
the  programme  are 

the  results  of 

share 
sustainable 

Qualitative indicators 
Stakeholders’ assessment of 

Desk research: 

•  Eurobarometer  data  (2021  edition  and  2024  edition)  and 

•  Change in participants’ perception 

EU Survey on Justice, Rights and Value 

•  Change in participants’ behaviour 

•  Change in participants’ practice 

•  Stakeholders’ assessment of the sustainability 

of the programme 

•  Stakeholders’ assessment of the tools sharing 

the results of the programme 

In-depth interviews with the beneficiaries 
In-depth interviews with the Programme Committee Members 
Survey with the beneficiaries 
Survey with the Programme Committee Members 
Text mining 

EQ3. What external factors have influenced (positively and negatively) the effectiveness of the programme? What is the impact of the novelties introduced in the 
current programme? 

Judgement criteria 
•  There are external factors that have influenced 
(positively and negatively) the effectiveness of 
the programme 

•  Stakeholders confirm there’s an impact of the 
the  current 

introduced 

key  novelties 
programme, more particularly: 

in 

•  There is evidence of a more prominent role of 

the rule of law aspect 
evidence 
is 

•  There 

Organisations 
explicitly 

that  Civil  Society 
targeted  more 

(CSOs)  are 

Quantitative indicator: 

Desk research 

Indicators 

Sources 

•  Number of civil society organisations reached 
by support and capacity building activities per 
specific objective 

Qualitative indicators: 

•  Stakeholders  providing  examples  of  external 
factors  influencing  the  effectiveness  of  the 
programme and their effects 

•  Rule  of  law  aspect  reflected  in  project 

applications 

•  Lists of beneficiaries 
•  A sample of projects applications 

Text mining 
In-depth interviews with beneficiaries and Programme 
Committee Members 
Surveys with beneficiaries 
Survey with Programme Committee Members 

•  There  is  evidence  of  enhanced  focus  on  anti-

discrimination in the access to justice 

•  Enhanced focus on anti-discrimination in the 
in  project 

reflected 

justice 

access 
to 
applications 

EQ4. How effective have the communication activities been in informing stakeholders about and engaging them in the programme as well as in increasing the 
programme’s visibility? 

Judgement criteria 

Indicators 

Sources 

73 

 
 
confirms 

•  Quantitative 

data 
performance per communication tool 
•  Stakeholders  feedback  confirms  good 
performance per communication tool 

good 

Quantitative indicators 

Desk research 

•  Number  of  people  reached  by  awareness 
and  dissemination 

information 

raising, 
activities 

•  Number  of  visits  to  the  pages  dealing  with 
criminal matters on the e-Justice portal 

•  Number  of  informational  posts  concerning 
awareness 
and 
dissemination activities available to the public 
on the e-Justice portal 

information 

raising, 

•  European Commission databases 
•  Literature and reports (e.g. multi annual work programmes, 

monitoring reports etc.) 

•  Social media analytics, website performance data and other 
communication data of communication tools and channels 
related to the programme 

 Survey with the Programme Committee Members 

•  Other  communication  indicators,  depending 
on  the  availability  of  data  and  tools  used, 
the  2022  EC  Communication 
based  on 
Indicators157 at output and result levels 

Qualitative indicators 

•  Assessment  of  communication  about  the 

programme by stakeholders 

•  Beneficiaries  confirming  the  communication 
tool,  which  they  were  reached  by,  about  the 
programme 

EQ5. How evenly are the effects observed so far distributed across the different target groups and participating countries? What are the likely reasons of variation in 
the programme’s effects across target groups and Member States? 

Judgement criteria 

Indicators 

Sources 

157 European Commission, 2022 EC Communication Indicators. Available at: https://commission.europa.eu/system/files/2023-02/2022%20EC%20Comm%20indicators.pdf. 

74 

 
 
•  The  quantitative  data  confirm  that  the 
effects  of  the  programme  are  evenly 
target 
distributed  across 
groups 

the  different 

•  The  quantitative  data  confirm  that  the 
effects  of  the  programme  are  evenly 
distributed across participating countries 

Quantitative indicators 

Desk research: 

•  Number of transnational networks 

•  Funding distribution per country 
•  Funding distribution per type of beneficiary 

•  European Commission databases (Programme data) 
•  Annual reports on European Judicial Training 

•  The European Criminal Records Information System 

•  Number  of  projects  with  good  practice 

•  Annual monitoring reports (reports on the implementation 

transfer from one country to another 

of the Annual Work Programmes (AWP) 

•  Number  of  projects  creating  first 

time 

transnational partnership 

•  Database for the justice programme 2021-2027 
• 

Justice programme - performance 

Text mining: 

•  Different target groups per specific objectives reflected in 

project reports 

Survey with the Programme Committee Members 

•  Number of applications and grants (action and 
operating) awarded by country and by year 

•  Number of applications and grants (action and 

operating) awarded by year per target group 

Qualitative indicator 

•  Frequency of each target group mentioned in 
project  application  per  specific  programme 
objective 

•  Stakeholders  feedback  on  identified  reasons 
of  beneficiaries  from  countries/target  groups 
less  often  applying  for  funding  under  the 
programme 

EQ6. How effectively has gender equality been mainstreamed in the programme design and implementation? What is the (prospective) impact of the Programme on 
gender equality? 

Judgement criteria 

Indicators 

Sources 

•  The data confirm that the gender equality 
has  been  effectively  mainstreamed  in  the 
Programme design and implementation 
•  Stakeholders  confirm  the  (prospective) 
impact  of  the  programme  on  gender 
equality 

Qualitative indicators 

Desk research 

•  Extent 

to  which  gender 

equality 

is 

mainstreamed in the projects 

•  A sample of project applications 
•  Text mining 

•  Promotion of gender equality as perceived by 

participants 

Case study 1: Evaluation of gender mainstreaming at the level of 
the programme and its activities 

75 

 
 
 
 
 
• 

Impact of the programme on gender equality 

Survey with Programme Committee Members 

EFFICIENCY 

EQ7.1 To what extent has the programme been cost effective so far? Have there been any inefficiencies or unnecessary burden on the different stakeholders? 
EQ7.2 What internal and external factors have influenced (positively or negatively) the cost-benefit ratio of the programme? 

Judgement criteria 
financial 

•  The 

that 
data 
programme’s  results  will  be  achieved  at 
initially expected (planned) costs 

confirm 

•  The costs of the programme are similar or 

lower that in the previous funding period 

•  Stakeholders identify any inefficiencies or 

unnecessary burdens 

•  There  are  identified  internal  and  external 
factors positively or negatively influencing 
the cost-benefit ratio of the Programme 

Quantitative indicators 

Desk research: 

Indicators 

Sources 

•  Number of awareness raising, information and 

•  European Commission databases (Programme data) 

•  Literature and reports (Annual reports on European Judicial 
Training,  The  European  Criminal  Records  Information 
System,  Annual  monitoring  reports  (reports  on 
the 
implementation of the Annual Work Programmes (AWP). 
Database  for  the  justice  programme  2021-2027,  Justice 
programme - performance 

Survey with the programme committee members 
In-depth interviews with DG JUST staff 
In-depth interviews with selected beneficiaries 
Survey with beneficiaries 

dissemination activities contracted 

•  Number of awareness raising, information and 

dissemination projects granted 

•  Number of mutual learning and exchange  of 

good practices activities contracted 

•  Number of mutual learning and exchange  of 

good practices projects granted 

•  Number of training activities contracted 

•  Number of training projects granted 

•  Number  of  analytical  and  monitoring 

activities contracted 

•  Number of analytical and monitoring projects 

granted 

•  Number  of  indirect  management  actions 

contracted 

•  Actual and planned cost of awareness raising 
information and dissemination activities 

•  Actual and planned cost of a mutual learning 
and exchange of good practices activities 

76 

 
 
 
•  Actual and planned cost of a training activities 

•  Actual 

and 

planned 

cost 

of 

1 

person/day/training 

•  Actual  and  planned  budget  of  procurement 

contracts 

Qualitative indicators 
•  Perception  on 

•  Stakeholders 

the  administrative  burden 
caused by the inefficiencies of the programme 
factors 
influencing  the  cost-benefit  ratio  of  the 
programme 

external 

identify 

EQ8.1. How timely and efficient is the programme’s administrative process, including for reporting and monitoring? 

•  Stakeholders 

Judgement criteria 
the 
confirm 
programme’s  administrative  process  is 
timely  and  efficient, 
for 
reporting and monitoring. 

including 

that 

Qualitative indicators 

Indicators 

•  Timing  of  the  programme  calls/application 

process 

Sources 

In-depth interviews with DG JUST staff 
In-depth interviews with the beneficiaries 
Survey with Programme Committee Members 

•  Examples 

of 

delays 
administrative  issues  with  calls/application 
process/reporting (at programme level) 

inefficiencies, 

•  Stakeholders’ 
assessment 
administrative 
process, 
reporting and monitoring 

of 

including 

the 
for 

EQ8.2. To what extent have simplification measures introduced in the current programme influenced its cost efficiency? 

Judgement criteria 
to  which 

•  Extent 

the 

measures 
programme influenced its cost efficiency 

introduced 

in 

simplification 
the  current 

Case study 2: The effects generated by the simplification efforts 

Sources 

Qualitative indicators 

Indicators 

•  Stakeholders’ assessment of the impact of the 
simplification  measures  (standardised  cost-
to 
options, 
simplify reporting, etc.) 

results-based  management 

77 

 
 
 
•  Stakeholders’ comparative assessment of the 
2021-2027 
process 

administrative 
programme with 2014-2020 Programme 

of 

RELEVANCE 

EQ9. To what extent is the programme still relevant to interested stakeholders and the citizens and how likely it is that it will continue being relevant in the near future? 
Is the programme addressed to relevant target groups? 

Judgement criteria 

Indicators 

Sources 

•  There  is  evidence  in  the  external  context 
that  the  programme  objectives  are  still 
relevant 

•  Evidence  that  the  programme  and  its 
objectives still meet the needs of the target 
audiences and address relevant issues. 

Qualitative indicators 
Stakeholders feedback on: 

•  Participants’ overall assessment of activities 

Desk research 

•  Selected projects where the evaluation of the project was 
carried out (e.g. survey of the participants of the project) 

•  Participants’ awareness 

•  Participants’ knowledge 

•  Participants’ skills 

•  Participants’ cooperation 

•  Target group of the participants 

Survey with Programme Committee Members 
Survey with beneficiaries and applicants 
In-depth interviews with selected beneficiaries 
In-depth interviews with other key stakeholders 

EQ10. To what extent has the programme integrated lessons learnt from its predecessor? 

Judgement criteria 

Indicators 

Sources 

•  Extent  to  which  there  are  examples  of 
the  previous 
recommendations 
programme 
being 
implemented  and  reflected  in  the  current 
programme design 

evaluations 

from 

Qualitative indicator 

•  Examples  of  recommendations  being/not 

being implemented 

In-depth interviews with DG JUST staff 
Survey with Programme Committee Members 
Survey with the beneficiaries 

EQ11. Since its introduction, how well does the programme adapt to the pace of progress generated by digital technologies? 

Judgement criteria 

Indicators 

Sources 

•  Extent to which the programme  has been 
able  to  adapt  to  the  pace  of  progress 
generated by digital technologies 

Quantitative indicators 

Desk research: 

•  Number of ICT tools activities contracted 

•  European Commission databases (Programme data) 

78 

 
 
•  Number of ICT tools projects granted 

Qualitative indicators 

•  Examples  of  Programme  design  being 
adapted to the pace of progress generated by 
digital technologies 

•  Literature  and  reports  (Annual  monitoring  reports 
(reports  on  the  implementation  of  the  Annual  Work 
Programmes (AWP), Database for the justice programme 
2021-2027, Justice programme - performance) 

In-depth interviews with DG JUST staff 
Survey with Programme Committee Members 
Survey with beneficiaries and applicants 
Case study 3: Adaptability and suitability of the programme for 
the combined impact of Covid-19 and new digital technologies 

COHERENCE 

EQ12.1 To what extent is the programme coherent and complementary with other EU and national policies and funding programmes that have similar objectives, as 
well as international commitments and objectives, including the Sustainable 17 Development Goals? 
EQ12.2 To what extent have the various components of the programme generated synergies and/or compensated possible trade-offs amongst them? 

Judgement criteria 

Indicators 

Sources 

•  Extent  to  which  the  objectives  of  the 
programme are in line with the wider EU 
policies  and  priorities,  and  international 
commitments,  such  as  the  UN  2030 
Sustainable Development Agenda 

•  Extent  that  there  are  overlaps  with  other 
types of EC and other funding programmes 
that support similar projects/objectives 
•  Extent  that  the  beneficiaries  managed  to 
achieve  synergies  between  their  projects 
and other funding programmes 

Qualitative indicators 

Desk research: 

•  Degree of establishment of the new European 
Rule  of  Law  Mechanism  in  line  with  the 
Political Guidelines 

•  Number  (share)  of  projects  that  develop 
synergies with projects financed by other EU 
sources 

•  Number of projects that are in line with wider 
EU  policies  and  priorities,  and  international 
commitments, 
the  UN  2030 
Sustainable Development Agenda 

such  as 

•  Number  of  projects  that  overlap  with  other 
types  of  EC  and  other  funding  programmes 
that support similar projects/objectives 

•  European Commission databases (Monitoring data from 

DG JUST) 

•  Literature  and  reports  (Annual  monitoring  reports 
(reports  on  the  implementation  of  the  Annual  Work 
Programmes (AWP), Database for the justice programme 
2021-2027, Justice programme - performance) 

In-depth interviews with DG JUST staff 
Survey with Programme Committee Members 
Survey with beneficiaries and applicants 
Text mining 

79 

 
 
 
 
•  Links between the programme objectives and 
wider  EU  policies  and  priorities,  and 
international commitments 

•  Examples of overlaps with other types of EC 
and  other  funding  programmes  that  support 
similar projects/objectives 

EU ADDED VALUE 

EQ13.1 To what extent does the programme provide additional value compared to what could have been reasonably achieved by Member States acting alone? What 
would be the most likely consequences of stopping the programme and what would be the most likely consequences of not proposing a follow-up programme under the 
next MFF? 
 EQ13.2 Are the subsidiarity arguments put forward in the prior impact assessment still valid? 

Judgement criteria 

Indicators 

Sources 

•  Extent to which it is possible to define the 
elements that represent the added value of 
the  programme  compared  to  what  could 
have been achieved by the Member States 
acting alone 

•  Extent to which stakeholders confirm that 
similar  effects,  including  implementation 
of transnational projects, would have been 
achieved  by  the  Member  States  acting 
alone 

•  Extent  to  which  stakeholders  identify 
negative 
potential 
consequences 
limitation or withdrawal of the programme 
that  subsidiarity 
arguments put forward in the prior impact 
assessment are still valid 

•  Stakeholders  confirm 

of 

Survey with Programme Committee Members 
In-depth interviews with selected beneficiaries 
In-depth interviews with other key stakeholders 

Qualitative indicators 

•  Beneficiaries  being  able  to  implement  their 
projects or similar actions without the funding 
provided  by  the  programme  (beneficiaries’ 
counterfactual assessment) 

•  Examples  of  existing  alternative  funding 

schemes 

•  Examples of achievements of the programme 
that would not have been possible without the 
EU-level 
(stakeholders’ 
assessment) 

intervention 

•  Stakeholders’ assessment of the consequences 
of  potential  limitation  or  withdrawal  of  the 
programme 

80 

 
 
 
 
 
ANNEX IV. OVERVIEW OF BENEFITS AND COSTS, SIMPLIFICATION AND BURDEN REDUCTION 

Justice programme 2014-2020 

i. Overview of costs and benefits 

Citizens/consumers158 

Businesses 

Administration159  

Quantitative 

Comment 

Quantitative 

Comment 

Quantitative 

Comment 

Other programme 
beneficiaries160  

Comment 

Qu
anti
tati
ve 

N/A 

N/A 

Direct 
compliance 
costs 
(administrat
ive costs) 

One-off 
(applicatio
n 
procedure) 

N/A 

N/A 

Recurring 
(reporting 
requiremen
ts) 

Average cost of preparing the application 41.7 man-days which amounted to 
34%  of  the  total  administrative  costs  (defined  as  all  costs  excluding  project 
implementation)161. 
A significant proportion of surveyed beneficiaries considered the application 
process to be challenging: 

42% of public authorities and 50% of other beneficiaries highlighted 
in  the  online  survey  that  preparing  the  application  was  time-
consuming.  

Reporting costs (i.e. time and human resources spent to prepare progress and 
final reports) on average amounted to 53.6 man-days162 (approximately 45% 
of the total administrative costs)163. 
A  portion  of  beneficiaries  considered  complying  with  the  programme’s 
reporting  requirements  was  challenging  (e.g.  26%  of  public  authorities  and 
22% of other beneficiaries). 
It  is  important  to  note  that  these  costs  are  partially  covered  by  the  EU 
contributions, which public authorities and other beneficiaries received.  

158 Citizens and consumers include the general public as well as individuals who participated in activities of the projects funded under the programme. 
159 In this context public administrations may apply for funding and participate in the programme as beneficiaries. 
160 These stakeholders include beneficiaries of the programme except for public authorities. 
161 Based on estimates provided in the first part of the ex post evaluation. 
162 The calculation was derived from information available in the first part of the ex post evaluation. The formula used: (average man-days for preparing the application / proportion of application preparation in the total admin costs) 

x proportion of reporting costs in the total administrative costs. 

163 Other administrative costs reported by stakeholders included overheads, cost of external services, equipment costs and proposal preparation. 

81 

 
 
 
 
 
 
 
 
 
 
Indirect 
costs 

One-off 
(opportunit
y costs) 

No 
quantitative 
data available.  

N/A 

Opportunity 
costs 
of 
stakeholders 
participating 
in 
project 
activities 
opportunity  

N/A 

Recurring 
(opportunit
y costs) 

N/A 

Recurring 
(project 
coordinatio
n) 

Direct 
benefits 

Recurrent 

N/A 

N/A 

No quantitative data available. 
The direct benefits are 
estimated with qualitative 
information. 

N/A 

164 Costs in this context cover the period of 2021 until the 20 December 2023. 
165 Costs in this context cover the period of 2021 until the 20 December 2023. 
167 Based on stakeholder consultation and desk research. 

099 

No quantitative data available. 
Other  beneficiaries  incurred  opportunity 
costs associated with applying to the justice 
programme  and  implementing  projects  in 
case of success. 
These  opportunity  costs  also  include  the 
cost  of  co-funding  the  project.  For  the 
whole programme these costs ranged from 
EUR 12 197 008 to EUR 48 788 031 for 
grants  (co-funding  share  of  the  eligible 
costs for grants estimated at 5-20%.)165. 

No quantitative data available. 
Public 
incurred 
authorities 
opportunity  costs  associated 
with  applying  to  the  justice 
programme  and  implementing 
projects in case of success. 
These  opportunity  costs  also 
include  the  cost  of  co-funding 
the  project.  For 
the  whole 
programme  these  costs  ranged 
from  EUR 3 025 
to 
EUR 12 100  399  for  grants 
(co-funding share of the eligible 
costs  for  grants  estimated  at  5-
20%.)164. 
No quantitative data available. 
Public authorities funded under 
faced 
the 
programme 
to 
opportunity  costs  related 
(i.e. 
project 
time and Full Time Equivalents, 
FTEs)  and  benefits  that  could 
have  resulted  if  an  alternative 
action would have been taken. 
No quantitative data available. 
Coordination costs related to project implementation included: 

No quantitative data available. 
Other  beneficiaries  funded  under 
the 
programme faced opportunity costs related 
to  project  implementation  (i.e.  time  and 
FTEs  spent)  and  benefits  that  could  have 
resulted if an alternative action would have 
been taken. 

implementation 

• Implementation of the project activities 
• Coordination of the project consortium 
• Travel costs related to project implementation. 

No quantitative data available. 
Direct benefits for public authorities funded under the stemming from project 
implementation include167: 

•  Capacity-building through collaboration with project partners. 
• 

Integrating new experiences into daily practice 

82 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
• 

Increased visibility through networking and knowledge sharing 

Through building capacity and knowledge sharing, the beneficiaries 
contributed to facilitating judicial cooperation across the EU and to building 
mutual trust. Furthermore, by building their capacity and improving their 
daily practice, the beneficiaries contributed to promoting more effective 
application of EU law and protection of fundamental rights. 

Findings from the stakeholder 
consultation indicate 
participants experienced 
several benefits, including166: 

• Increased 

public 
rights, 
awareness  about 
principles 
values 
derived 
law 
(reported  by  65%  of 
beneficiaries); 

and 
from  EU 

skills 

• Increased 
and 
knowledge 
developed 
through the funded training 
activities  (reported  by  75% 
beneficiaries); and 

• More 

effective, 
comprehensive 
and 
consistent implementation 
of EU law (reported by 67% 
of beneficiaries). 

Through the funded projects 
fundamental rights were 
promoted, including through: 
• 

and 

equal 
Inclusivity 
access:  This  programme 
targeted 
and 
marginalised  groups,  69% 
focused  on 
of  projects 
inclusivity. 

diverse 

•  Awareness raising efforts: 
This  programme  showed  a 
focus  on  integrating  anti-
discrimination 
and 
awareness raising principles 
and 
into 
training 
Awareness 
campaigns. 

166 Based on feedback provided by beneficiaries in the online survey. 

83 

 
 
 
 
 
 
 
•  Support 

modules  were  included  in 
54% of 2014-2020 projects. 
for  vulnerable 
groups:  The  2014-2020 
period laid the groundwork 
for 
support 
services,  with  54%  of 
these 
projects  addressing 
needs. 

inclusive 

Indirect 
benefits 

Recurrent 

previous 

The quantification of indirect 
benefits is not available. 
justice 
The 
programme reached its general 
and  specific  objectives,  which 
translated  in  various  benefits 
for the public, including: 

• Better access to justice (e.g. 
improved support for victims 
of  crime,  better  access  to 
alternative 
courts 
methods 
dispute 
resolution) 

and 

for 

• Improved 

recognition 
decisions 
to 

judicial 
facilitated 
cooperation 
of 
mutual 
and 
judicial 
a  more 
contributed 
application  of 
consistent 
Union 
in  civil  and 
criminal  matters.  These 
effects  may  have  given 
citizens greater confidence in 
support 
legal 
accessing 
across different jurisdictions; 
and 

law 

N/A 

No quantitative information available. 

Indirect benefits reported by public authorities and other types of 
beneficiaries include formation of long-term partnerships (reported by 77% 
of beneficiaries) and up-take of digital tools (mentioned by 29% of 
beneficiaries). 

Formation  of  long-term  partnerships  was  accomplished  through  networking 
activities  and  knowledge  sharing  which  took  place  during  the  project 
implementation. Establishment of partnerships contributed to: 

• 

• 

• 

building mutual trust between judicial actors, 

enhancing the implementation and application of the Rule of Law and 

facilitating the protection of fundamental rights. 

Project  implementation  (especially  during  the  COVID-19 pandemic)  further 
familiarised  the  beneficiaries  with  available  digital  tools  such  as  websites, 
apps, and e-learning platforms. Implementation of various digital tools during 
the projects also contributed to the digitalisation of work in the long term. This, 
in turn, facilitated improvements of the IT infrastructure and interoperability 
of IT systems across the EU. 

• Better  protection  under  EU 
law 
through 
facilitated 
training and mutual learning 

84 

 
 
activities which helped legal 
practitioners develop a better 
understanding and awareness 
of EU legislation. 

Cost saving 

N/A 

N/A 

N/A 

N/A 

Recurring 
(Digitalisat
ion  of  the 
application 
procedure) 
Recurring 
(Up-take of 
digital 
tools 
project 
implement
ation) 

for 

Recurring 
(Extended 
duration  of 
AGs) 
Recurring 
(Transition 
4-year 
to 
FPAs170 for 
OGs) 

N/A 

N/A 

N/A 

N/A 

168 Based on the results of the text-mining exercise which involved an analysis of a representative sample of 62 projects. 
169 According to the findings of the first ex post evaluation. 
170 Framework Partnership Agreements. 
171 According to the findings of the first part ex post evaluation. 
172 According to the Annual Work Programmes. 
173 Based on the programme data (2014-2020). 

85 

No quantitative data is available. 

Introduction of eGrants has produced significant cost savings for all types of 
beneficiaries participating in the programme. 

No quantitative data available. 

Due to the restrictions imposed by the COVID-19 pandemic, in-person events 
shifted online. According to the findings, this affected around 40% of the 
projects168. Beneficiaries of these projects reported cost saving for travel and 
accommodation. However, it is challenging to estimate the exact effect the 
pandemic had on travel cost savings since it also caused delays and 
deviations which are harder to precisely quantify.  

No quantitative data is available. 

Extending  the  duration  of  AGs  from  24  to  at  least  30  months  enabled 
beneficiaries to better develop their projects169. 

No quantitative data is available. 

Introduction of 4-year FPAs and related annual operating grants increased the 
capacity of beneficiaries and enabled them to plan their activities and pursue 
funding opportunities within a more comfortable timeframe171. 
Two calls for 4-year FPAs were launched in 2017 covering both the JCOO and 
JACC objectives172. Under these calls 13 partnership agreements were signed 
7 under the JACC objective, and 6 under the JCOO objective173. 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
ii. Potential simplification and burden reduction 

Citizens/consumers 

Businesses 

Administration 

Other programme beneficiaries 

Quantitative 

Comment 

Quantitative 

Comment 

Quantitative 

Comment 

Quantitative 

Comment 

N/A 

N/A 

No quantitative 
data is available. 

Simplification 
of the 
application 
procedure, 
reporting 
requirements 
and financial 
rules 

N/A 

Improvements 
of the EU 
Funding & 
Tenders 
portal 

N/A 

No quantitative 
data is available. 

86 

Participation in 
the programme 
involved 
significant 
administrative 
burdens for 
public 
authorities 
applying for 
funding or 
implementing 
projects. There is 
potential to 
streamline these 
processes 
through 
simplification 
measures. 

Public 
authorities 
mentioned 
experiencing 
challenges when 
using the EU 
Funding & 
Tenders portal. 
Potential for cost 
savings by 

No quantitative 
data is available. 

Participation in the 
programme also involved 
significant administrative 
burdens for other types of 
beneficiaries indicating 
they could have also 
benefited form more 
streamlined procedures.  

No quantitative 
data is available. 

Other types of 
beneficiaries also 
reported experiencing 
challenges with the 
Funding & Tenders 
portal. Potential for cost 
savings by making the 
portal more user-friendly. 

 
 
 
 
 
 
 
 
 
 
making the 
portal more user-
friendly. 

87 

 
Justice programme 2021-2027 

i. Overview of costs and benefits 

Citizens/consumers174 

Businesses 

Administration175 

Other programme beneficiaries176 

Quantitat
ive 

N/A 

Direct 
compliance 
costs 
(administrati
ve costs) 

One-off 
(application 
procedure) 

One 
(reporting) 

off 

No 
quantitativ
e 
data 
available.  

Comment 

Quantitative 

Comment 

Quantitative 

Comment 

Quantitative 

Comment 

N/A 

Cost of preparing the application: 
No quantitative data available. 
A  high  proportion  of  surveyed  beneficiaries  considered  the  application 
process to be time-consuming: 

90%  of  public  authorities177  and  88%  of  other  beneficiaries178 
highlighted in the online survey that preparing the application was 
time-consuming). 

N/A 

N/A 

N/A 

Reporting  costs 
(i.e.  time  spent 
by  participants 
in 
project 
activities 
providing 
feedback  -  e.g. 
EU  Survey  on 
Justice,  Rights 
and Values 

Recurring 
(reporting 
requirements) 

N/A 

N/A 

Reporting costs (i.e. time and human resources spent to prepare progress 
and final reports): 
No quantitative data available. 
Beneficiaries mentioned a significant amount of resources was invested to 
comply with the programme’s reporting requirements (e.g. 80% of public 

174 Citizens and consumers include the general public as well as individuals who participated in activities of the projects funded under the programme. 
175 In this context public administrations may apply for funding and participate in the programme as beneficiaries. 
176 These stakeholders include beneficiaries of the programme except for public authorities. 
177 Represents those who agreed fully (70%) and agreed to some extent (20%). 10% disagreed. 
178 Represents those who agreed fully (65%) and those who agree to some extent (23%). 13% disagreed. 

88 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Indirect costs 

Recurring 
(opportunity 
costs) 

N/A 

N/A 

Recurring 
(project 
coordination) 

N/A 

Direct 
benefits 

One-off 
(budget 
committed)182 

N/A 

N/A 

N/A 

authorities179  and  63%  of  other  beneficiaries180  highlighted  in  the  online 
survey  reporting  requirements  were  resource-intensive).  Half  of  the 
beneficiaries  highlighted  that  a  high  amount  of  human  resources  was 
needed to comply with the reporting requirements181. 
It  is  important  to  note  that  these  costs  are  partially  covered  by  the  EU 
contributions, which public authorities and other beneficiaries received.  

No quantitative data available. 
Other  beneficiaries  funded  under 
the  programme  faced  opportunity 
costs 
project 
implementation (i.e. time and FTEs 
spent) and benefits that could have 
resulted  if  an  alternative  action 
would have been taken. 

related 

to 

No quantitative data available. 
Public authorities funded under the 
programme faced opportunity costs 
related  to  project  implementation 
(i.e.  time  and  FTEs  spent)  and 
benefits  that  could  have  resulted  if 
an  alternative  action  would  have 
been taken. 
No quantitative data available. 
Coordination costs: 
• Implementation of the project activities 
• Coordination of the project consortium 
• Travel costs related to project implementation. 
Budget 
EUR 25 821 
committed for 
817 
public 
authorities until 
December 2023. 

EUR 68 578 
280 

Budget 
committed for 
other 
beneficiaries 
excluding 
public 
authorities until 
December 
2023. 

Recurrent 

No quantitative data 
available. The direct benefits 
are estimated with qualitative 
information. 

N/A 

No quantitative data available. 

Thus far, direct benefits for public authorities funded under the stemming 
from project implementation include185: 

179 Represents those who agreed fully (40%) and those who agreed to some extent (40%). 20% disagreed. 
180 Represents those who agreed fully (40%) and those who agreed to some extent (40%). 20% disagreed. 
181 Feedback provided during the in-depth interviews. 
182 This estimate is based on the following: % share of the budget committed by beneficiary type was based on analysis of grants funded in 2014-2023. This % share was then calculated over the total amount committed in the period 

2014-2023. 

185 Based on stakeholder consultation and desk research. 

89 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
•  Capacity-building through collaboration with project partners. 
•  Development of ICT and other tools 
• 

Increased visibility through networking and knowledge sharing 

Through building capacity and knowledge sharing, the beneficiaries 
contributed to building effective judiciary, facilitating judicial cooperation 
across the EU and to building mutual trust. Furthermore, by building their 
capacity and improving their daily practice, the beneficiaries contributed 
to better enforcement of the Rule of Law and protection of fundamental 
rights. 

Through the funded projects 
approximately 850 events and 
activities were organised. 
These activities reached 
approximately 2 500 000 
individuals, including 1 250 
000 women. 

Findings from the stakeholder 
consultation indicate 
participants experienced 
several benefits, including183: 
• Increased  awareness  of 
topics covered which was 
reported 
92% 
participants; 

by 

• Increased  knowledge  of 
topics  covered  which  was 
reported  by  91%  of 
participants; 

• Increased 

skills 

mentioned by 76%; and 

• Enhanced 

cooperation 
experienced  by  83%  of 
participants. 

Through the funded projects 
fundamental rights were 
promoted, including 
through184: 

• 

Inclusivity  and  equal 
access:  35%  of  projects 
have  been  dedicated  to 
ensuring  that  all  legal 
professionals,  regardless 
have 
of 
latest 
access 

background, 
the 
to 

183 Based on feedback collected through the EU survey on Justice, Rights and Values. 
184 Based on findings from interviews and text mining of project documents. 

90 

 
 
 
 
 
 
 
technological  tools  and 
training opportunities. 

•  Awareness  raising  and 
sensitisation efforts: 32% 
of  projects 
incorporated 
anti-discrimination 
training,  whilst  24%  of 
launched 
projects  have 
public campaigns. 

•  Support  for  vulnerable 
groups:  32%  of  projects 
the  needs  of 
addressed 
migrants, refugees, LGBTI 
individuals,  and  persons 
with 
disabilities.  The 
programme  demonstrates 
an 
to 
improved  ability 
mainstream 
gender 
equality,  with  projects 
increasing  the  capacity  of 
to  support 
practitioners 
victims  with 
gender-
sensitive approaches. 
•  Rights  of  the  child:  A 
protecting 
on 
focus 
children’s  rights  in  legal 
proceedings,  with  13%  of 
projects 
ensuring 
adherence to EU standards 
for  child-friendly 
legal 
procedures. 

The quantification of indirect 
benefits is not available. 
The  justice  programme  has 
made  significant  progress  in 
reaching 
its  general  and 
specific objectives, which has 
manifested in various benefits 

Indirect 
benefits 

Recurrent 

N/A 

No quantitative information available. 

Indirect benefits reported by public authorities and other types of 
beneficiaries thus far include formation of long-term partnerships 
(reported by 92% of beneficiaries), fostering gender equality (36% of 

91 

 
 
for 
the 
including: 

general 

public, 

• Better  access  to  justice 
(e.g.  improved  support  for 
victims  of  crime,  better 
and 
access 
courts 
to 
for 
alternative  methods 
dispute resolution) 

• Improved 

decisions 

judicial 
cooperation, 
the 
facilitates 
programme 
mutual  recognition  of  and 
judicial 
and 
contributed 
to  a  more 
consistent  application  of 
Union  law  in  civil  and 
criminal  matters.  These 
effects  may  have  given 
citizens  greater  confidence 
in  accessing  legal  support 
across 
different 
jurisdictions; and 

projects)186 and up-take of digital tools (mentioned by 29% of 
beneficiaries). 

through 
Formation  of 
networking activities and knowledge sharing which took place during the 
project implementation. Establishment of partnerships contributed to: 

long-term  partnerships  was  accomplished 

• 

• 

• 

building mutual trust between judicial actors, 

enhancing the implementation and application of Union law and 

facilitating the protection of fundamental rights. 

Project  implementation  (especially  during  the  COVID-19  pandemic) 
further  familiarised  the  beneficiaries  with  available  digital  tools  such  as 
websites, apps, and e-learning platforms. Implementation of various digital 
tools during the projects also contributed to the digitalisation of work in the 
long term. This, in turn facilitated improvements of the IT infrastructure 
and interoperability of IT systems across the EU. 

• Better protection under EU 
through 
facilitated 
law 
training 
and  mutual 
learning  activities  which 
legal  practitioners 
help 
better 
develop 
and 
understanding 
awareness 
EU 
legislation. 

of 

a 

Cost saving 

N/A 

Recurrent 
(Simplification 
– 
measures 
sum 
lump 

N/A 

Introduction of lump sum budgets and the unit costs for travel and 
accommodation have contributed to simplifying the reporting 
requirements and the application procedure. 

186 According to the findings, 36% of projects received a score of 2 (3%) or a score of 1 (33%), which was an improvement since the previous funding period in which 15% of projects received a score of 1 or 2. For more details, see 
Case study 1 in Annex 2. 

92 

 
 
 
 
 
budgets 
standard 
costs) 

and 
unit 

N/A 

Recurrent 
(Simplification 
of  programme 
management) 

N/A 

Recurring  (Up-
take  of  digital 
tools for project 
implementation 

N/A 

N/A 

No quantitative data available, however consultation with both public 
authorities and other beneficiaries indicates that these measures 
contributed to reducing administrative burdens. However, beneficiaries do 
not master yet the new rules which increased the time needed by the 
beneficiaries to prepare their application forms. 

Since the new measures were introduced recently, it is still too early to fully 
assess their effect on administrative burdens. 

Simplification of programme procedures contributed to a reduction in the 
indicators time-to-inform, time-to-sign and time-to-grant. Although no 
quantitative estimations are available, these improvements have benefited 
both public authorities and other types of beneficiaries in terms of time 
savings. 

No quantitative data available. 

Due to the restrictions imposed by the COVID-19 pandemic, in-person 
events shifted online. According to the findings, this affected around 40% 
of the projects187. Beneficiaries of these projects reported cost saving for 
travel and accommodation. However, it is challenging to estimate the 
exact effect the pandemic had on travel cost savings since it also caused 
delays and deviations which are harder to precisely quantify.  

ii. Potential simplification and burden reduction 

Citizens/consumers 

Businesses 

Administration  

Other programme beneficiaries 

Quantitative 

Comment 

Quantitative 

Comment 

Quantitative 

Comment 

Quantitative 

Comment 

N/A 

N/A 

No quantification is available. Consulted beneficiaries highlighted more guidance on 
the use of unit costs and lump sum budgets would help further reduce administrative 
burdens. 

Guidance on 
the 
application 
process and 
reporting 
requirements 

187 Based on the results of the text-mining exercise which involved an analysis a representative sample of 62 projects (see Annex 11 for more details). 

93 

 
 
 
 
 
 
 
 
 
 
 
 
 
N/A 

Guidance on 
the use of the 
EU Funding 
& Tenders 
portal 

N/A 

No quantitative data is available, however a significant portion of stakeholders raised 
concerns about the user-friendliness of the EU Funding & Tenders portal. More 
guidance on how to use the portal could further reduce administrative burdens. 

94 

 
 
 
 
ANNEX V. STAKEHOLDERS CONSULTATION - SYNOPSIS REPORT 

Objectives of the consultation 

The 2014-2020 and 2021-2027 justice programmes are two consecutive funding programmes contributing to the 
development of a European area of justice based on the rule of law. As required by law188, the European Commission 
conducted an evaluation in two parts: (1) an ex post evaluation of the 2014-2020 justice programme, assessing its 
longer-term impact and sustainability effects, based on the results of the first part189 of the evaluation finalised in 
2022; and (2) an interim evaluation of the 2021-2027 justice programme, assessing its preliminary achievements. 
The combined ex post and interim evaluation of the two programmes aimed to identify how the current programme 
took account of lessons learnt under its predecessor, and how results can be used for further improvement and to 
feed into the next funding cycle. 

The  purpose  of  the  consultation  was  to  strengthen  the  evidence  base  by  gathering  the  views  of  all  relevant 
stakeholders  on  the  justice  programmes’  effectiveness,  efficiency,  relevance,  coherence,  and  EU  added  value. 
Feedback was collected from a wide range of stakeholders including beneficiaries of both programmes, unsuccessful 
applicants,  programme  committee  members  (PCMs),  civil  society  organisations,  public  administrations,  and  the 
public. 

Consultation activities and tools 

The  consultation  included  three  surveys  (with  beneficiaries,  unsuccessful  applicant  and  PCMs),  follow-up 
interviews with a sample of stakeholders (representing beneficiaries and PCMs), an open public consultation, and a 
call for evidence. 

For the online surveys, three separate sets of questions were developed to ensure a clear distinction between the 
evaluations of the two programmes. Since Programme Committee Members had already provided feedback on the 
2014-2020 justice programme during the first part of the  ex post evaluation, their survey focused on the  current 
edition.  The  beneficiaries’  and  unsuccessful  applicants’  surveys  covered  both  iterations  of  the  programme.  The 
surveys were launched on 29 March and closed on 24 May. 

To maximise response rates, personalised email invitations were sent, followed by seven weekly reminders. The 
surveys were  conducted using the Qualtrics platform. The data  analysis was performed after the surveys closed, 
separated  by  funding  period  and  stakeholder  category,  incorporating  both  quantitative  (closed  responses)  and 
qualitative (open-ended feedback) insights. 

The  interviews  were  conducted  between  April  11  and  July  31  with  beneficiaries  and  Programme  Committee 
Members. The aim of the interviews was to gather more in-depth views from these key stakeholder groups about 
their experiences with the justice programmes. The interviews followed a semi-structured guide, covering topics 
such as programme effectiveness, efficiency, relevance, added value, the impact of COVID-19, and digitalisation. 
The  discussion  also  included  topics  such  as  gender  mainstreaming  and  simplification,  which  were  also  further 
analysed through case studies. 

A Call for Evidence and an Open Public Consultation (OPC) were launched by the European Commission: 

•  The call for evidence was launched on 22 August 2023 until 19 September 2023. 

•  The public consultation run between 3 April 2024 and 28 June 2024. 

The  consultations  were  published  on  the  Commission’s  Have  your  Say  platform.  The  main  objective  of  these 
consultations  was  to  strengthen  the  evidence  base  of  the  evaluation  by  gathering  the  views  of  all  relevant 
stakeholders for the justice programmes. However, only eight responses were collected through the OPC – from two 
EU  citizens,  five  organisations  and  one  national  authority  –  across  seven  Member  States  (i.e.  Austria,  France, 
Germany, Greece, Lithuania, the Netherlands and 2 from Spain). Due to this low response rate, the findings of the 
OPC  are  not  considered  to  be  a  representative  sample  of  the  EU  population.  As  such,  while  considered  when 
triangulating the data and drafting the findings for the evaluation, care was taken not to artificially inflate the findings 
stemming from the analysis of the OPC. 

188 Regulation (EU) No 1382/2013 of 17 December 2013 and Regulation (EU) No 2021/693 of 28 April 2021. 
189 COM/2022/121 final 

95 

 
 
 
Overview on the number of stakeholders consulted by different consultation activities is available in the table below. 

Table 1: Summary of the types of activities and feedback received190 

Beneficiaries 

Unsuccessful 
applicants 

PCMs 

Other191 

Survey 

Contacted 

Responses 

Interviews 

Contacted 

OPC 

Call for 
evidence 

Responses 

Contacted 

Responses 

Contacted 

Responses 

274 

109 

265 

33 

N/A 

0 

N/A 

0 

454 

79 

N/A 

N/A 

N/A 

0 

N/A 

0 

78 

27 

35 

5 

N/A 

0 

N/A 

0 

N/A 

N/A 

N/A 

N/A 

N/A 

8 

N/A 

6 

Main stakeholder feedback per consultation activity 

This  section  presents  the  results  of  the  stakeholder  consultation.  The  stakeholder  feedback  is  presented  in  four 
different sections including survey results, interview feedback as well as responses to the open public consultation 
and the call for evidence. 

Surveys 

The survey results are presented in two sections: 

•  Surveys for the beneficiaries and unsuccessful applicants 
•  Survey for the Programme Committee Members 

Survey results are categorised by evaluation criteria (i.e. effectiveness, efficiency, relevance, coherence, and EU 
added value). 

Survey of beneficiaries and unsuccessful applicants 

Effectiveness 

Contributions to programme objectives: In the 2014-2020 period, beneficiaries believed their projects contributed 
to key objectives such as training stakeholders (75%), improving EU law implementation (67%), increasing public 
awareness of EU rights and values (65%), and building mutual trust among Member States (62%). By the 2021-
2027 programming period, the perceived contributions to the programme’s objectives had become more widespread. 
While the same key objectives remained important, beneficiaries reported even higher contributions to improving 
access  to  civil  society  organisations’  (CSO)  services  (46%)  and  enhancing  cross-border  system  interoperability 
(46%), showing progress in areas that had previously been considered more challenging. 

Application  process:  Both  successful  and  unsuccessful  applicants  had  similar  experiences  with  the  application 
process, with feedback suggesting it was a time-consuming task (reported by 91% of applicants for the previous 
programme and 89% for the current programme). A significant percentage of respondents found the administrative 
requirements  burdensome  –  74%  in  the  2014-2020  period  and  71%  in  the  2021-2027  period.  Despite  these 
challenges,  the  majority  of  respondents  appreciated  that  the  conditions  for  receiving  support  were  generally 
transparent, with roughly 85% agreeing that they had adequate information about funding opportunities in advance. 

190 In addition to consulting with external stakeholders, the consultation also involved interviews and an internal validation workshop with staff 
from DG Justice and Consumers (JUST). 
191 The public, civil society organisations and national authorities. 

96 

 
 
 
 
 
Sustainability  of  project  outputs  beyond  the  funding  period:  Most  beneficiaries  across  both  programming 
periods believed their project results would last beyond the project timeframe (reported by 93% of respondents from 
the  previous  and  94%  from  the  current  programme).  Sustainability  was  primarily  linked  to  network-building, 
capacity-building, and the development of IT infrastructure192. 

Information about the justice programme: Across both programming periods, the majority of stakeholders found 
out about the  programme through the  European Commission’s website, followed by communication with peers, 
colleagues,  or  other  networks.  Interestingly,  the  2021-2027  period  saw  a  decline  in  the  number  of  stakeholders 
learning about the programme through informal channels like peer recommendations, which decreased from 27.2% 
to 17.3%. Communication from national, regional, or local governments also decreased significantly. Nevertheless, 
stakeholders generally felt that the programme was well-known among potential applicants (reported by roughly 
70%). 

Efficiency 

Cost-benefit  ratio:  Across  both  programming  periods,  the  majority  of  beneficiaries  felt  that  the  benefits  of 
participation outweighed the costs (75% of beneficiaries from the 2014-2020 period and 72% from the 2021-2027 
period). Despite the positive feedback, differences were observed between stakeholder types. Public authorities, for 
example, were most likely to report that the costs outweighed the benefits. 

External challenges: A significant factor affecting the efficiency of projects funded by the justice programme was 
the  impact  of  external  challenges,  most  notably  the  COVID-19  pandemic  (identified  by  36%  in  the  previous 
programme  and  34%  in  the  current  programme).  The  pandemic  caused  delays  in  the  implementation  of  some 
projects, particularly those involving international cooperation, seminars, and conferences. As these activities were 
postponed or moved online, many beneficiaries faced difficulties adapting to the new formats, which in turn required 
additional  resources  and  time193.  Other  external  challenges  mentioned  by  beneficiaries  included  economic  and 
political factors. For example, 11% of respondents in the 2014-2020 period and 5% in the 2021-2027 period cited 
economic  issues,  such  as  national  budget  constraints  or  changing  funding  priorities,  as  a  limiting  factor. 
Additionally, political instability was raised as a concern by 6% of respondents in the previous programme and 16% 
in the current programme. 

Administrative  processes:  Efficiency  in  administrative  processes  was  a  concern  for beneficiaries,  with  49%  of 
respondents reporting issues related to programme requirements, especially reporting and monitoring. Small and 
medium-sized  NGOs  struggled  the  most  due  to  limited  staff,  while  64%  of  public  authorities  and  60%  of 
international organisations reported administrative challenges.  In contrast, 60% of CSOs and NGOs and 51% of 
higher education institutions faced no issues. Improvements were noted by 42% of beneficiaries, with 74% viewing 
lump-sum contributions positively, and 69% welcoming the e-Grants system. However, 38% found the standardised 
cost unit system complex, especially public authorities managing cross-jurisdictional projects. Still, 78% felt the 
changes simplified the funding process. 

Relevance 

When asked whether the objectives of the justice programme aligned with the needs of the justice sector in their 
country,  beneficiaries  from  both  programming  periods  responded  positively.  Specifically,  79.4%  of  respondents 
from  the  2014-2020  period  and  87.5%  from  the  2021-2027  period  agreed  that  the  programme’s  objectives 
corresponded  well  to  sector  needs.  Nonetheless,  some  discrepancies  emerged  when  disaggregating  the  data  by 
organisation  type.  For  instance,  a  minority  of  higher  education  institutions,  CSOs  and  NGOs  felt  that  the 
programme’s objectives did not fully align with their needs. Additionally, the programme’s alignment with the needs 
of target groups was largely positive. Only a small number of public authorities (8%) expressed concerns about the 
relevance of project outputs to their specific target groups. 

Coherence and EU added value 

A key measure of the programme’s added value is whether beneficiaries could have secured funding from alternative 
sources if the justice programme were unavailable. Only 7% of respondents from the 2014-2020 period and 8% from 
the 2021-2027 period believed they could have accessed other EU financial instruments. Similarly, only 9%-10% 
thought they could have obtained funding from national sources. These figures underline the unique role the justice 
programme plays in supporting projects that might otherwise struggle to secure funding. 

192 For example, the creation of cross-border cooperation networks supported the day-to-day functioning of justice systems, while IT infrastructure 
improvements facilitated the multiplication of project outcomes, such as better training dissemination and more efficient communication. 
193 For instance, some projects designed around in-person judicial training or cross-border legal cooperation had to be restructured, reducing the 
overall impact of their outputs. 

97 

 
 
 
Survey of the Programme Committee Members 

Effectiveness 

Achievements of the current programme: Out of the 58% of respondents who had an opinion, all of them agreed 
with  the  statement  that  the  current  programme  is  on  course  to  achieving  its  objectives.  Programme  Committee 
Members (PCMs) believed that the current programme is significantly contributing to supporting and promoting 
judicial  training  (81%),  followed  by  facilitating  and  supporting  judicial  cooperation  (70%).  With  regards  to 
facilitating  effective  and  non-discriminatory  access  to  justice,  half  (50%)  of  respondents  believed  that  the 
programme is having a significant contribution, while 38% believed that it is having no contribution to this objective 
at all. 

Current programme’s performance: PCMs found that the justice programme managed to target the most relevant 
groups and policy areas (76% for each) and to meet its objectives (68%). However, fewer respondents (52%) thought 
the current programme contributed to simplifying procedures (including by reduction of overlaps, and duplications 
in requirements and necessary tasks). 

Sustainability of the programme: The vast majority of PCMs believed that the current programme’s results will 
be long lasting, including all the results listed in the survey,  i.e. the creation of tools and procedures to be used 
beyond project completion (96%), partnerships that will continue operating beyond the funding period (92%), the 
definition of training and other activities that last or will last beyond project completion (92%), acquisition of new 
skills and knowledge (92%), increased awareness (88%). 

Key  outcomes  of  the  EU  justice  programme:  The  majority  of  PCMs  believed  that  the  justice  programme 
contributes  to:  (i)  increasing  mutual  trust  and  cross-border  cooperation  (96%),  (ii)  enhancing  cross-border 
interoperability  of  systems  (96%),  and  (iii)  training  stakeholders  on  Union  law  and  judicial  instruments  (92%). 
Fewer respondents believed that the programme focuses on: (i) improving the implementation of Union law (88%), 
(ii) raising public awareness of rights and principles under Union law (83%), and (iii) improving citizens’ access to 
civil society services (82%). 

Visibility of the programme: Regarding awareness among potential applicants, 36% of respondents found that the 
justice programme is well known in their country among eligible organisations. However, 48% believed that the 
programme is not well known among all eligible organisations and 16% of respondents thought that the programme 
is not known at all among the eligible organisations. 

Barriers  to  applying  for  justice  programme  funding:  Regarding  the  reasons  why  certain  organisations  are 
reluctant  to  apply  for  funding  from  the  justice  programme,  respondents  found  that  the main  reasons  are:  (i)  the 
complexity  of  the  application  process  (33%),  (ii)  lack  of  awareness  (16%),  (iii)  language  barriers  (14%),  (iv) 
insufficient support for applicants (13%), and (v) perceived low success rates (10%). 

Efficiency 

PCMs  saw  efficiency  improvements  but  noted  ongoing  challenges.  The  standardised  unit  costs  caused  issues, 
especially as inflation raised travel and accommodation costs. The EUR 350 000 cap on criminal justice projects 
limited  their  scope,  restricting  cross-border  involvement.  Some  PCMs  found  the  application  process  costly.  A 
majority (52%) felt the 2021-2027 programme’s benefits matched its costs, 40% said benefits outweighed costs, and 
8% felt costs were higher. 

Relevance 

Programme’s  relevance  to  current  needs  was  highly  rated  by  most  PCMs,  with  67%  indicating  a  high  level  of 
responsiveness  and  only  8%  reporting  low  responsiveness.  Several  respondents  highlighted  the  importance  of 
continued funding for e-Justice and other digital infrastructure projects, reflecting the increasing role of technology 
in modernising the justice system. 

Coherence and EU added value 

Half of the PCMs surveyed felt that the justice programme was consistent with national policies in the justice sector, 
while 46% expressed no strong opinion on the matter. The programme was viewed as particularly coherent with the 
EU Charter of Fundamental Rights and the European Agenda for Justice, aligning with broader EU objectives aimed 
at promoting judicial cooperation and ensuring access to justice for all. 

As for EU added value, over 56% of PCMs believed that the outcomes achieved through the justice programme 
could not have been replicated by Member State’s action alone. They believed this was especially true in smaller 
Member States, where resources for undertaking similar projects without EU support would have been insufficient. 

98 

 
 
Interviews 

Interviews with the beneficiaries 

Effectiveness 

Successes  and  positive  outcomes:  Many  beneficiaries  reported  their  projects  successfully  met  or  exceeded 
objectives, particularly improving cooperation between organisations from different EU Member States in justice 
and probation services. Projects enhanced best practice sharing, interagency cooperation, data sharing, and trust on 
issues like radicalisation and extremism. Probation services benefited from shared knowledge, improving offender 
rehabilitation  and  promoting  alternatives  to  pre-trial  detention,  reducing  prison  populations.  Beneficiaries  noted 
these efforts supported criminal justice systems and led to a safer Europe by addressing social issues. They also 
praised  training  guides  and manuals,  which  continue  to  professionalise  probation  and  justice  systems  and  foster 
collaboration between court and community services. 

Challenges encountered: While overall feedback was positive, some beneficiaries mentioned challenges affecting 
project progress. External factors like legislative and political changes caused delays or required adjustments. The 
COVID-19  pandemic  was  particularly  disruptive,  postponing  face-to-face  meetings  and  training  events.  Travel 
restrictions delayed activities, though this did not hinder progress overall. The shift to online training and hybrid 
events  effectively  addressed  pandemic  challenges,  with  beneficiaries  praising  the  Commission’s  flexibility. 
Additionally, engaging participants, especially judges, proved difficult due to their busy schedules, highlighting the 
need for more flexible, remote participation options. 

Unintended  and  long-term  effects:  Several  beneficiaries  noted  positive  unintended  outcomes,  such  as  deeper 
cooperation within consortia and stronger EU partnerships which were not originally envisaged. Many highlighted 
the creation of long-term professional networks that continue to deliver benefits. For example, a juvenile justice 
project produced practical guides still used in detention centres, shaping best practices and improving services for 
offenders and victims. The development of models and training tools has also ensured that project outcomes remain 
valuable for future initiatives. 

Sustainability and future potential: Many beneficiaries expressed confidence in the projects’ sustainability, noting 
that tools, methodologies, and materials will continue to support Europe’s justice systems. Training programmes for 
legal  professionals  and  probation  officers  were  frequently  cited  as  long-lasting  resources.  Beneficiaries  also 
highlighted  the  projects’  potential  to  influence  future  policies,  laying  the  groundwork  for  reforms  in  areas  like 
victims’ rights and juvenile justice. They were optimistic that the outcomes could be integrated into wider European 
frameworks, promoting better standards continent-wide. 

External factors and programme design: While beneficiaries were generally pleased with the outcomes, several 
noted external factors that had affected project implementation. Aside from COVID-19, which was seen as the most 
significant  challenge,  beneficiaries  mentioned  inflation  related  to  global  events  like  Russia’s  war  of  aggression 
against Ukraine. These external issues led to increased costs and budgetary constraints, but beneficiaries appreciated 
the European Commission’s flexibility in allowing budget adjustments where necessary. 

Overall,  beneficiaries  found  no  major  flaws  in  the  programme  design  but  suggested  more  flexibility  in  budget 
reallocation and planning to address unforeseen changes. Many also desired longer project durations, noting that 
two years was not enough for full implementation and evaluation. Longer timeframes, they noted, would allow for 
better advocacy, follow-up, and measurement of long-term impact. 

Efficiency 

Cost  efficiency  and  allocation:  Beneficiaries  reported  that  the  greatest  programme  costs  were  tied  to  human 
resources,  especially  for  preparing  applications,  implementing  projects,  and  reporting.  Managing  complex 
application  systems  and  meeting  reporting  requirements  also  added  financial  strain.  One  organisation  noted 
difficulties in accurately calculating budgets, particularly for research projects, with smaller institutions facing co-
funding  gaps  of  up  to  20%.  Larger  institutions  handled  this  better  using  national  research  funding.  While 
beneficiaries appreciated salary support, many felt administrative tasks exceeded the funding provided. One noted 
that  despite  partial  salary  coverage,  the  burdensome  paperwork,  including  documenting  hours  and  signatures, 
outweighed the financial assistance received. 

Cost-benefit  ratio:  Most beneficiaries  felt  the  benefits  of participating  in  the  programme  outweighed  the  costs. 
Many highlighted the long-term value of networking and professional relationships established through the projects. 
One participant noted that the programme’s main advantage was fostering collaboration with European partners, 
increasing  visibility,  and  advancing  work  in  areas  like  restorative  justice  and  victims’  rights.  Practical  benefits 
included  access  to  training  resources  and  knowledge  exchange  with  international  counterparts.  For  some 
organisations, these intangible benefits were more valuable than financial gains, with one noting that the exposure 

99 

 
gained  far  exceeded  financial  costs.  However,  a  few  beneficiaries,  particularly  smaller  organisations,  expressed 
frustration  with  co-funding  requirements,  limiting  their  participation  due  to  challenges  in  securing  additional 
funding. 

Administrative  burden  and  challenges:  A  recurring  theme  in  feedback  was  the  administrative  burden  of  the 
programme,  especially  during  applications  and  final  reporting.  Beneficiaries  highlighted  the  complexity  and 
bureaucracy  involved  in  financial  reporting,  with  one  participant  noting  the  excessive  paperwork  and  multiple 
signatures  required  for  salary  and  time  reporting.  Others  encountered  difficulties  with  the  EU’s  online  portals, 
including login issues and inconsistent guidance from project officers. Many expressed a desire for greater flexibility 
in project timelines and reporting schedules, especially during unexpected delays, such as when deadlines coincided 
with holidays. They also suggested that project design should account for potential delays due to late funding. 

Suggestions  for  improvement:  Beneficiaries  provided  several  suggestions  for  improving  the  efficiency  of  the 
programme.  A  common  recommendation  was  to  simplify  reporting  requirements,  especially  for  financial 
documentation, by reducing paperwork  and increasing the use of digital systems. They also advocated for more 
flexible  budgeting  rules  that  would  allow  fund  reallocation  without  a  full  project  amendment,  saving  time  and 
reducing administrative burden. Another key recommendation was the introduction of a lump-sum budget model, 
which  beneficiaries  believed  would  alleviate  administrative  complexity  in  tracking  expenses.  Additionally, 
beneficiaries recommended greater flexibility in project  timelines, including start dates and reporting periods, to 
better align activities with operational schedules and avoid rushed deadlines, particularly due to funding delays. 

Relevance 

Feedback from programme beneficiaries indicated a largely positive view of the justice programme’s objectives, 
which align well with their needs. Some beneficiaries suggested the programme could better address specific needs 
by  focusing  more  on  emerging  issues,  like  digitalisation  and  Artificial  Intelligence  (AI),  while  also  supporting 
unresolved challenges, such as those related to child justice. There was a call for a better balance between new and 
ongoing  issues.  Several  organisations  recommended  extending  project  cycle  lengths  to  allow  more  time  for 
successful initiatives and adapting to changes like  new regulations. While  the programme’s objectives fit many, 
smaller  organisations  expressed  concern  over  administrative  and  financial  burdens,  particularly  co-funding 
requirements. Some suggested introducing smaller grants to make participation more manageable and beneficial. 

EU added value 

Beneficiaries  view  the  justice  programme  as  a  vital  funding  source,  especially  for  organisations  working  on 
transnational  projects  or  niche  areas  like  child  justice.  Many  stated  that  without  this  support,  they  could  not 
implement their projects on the same scale across multiple EU Member States. While other funding options exist, 
such as from private entities and national grants, these alternatives do not provide the same level of support. Smaller 
organisations,  particularly  those  working  on  victim  support  or  legal  education,  consider  the  justice  programme 
essential, as other programmes often lack the necessary scope or focus. One beneficiary noted that while programmes 
like Erasmus+  or Horizon  Europe offer  some assistance,  they tend  to be  more  academic  or restricted  to  smaller 
projects. A few  highlighted that without the justice  programme, funding specific activities, such as cross-border 
training, would have been challenging. For example, one organisation working with dogs in the justice system stated 
that training and placement costs would have been unfeasible without EU support. 

COVID-19 

The  COVID-19  pandemic  had  a  significant  impact  on  many  projects,  with  most  beneficiaries  adapting  to  the 
challenges by transitioning to digital solutions. Online tools and virtual meetings became standard, enabling projects 
to continue despite restrictions on in-person interactions. Some organisations found unexpected benefits in reaching 
wider audiences, but many noted that online formats could not fully replace in-person engagement, especially for 
child participation or hands-on activities like training dogs. While digitalisation was seen as a by-product rather than 
a core objective, it became essential for project continuity. The justice programme was praised for its flexibility, 
offering contract extensions and allowing format adaptations. However, beneficiaries faced challenges, including 
funding  delays  and  difficulties  with  grant  amendments.  Overall,  while  the  pandemic  forced  organisations  to 
innovate,  the  rapid  shift  to  digital  tools  highlighted  limitations  in  maintaining  interaction  quality  and  project 
outcomes. The justice programme’s responsiveness and support during this time were greatly appreciated. 

Gender mainstreaming 

Feedback  from  beneficiaries  on  gender  equality  and  mainstreaming  reveals  a  mixed  approach.  Some  projects 
actively integrate gender considerations, while others focus on their specific work areas where gender equality may 
not be central. For instance, organisations working with vulnerable groups like crime victims acknowledged that 
gender inequality is part of their focus, such as in intimate partner violence, but they may not explicitly frame their 
work as gender mainstreaming. Several beneficiaries highlighted their efforts to promote gender equality through 
internal  policies  and  ensuring  gender balance  in  project  participation,  including  using  gender  lenses  in  proposal 

100 

 
writing and conducting gender-sensitive analyses. However, achieving gender balance can be challenging in fields 
like youth in conflict with the law, where over 90% of the population is male. Some projects do not prioritise gender 
equality as a core objective but maintain non-discriminatory activities. A few beneficiaries also noted difficulties in 
understanding  the  European  Commission’s  gender  mainstreaming  approach,  particularly  in  balancing  gender 
participation and collecting gender-specific data due to privacy concerns. In conclusion, while gender mainstreaming 
is promoted in some areas, it is not universally prioritised. Nevertheless, there is a commitment to non-discriminatory 
practices, with some beneficiaries actively incorporating gender equality into their projects. 

Simplification efforts 

Beneficiaries’ feedback on the justice programme highlights both positive changes and ongoing challenges. Many 
noted that reporting processes have become more streamlined compared to previous periods, particularly with the 
introduction of unit costs for travel and accommodation, although this posed challenges for organisations in smaller 
countries where travel costs may exceed fixed amounts. Beneficiaries appreciated the reduced administrative burden, 
especially  in  the  final  reporting  phases,  which  have  become  less  cumbersome  while  retaining  necessary  detail. 
Improved  communication  about  calls  for  proposals  has  also  helped  organisations  plan  better.  However,  some 
beneficiaries  felt  the  process  is  more  anonymous  due  to  less  direct  contact  with  project  officers,  with  much 
communication  now  occurring  digitally,  making  it  feel  impersonal  at  times.  Additionally,  the  new  budgeting 
methodology and unit cost rules have raised questions; the learning process is ongoing. While the programme has 
made strides in reducing administrative burdens and streamlining reporting, further clarity and personal interaction 
could enhance understanding and implementation of new processes. 

Interviews with the Programme Committee Members 

Effectiveness 

PCMs provided valuable insights into the effectiveness of the justice programme, particularly regarding challenges 
in  cross-border  collaboration.  Several  PCMs  highlighted  the  difficulty  of  identifying  partners  in  other  Member 
States, especially from less-represented countries. One PCM noted that they often worked with familiar partners to 
avoid the time-consuming process of explaining the application and reporting procedures to organisations that were 
new  to  the  programme.  This  indicates  that  while  the  programme  is  effective  in  building  networks  among  more 
experienced participants, it struggles to bring in new stakeholders from less-represented Member States, potentially 
limiting its broader impact. Another key point related to the sustainability of partnerships and projects. One PCM 
expressed  concern  over  the  EUR 350 000  cap  on  criminal  justice  projects,  which  they  felt  restricted  the 
programme’s ability to support larger-scale, multi-country initiatives. This limitation reduced the effectiveness of 
projects that aimed to involve more Member States or stakeholders, such as pilot projects that tested innovative tools 
like web radio for remote court hearings. 

Efficiency 

The interviews revealed that, despite improvements, the programme continues to face  administrative challenges, 
especially in relation to the unit cost standardisation. Several PCMs mentioned that the unit costs were not reflective 
of real-world expenses, particularly given rising costs due to inflation. Public administrations struggled with the dual 
reporting requirements of actual costs and unit costs, adding complexity to the process. Furthermore, the application 
process was identified as a significant burden. One PCM quantified the internal cost of preparing a project proposal, 
which reached between EUR 20 000 and 30 000, a figure deemed high even for well-resourced organisations. The 
complexity and time involved in preparing cross-border applications were also noted, with one PCM mentioning 
that coordinating with authorities in multiple countries took considerable resources that were not always accounted 
for in initial budgets. While the high success rate of applications somewhat mitigated these concerns, the overall 
efficiency of the process remains an area for improvement. 

Relevance 

The programme’s alignment with the needs of the justice sector was generally well-regarded by the PCMs Many 
pointed out the growing importance of digitalisation in judicial cooperation, with specific emphasis on e-justice and 
digital  architecture  projects.  PCMs  recognised  the  need  for  continued  funding  in  these  areas,  as  the  digital 
transformation of justice systems is critical for improving cooperation between Member States and increasing the 
efficiency of judicial processes. 

Coherence 

The interviews also touched on the coherence of the programme, particularly in relation to its alignment with 
national  policies.  Some  PCMs  found  it  challenging  to  involve  partners  from  certain  Member  States,  citing 
difficulties in identifying the appropriate contact persons within national Ministries of Justice. Even when the right 
contacts  were  found,  further  delays  occurred  as  these  individuals  had  to  navigate  their  own  internal  procedures 

101 

 
before committing to the programme. This suggests a need for better integration and coordination between the justice 
programme and national-level actors to ensure smoother collaboration. 

EU Added Value 

The PCMs largely agreed on the added value of the justice programme, especially in areas where national resources 
alone would not be sufficient to achieve the same outcomes. One PCM specifically highlighted the visibility of the 
programme, noting a positive development in their Member State where a technical coordination role was created 
within  the  Ministry  of  Justice  to  act  as  a  liaison  between  EU  funding  programmes  and  thematic  departments. 
However, the overall visibility of the programme remained a concern, with several PCMs mentioning that there are 
eligible organisations still unfamiliar with the programme. PCMs also suggested that some eligible organisations 
may not apply since they perceive the barriers to entry high due to the complexity of the application process. 

PCMs  also  emphasised  the  unique  role  the  programme  plays  in  fostering  cross-border  judicial  cooperation.  In 
smaller Member States, in particular, the programme’s funding enables participation in projects that would otherwise 
be beyond their reach due to limited national resources. The ongoing support for building long-term partnerships 
and enhancing judicial cooperation between Member States was seen as a crucial benefit that national programmes 
could not replicate. 

Open public consultation 

Overall, respondents to the public consultation considered the justice programmes’ overall and specific objectives 
to be highly relevant. Among their specific objectives, judicial training and access to justice were considered the 
most relevant: 

• 
75% of respondents considering them relevant to a very high extent, judicial cooperation to a lesser extent, 
•  while drugs policy, which was covered by one of the specific objectives of the 2014-2020 programme, was 
perceived as the least relevant, it was still considered relevant to a very or high extent by 50%respondents. 

Similarly, most respondents considered the activities funded by the justice programmes to be relevant, in particular 
those  related  to  developing  and  maintaining  information  and  communication  technology194,  with  digitalisation 
perceived  as  an  important  dimension  of  the  programme.  The  activities  considered  the  least  relevant  were  those 
targeted at supporting Civil Society Organisations (CSOs)195. 

Respondents considered the programmes effective in achieving their objectives and intended results. In particular, 
in  improving  access  for  all  citizens  to  CSOs  services,  counselling  activities  and  support  activities  (assessed  as 
achieved to a very or high extent by 62%); training stakeholders (assessed as achieved to a very or high extent by 
50%); and increasing cross border interoperability of systems (assessed as achieved to a very or high extent by 50%). 
In that regard, respondents highlighted the development of E-CODEX and ECRIS, as well as the training in the use 
of IT tools, as particularly relevant and effective. The areas where the programmes were considered to have been 
less successful concerned (increasing) mutual trust among Member States and (improving) the implementation of 
Union law instruments and policies (both assessed as achieved to a moderate or low extent by 50%). Generally, 
respondents were also less positive regarding the programme’s visibility, considering that the programmes were not 
well-known (50%) among potential applicants. 

In terms of coherence, most respondents were not able to compare the justice programme with other programmes 
(only three out of eight indicated that they were familiar with other similar EU initiatives and/or national initiatives 
in  the  field  of  justice).  Two  respondents  considered  the  justice  programmes  to  be  complementary  to  other  EU 
programmes or initiatives to a great extent, and one to a limited extent. 

There was consensus among respondents regarding the EU added value of the programmes, with all considering that 
the EU was best placed to fund the type of activities under the justice programme as the EU involvement ensures 
that all Member States adhere to the same standards, resulting in more equal access to justice. As a result, most 
respondents considered that, if the current justice programme were to stop,  Member States would not be able to 
achieve similar results at the national, regional, or local levels and technical cooperation would become limited to 
smaller-scale bilateral projects between neighbouring Member States, while some organisations would not be able 
to secure funding at all. 

A  summary  report  of  the  public  consultation  was  published  on  the  European  Commission’s  Have  your  say 
platform196 in August 2024. 

194 Considered highly relevant by 62% of respondents. 
195 Indicated by 62% of respondents. 
196 https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/13930-Justice-programme-final-evaluation-of-the-2014-2020-
programme-and-interim-evaluation-of-the-2021-2027-programme_en 

102 

 
 
Call for Evidence 

The Commission received thirteen contributions to the Call for Evidence of which seven contributions were removed 
as not compliant with EC rules for publishing feedback and suggestions. Overall, the contributors acknowledged the 
positive impact of the justice programme in advancing legal cooperation, training, and access to justice across the 
EU. The programme was also seen as well-aligned with broader strategic goals. However, stakeholders also noted 
scope  for  improvement  with  calls  for:  simplification,  more  support  for  people  with  disabilities  among  other 
vulnerable groups, enhanced awareness and training, and reconsidering funding for drugs policy projects under the 
justice programme 2021-2027. 

103 

 
 
 
 
 
TABLE OF ABBREVIATIONS 

Term or acronym 

Meaning  

AGs 

AI 

Action Grants 

Artificial Intelligence 

ANOVA 

Analysis of Variance 

AWP 

CEA 

CER 

CERV 

CSOs 

DEP 

DG BUDG 

DG COMM 

DG COMP 

DG CNECT 

DG DGT 

DG DIGIT 

DG EAC 

DG EMPL 

DG ESTAT 

DG INTPA 

DG JUST 

DG NEAR 

Annual Work Programmes 

Cost-Effectiveness Analysis 

Cost-Effectiveness Ratio 

Citizens, Equality, Rights and Values (programme) 

Civil Society Organisations 

Digital Europe Programme 

Directorate-General for Budget 

Directorate-General for Communication 

Directorate-General for Competition 

Directorate-General for Communications Networks, 
Content and Technology 

Directorate-General for Translation 

Directorate-General for Digital Services 

Directorate-General for Education, Youth, Sport and 
Culture 

Directorate-General for Employment, Social Affairs 
and Inclusion 

Directorate-General for European Statistics 

Directorate-General for International Partnerships 

Directorate-General for Justice and Consumers 

Directorate-General for European Neighbourhood and 
Enlargement Negotiations 

104 

 
 
DG OLAF 

Directorate-General for Anti-Fraud / European anti-
fraud Office 

DG REFORM 

Directorate-General for Structural Reform Support 

DG REGIO 

DG RTD 

DG SANTE 

EAW 

EC 

Directorate-General for Regional and Urban Policy 

Directorate-General for Research and Innovation 

Directorate-General for Health and Food Safety 

European Arrest Warrant 

European Commission 

E-CODEX 

E-Justice Communication via Online Data Exchange 

ECRIS 

EJTN 

EMCDDA 

EU 

FPAs 

FTE 

GDPR 

ICT 

IO 

ISF 

ISSG 

IT 

JAAC 

JCOO 

JDRU 

JTRA 

European Criminal Records Information System 

European Judicial Training Network 

European Monitoring Centre for Drugs and Drug 
Addiction 

European Union 

Framework Partnership Agreements 

Full Time Equivalent 

General Data Protection Regulation 

Information and Communication Technology 

International Organisation 

Internal Security Fund 

Inter-Service Steering Group 

Information Technology 

Access to justice (specific objective) 

Judicial cooperation in civil and criminal matters 
(specific objective) 

Drugs policy initiatives (specific objective) 

Judicial training (specific objective) 

LGBTI 

Lesbian, Gay, Bisexual, Transexual and Intersexual 

105 

 
MFF 

MS 

N/A 

NGOs 

NLP 

OGs 

OPC 

PCMs 

REC 

RRF 

RSB 

SDGs 

SWD 

ToR 

TSI 

Multiannual Financial Framework 

Member State 

Not available 

Non-Governmental Organisations 

Natural Language Processing models 

Operation Grants 

Open Public Consultation 

Programme Committee Members 

Rights, Equality and Citizenship (programme) 

Recovery and Resilience Facility 

Regulatory Scrutiny Board 

Sustainable Development Goals 

Staff working document 

Terms of Reference 

Technical Support Instrument 

106