COMMISSION STAFF WORKING DOCUMENT EVALUATION of e-Cohesion in 2014-2020
Date: 2023-07-12
Institutions: Directorate-General for Regional and Urban Policy, European Commission
Subjects: Cohesion Fund, European Regional Development Fund, European Structural and Investment Funds, administrative formalities, data processing, distribution of EU funding, economic and social cohesion, exchange of information, information system, operational programme

EN 

  EN 

   EUROPEAN COMMISSION  Brussels, 12.7.2023  SWD(2023) 258 final  COMMISSION STAFF WORKING DOCUMENT  EVALUATION  of e-Cohesion in 2014-2020 {SWD(2023) 259 final}  
 
 
Table of contents 

1. 

INTRODUCTION ....................................................................................................... 4 

Purpose and scope of the evaluation ............................................................................... 4 

Methodological approach ............................................................................................... 5 

2.  WHAT WAS THE EXPECTED OUTCOME OF THE INTERVENTION? ............. 6 

2.1  Description of the intervention and its objectives ............................................... 6 

2.2 Point of comparison ................................................................................................ 10 

3.  HOW  HAS  THE  SITUATION  EVOLVED  OVER  THE  EVALUATION 

PERIOD? ................................................................................................................... 11 

4.  EVALUATION FINDINGS ..................................................................................... 13 

4.1.  TO  WHAT  EXTENT  WAS  THE  INTERVENTION  SUCCESSFUL  AND 

WHY? ........................................................................................................................ 13 

Effectiveness ................................................................................................................. 13 

Efficiency ...................................................................................................................... 17 

Coherence ..................................................................................................................... 21 

User-friendliness ........................................................................................................... 22 

4.2.  HOW DID THE EU INTERVENTION MAKE A DIFFERENCE? ........................ 24 

EU added value ............................................................................................................. 24 

4.3.  IS THE INTERVENTION STILL RELEVANT? .................................................... 25 

Relevance ...................................................................................................................... 25 

5.  WHAT ARE THE CONCLUSIONS AND LESSONS LEARNED? ....................... 28 

5.1 Conclusions and lessons learned............................................................................. 28 

5.2. Good practices for improving the functioning of e-Cohesion systems ................. 30 

5.3 Implementation of e-Cohesion in the 2021-2027 programming period ................. 31 

ANNEX I: PROCEDURAL INFORMATION ................................................................. 33 

1.  Lead DG and Decide reference ............................................................................. 33 

2.  Derogations granted............................................................................................... 33 

3.  Organisation and timing ........................................................................................ 33 

4.  Evidence and sources ............................................................................................ 33 

5.  External expertise .................................................................................................. 33 

ANNEX II. METHODOLOGY AND ANALYTICAL MODELS USED ....................... 34 

1 

 
 
 
Desk research ................................................................................................................ 34 

Large-scale surveys....................................................................................................... 34 

Case studies................................................................................................................... 35 

Webinar with the representatives of authorities ........................................................... 35 

ANNEX III. EVALUATION MATRIX AND, WHERE RELEVANT, DETAILS 

ON ANSWERS TO THE EVALUATION QUESTIONS (BY CRITERION) ........ 37 

ANNEX  IV.  OVERVIEW  OF  BENEFITS  AND  COSTS  AND  TABLE  ON 

SIMPLIFICATION AND BURDEN REDUCTION ................................................ 43 

ANNEX V. STAKEHOLDERS CONSULTATION - SYNOPSIS REPORT ................. 46 

1.  Strategy .................................................................................................................. 46 

Objectives .................................................................................................................. 46 

Tools .......................................................................................................................... 46 

Stakeholder participation .......................................................................................... 47 

2.  Delivery and results ............................................................................................... 47 

Roadmap ................................................................................................................... 47 

Large-scale survey .................................................................................................... 47 

Interviews .................................................................................................................. 51 

Webinar ..................................................................................................................... 53 

2 

 
 
 
 
 
Glossary 

Term or acronym 

Meaning or definition 

AA 

CA 

CF 

CPR 

EC 

ENI CBC 

ERDF 

ESIF 

EU 

IA 

IB 

IT 

MA 

MS  

OP 

Audit Authority 

Certifying Authority 

Cohesion Fund 

Common Provisions Regulation 

European Commission 

European Neighbourhood Instrument Cross-Border 
Cooperation 

European Regional Development Fund 

European Structural and Investment Funds 

European Union 

Impact assessment 

Intermediate Body 

Information technology 

Managing Authority 

Member State 

Operational Programme 

REACT-EU 

Recovery Assistance for Cohesion and the Territories 
of Europe 

RRF  

SFC 

UK 

Recovery and Resilience Facility 

System for Fund Management in the European Union 

United Kingdom 

3 

 
 
1. 

INTRODUCTION 

Purpose and scope of the evaluation 

 “e-Cohesion”  refers  to  electronic  data  exchange  systems  between  beneficiaries  and 
programme  authorities  in  relation  to  cohesion  policy  funds.  The  Common  Provisions 
Regulation1 (‘the CPR’) governing the 2014-2020 programming period of the European 
Structural  and  Investment  (ESI)  Funds  introduced,  for  the  first  time,  requirements 
designed  to  simplify  and  streamline  the  implementation  of  cohesion  policy 
programmes by facilitating their interoperability with national and EU frameworks for 
the  exchange  of  information.  The  requirements  were  also  intended  to  lower  the 
administrative  burden  on  beneficiaries  by  enabling  them  to  submit  information  on 
programme  implementation  automatically.  e-Cohesion  systems  were  recommended  for 
progress  reporting,  submitting  and  managing  payment  claims  and  exchanging 
information relating to management verifications and audits. 

The evaluation of the introduction of e-Cohesion in the 2014-2020 period sought to: 

  collect  and  provide  up-to-date  information  on  the  implementation  and 
performance  of  e-Cohesion  systems  in  the  Member  States  during  the 
2014-2020 programming period; 
identify  both  good  practices  and  challenges  that  would  be  useful  for  those 
Member  States  who  have  yet  to  implement  e-Cohesion  systems  or  plan  to 
improve their existing systems; 

 

  assess  the  extent  to  which  the  e-Cohesion  requirement  has  delivered  the 

intended policy results; 

  analyse  the  options  for  increasing  the  effectiveness,  efficiency,  coherence 
and  user-friendliness  of  the  e-Cohesion  systems  during  the  2021-2027 
programming period. 

the  2014-2020  period,  Member  States  were  required 

In 
that,  by 
31 December 2015,  all exchanges of information between beneficiaries  and programme 
authorities (managing authorities (MAs), intermediate bodies (IBs), certifying authorities 
(CAs) and audit authorities (AAs)) could be carried out through electronic data exchange 
systems.  However,  beneficiaries  still  had  the  option  to  exchange  information  with 
programme authorities in other ways (such as paper-based communication). 

to  ensure 

1 Regulation (EU) No 1303/2013 of the European Parliament and of the  Council of 17 December 2013 
laying  down  common  provisions  on  the  European  Regional  Development  Fund,  the  European  Social 
Fund,  the  Cohesion  Fund,  the  European  Agricultural  Fund  for  Rural  Development  and  the  European 
Maritime  and  Fisheries  Fund  and  laying  down  general  provisions  on  the  European  Regional 
Development  Fund,  the  European  Social  Fund,  the  Cohesion  Fund  and  the  European  Maritime  and 
Fisheries Fund and repealing Council Regulation (EC) No 1083/2006. 

4 

 
 
                                                           
The CPR referred to two types of electronic systems: electronic data exchange systems, 
i.e.  e-Cohesion  systems  (Article  122(3))  and  the  information  monitoring  systems  for 
recording  and  storing  data  (Article  125(2)(d)).  The  first  is  an  information  exchange 
interface  between  programme  authorities  and  beneficiaries.  The  second  is  an  internal 
financial 
system 
management,  verification  and  audit  activities.  In  practice,  the  two  systems  can  be 
integrated  into  a  single  functioning  module  or  they  can  interact  as  two  distinct  and 
complementary modules. The current evaluation focused solely on e-Cohesion systems. 

for  programme  authorities  used 

for  monitoring,  evaluation, 

The  evaluation  of  e-Cohesion  was  carried  out  between  November  2020  and  May  2022 
and  covered  the  ERDF  and  CF  operational  programmes  (OPs)  from  the  2014-2020 
programming  period2,  including  Interreg  programmes.  The  evaluation  thus  examined 
302 programmes funded by ERDF and CF in the 2014-2020 programming period, 
for which 108 e-Cohesion systems were mapped and analysed. 

The  new  legislative  framework  governing  the  2021-2027  programming  period  has 
retained and strengthened the requirements on e-Cohesion, by rendering the exchange of 
information  outside  e-Cohesion  systems  an  exception,  at  the  explicit  request  of 
beneficiaries. 

Methodological approach 

The  evaluation  followed  the  approach  set  out  in  the  Better  Regulation  framework.  The 
analysis  was  organised  around  the  five  evaluation  criteria  included  in  the  regulation  – 
effectiveness,  efficiency,  relevance,  coherence,  and  EU  added  value  –  plus  a  sixth, 
namely the user-friendliness of the IT systems. 

The evaluation methods included3: 

  desk research/documentary research; and 
 

field research based on: 

o 

large-scale surveys of all types of e-Cohesion system users (beneficiaries, 
managing authorities, intermediate bodies, certifying authorities, and audit 
authorities); 

o  case studies and interviews with stakeholders; and 
o  a webinar with the representatives of the authorities. 

The desk research consisted of online searches (using the websites of e-Cohesion systems 
and  the  websites  of  the  programme  authorities)  and  a  review  of  numerous  documents 
available in  the System  for Fund Management  in  the European  Union (SFC), online or 
shared  by  stakeholders  (e.g.  user  manuals  or  presentations  for  users;  guidelines;  IT 
system descriptions; relevant national evaluations and reports). 

2  The  evaluation  did  not  include  any  OPs  implemented  in  the  United  Kingdom  as  it  was  no  longer  a 

member of the EU at the time the evaluation was commissioned. 

3 The evaluation methods are described in more detail in Annex II, ‘Methodology and Analytical models 

used’. 

5 

 
                                                           
The organisation of a large-scale survey required significant effort given that hundreds of 
thousands  of  projects  and  beneficiaries  were  supported  via  ERDF  and  CF  in  the 
2014-2020  programming  period  across  all  Member  States.  In  addition,  the  number  of 
users  of  e-Cohesion  systems  varies  significantly  between  Member  States,  given  the 
difference  in  size  of  Member  States,  the  different  number  of  programmes  and  the  total 
funding  allocated.  Moreover,  the  ERDF  and  CF  support  is  implemented  differently  in 
each Member State: some countries implement programmes in a centralised way, others 
use a decentralised approach. Therefore, instead of constructing a representative sample 
of users of e-Cohesion, the surveys were opened up to all users. This approach resulted in 
the largest  possible number of survey  addressees and  provided a rich body  of evidence 
for  the  evaluation:  6 248  beneficiaries  and  455  representatives  of  programme 
authorities from all 27 Member States replied to the surveys. Although the statistical 
significance could not be verified (as the total population of users of e-Cohesion remains 
unknown), the findings of the survey have been cross-checked against other data sources, 
such as desk research findings and case studies, in order to draw reliable conclusions. 

The  surveys  were  complemented  by  a  webinar  with  the  Member  State  authorities 
(managing  authorities,  intermediate  bodies,  certifying  authorities  and  audit  authorities), 
which enabled the contractor to follow up on certain survey questions and collect further 
opinions.  The  webinar  took  place  on  23 November  2021  and  was  attended  by  113 
participants from 25 Member States. 

To collect more detailed information  on best practice and the challenges of introducing 
e-Cohesion systems,  the evaluators  also  carried  out  in-depth  analysis  of  six  e-Cohesion 
systems. 84 interviews with stakeholders were conducted as part of the field research for 
the case studies. 

2.  WHAT WAS THE EXPECTED OUTCOME OF THE INTERVENTION? 

2.1   Description of the intervention and its objectives 

The objective of e-Cohesion was to support the reduction of administrative burden 
for  beneficiaries  and  costs  for  authorities,  as  an  important  part  of  the  cohesion 
policy simplification effort. 

During the lifetime of a project or operation supported by cohesion policy, beneficiaries 
and  programme  authorities  must  exchange  information  at  certain  moments  during 
implementation.  In  the  absence  of  a  dedicated  electronic  data  exchange  system,  these 
exchanges were  carried  out  based on  document  sharing via  emails  or other means.  If  a 
document  was  required  by  more  than  one  authority,  the  beneficiary  had  to  submit  it 
separately  to  each  of  them.  The  paper-based  exchange  of  information  was  time-
consuming  and  placed  a  significant  burden  on  beneficiaries,  in  particular  smaller  ones 
with limited administrative capability. 

6 

 
The e-Cohesion systems were introduced to facilitate interoperability with national 
and  EU  frameworks  and  enable  beneficiaries  to  submit  the  relevant  information 
only once. These are the two key principles of e-Cohesion. 

The  provision  envisaged  a  scenario  in  which,  once  project  implementation  starts,  all 
exchanges  of  documents  between  the  beneficiaries  and  the  programme  authorities  take 
place electronically on a dedicated platform; the latter can store and make available the 
documents  already  submitted  for  other  purposes  (audits,  verifications),  without  asking 
the  beneficiary  to  resubmit  them.  In  those  cases  where  interoperability  with  other 
national  databases  (e.g.  business  or tax databases) is  in  place, the information allowing 
the identification of the beneficiary can also be retrieved from a common database. The 
administrative  burden  on  the  beneficiary  is  further  reduced  thanks  to  the  interactive 
forms  and  system  alerts,  which  guide  them  through  the  various  steps  of  the 
implementation  process.  The  beneficiary  also  takes  advantage  of  the  automatic 
calculations and embedded controls to reduce the risk of errors. The progress reports and 
payment claims are also processed electronically via the system, while the security and 
integrity of the data are ensured. Finally, interoperability with EC systems such as SFC4 
would  further  ensure  the  automatic  extraction,  aggregation  and  submission  of  the 
programme data to the EC. 

To  take  advantage  of  the  digitalisation  of  the  exchange  of  information,  additional 
simplification measures related to e-Cohesion systems were introduced at European level 
under  Commission  Implementing  Regulation  (EU)  No  1011/2014  (‘the  implementing 
Regulation’)5. The implementing Regulation detailed key processes, functionalities, and 
data  security  requirements  that  complement  the  key  processes  laid  down  in  the  CPR. 
These requirements are described in more detail in Table 1. 

Table 1: The principles, key processes, functionalities, and data security requirements of e-
Cohesion as listed in Commission Implementing Regulation (EU) No 1011/2014 

Category 

Principles 

Dimension 

Interoperability. The systems referred to in the first subparagraph shall 
facilitate interoperability with national and Union frameworks (…) 
Once-only encoding. The systems referred to in the first subparagraph 
shall (…) allow for the beneficiaries to submit all information referred 
to in the first subparagraph only once. Submission of documents and 
data through the electronic data exchange systems shall be made only 
once as regards the same operation for all authorities implementing the 

4 SFC is the electronic exchange of information concerning shared management funds, between Member 

States and the European Commission, provided by Art. 74(4) CPR. 

5 Commission Implementing Regulation (EU) No 1011/2014 of 22 September 2014 laying down detailed 
rules for implementing Regulation (EU) No 1303/2013 of the European Parliament and of the Council 
as regards the models for submission of certain information to the Commission and the detailed rules 
concerning  the  exchanges  of  information  between  beneficiaries  and  managing  authorities,  certifying 
bodies  
authorities, 
https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32014R1011  

intermediate 

authorities 

audit 

and 

7 

 
                                                           
Category 

Dimension 

same programme. 

Key processes  Reporting on progress 

Functionalities 

Data security 
requirements 

Payment claims 
Exchange of information related to management verifications and audits 
Interactive forms and/or forms prefilled by the system on the basis of 
data that are stored at consecutive steps in the procedures 
Automatic calculations where applicable 
Automatic embedded controls which reduce repeated exchanges of 
documents or information as far as possible 
System-generated alerts to inform the beneficiary that certain actions 
can be performed 
Online status tracking, allowing the beneficiary to monitor the current 
status of the project 
Availability of all previous data and documents processed by the 
electronic data exchange system. 
Exchanges of data and transactions shall bear an electronic signature 
compatible with one of the three types of electronic signature defined 
by Directive 1999/93/EC of the European Parliament and of the Council 
The electronic data exchange systems shall be accessible either directly 
through an interactive user interface (a web application) or via a 
technical interface that allows for automatic synchronisation and 
transmission of data between beneficiaries’ and Member States’ 
systems 
Data security 
Data integrity 
Data confidentiality. When processing information, the electronic data 
exchange systems shall guarantee the protection of privacy of personal 
data for individuals and commercial confidentiality for legal entities. 
Authentication of the sender 
The electronic data exchange systems shall be available and operational 
during and outside standard office hours, except for technical 
maintenance activities. 

Source: Evaluation of e-Cohesion 2014-2020, final report, 2022 

The  implementing  Regulation  clearly  requires  that,  as  a  minimum,  e-Cohesion  systems 
are  used  for  progress  reporting,  submitting  payment  claims  and  the  exchange  of 
information  related  to  management  verifications  and  audits  between  beneficiaries  and 
programme authorities. However, there are earlier stages in the project lifecycle, such as 
the  project  application,  selection  and  contracting,  for  which  the  legislation  does  not 
require  the  use  of  e-Cohesion  systems.  Thus,  the  programme  authorities  were  able  to 
choose  whether  or  not  to  use  e-Cohesion  systems  for  the  stages  preceding  project 
implementation. 

Based on the legal acts described above, the objectives of the introduction of e-Cohesion 
systems can be summarised as follows: 

8 

 
reduce multiple transmissions of the same information; 
improve data security (confidentiality, integrity, availability, non-repudiation); 

 
 
  ensure data privacy; 
 
 
 

improve communication; 
improve transparency and accessibility of relevant information; and 
improve  data  quality  (fewer  calculation  errors,  fewer  missing  values,  correct 
format, fewer other inconsistencies, high up-to-datedness). 

Based on these objectives, the intervention logic of the e-Cohesion initiative is presented 
in Figure 1. 

Figure 1: The intervention logic of e-Cohesion 

Source: Evaluation of e-Cohesion 2014-2020, final report, (2022). 

9 

 
 
 
2.2 Point of comparison 

The  Impact  Assessment  (IA)  accompanying  the  proposal  for  the  CPR  2014-20206 
analysed, among other aspects, the issues related to the delivery of the cohesion policy. It 
noted  that,  despite  the  progress  in  IT  systems  throughout  the  EU,  the  communication 
between  beneficiaries  and  programme  authorities  was  still  largely  paper-based.  It  also 
mentioned  that  for  the  delivery  of  cohesion  policy  investments,  the  heaviest  costs  for 
beneficiaries  are  linked  to  the  processes  of  applying  for  funding,  reporting  and  storing 
documents. Considerable costs were associated with the transcription and aggregation of 
financial  and  monitoring  data  on  paper,  while  control  costs  were  deemed  to  be  higher 
than necessary, as supporting documents were not always easily available and accessible. 

As such, one of the avenues for reducing the administrative burden for the beneficiaries 
was to  do away with  the need to  copy, submit or resubmit and retain large volumes of 
documents on paper. 

In this respect, the IA assessed several options for the introduction of a legal provision on 
the  use  of  electronic  exchange  of  data  between  the  beneficiaries  and  programme 
authorities  and  retained  the  proportional  approach  (oblige  the  managing  authorities  to 
make available electronic data exchange systems and leave the option to beneficiaries to 
make use of them or choose other data exchange channels). 

the 
Consequently,  making  electronic  data  exchange  systems  available  between 
beneficiaries  of  cohesion  policy  support  and  the  programme  authorities  was  one  of  the 
responsibilities of the Member States under  Article 122(3) of the CPR 2014-2020. This 
was the first time such a requirement was included in the legislation governing cohesion 
policy.  The  deadline  provided  by  the  legislation  for  doing  so  was  31 December  2015. 
Subsequently,  the  e-Cohesion  requirement  was  retained  in  Article 69(8)  of  the  CPR 
2021-2027. 

Having  analysed  the  expected  benefits  of  introducing  e-Cohesion,  the  evaluation  study 
provides evidence on how the benefits of e-Cohesion compared to its costs and whether it 
resulted in simplification and a reduction in the administrative burden. 

6  Commission  Staff  Working  Paper  Impact  Assessment  accompanying  the  document  Proposal  for  a 
REGULATION  OF  THE  EUROPEAN  PARLIAMENT  AND  OF  THE  COUNCIL  laying  down 
common  provisions  on  the  European  Regional  Development  Fund,  the  European  Social  Fund,  the 
Cohesion  Fund,  the  European  Agricultural  Fund  for  Rural  Development  and  the  European  Maritime 
and Fisheries Fund covered by the Common Strategic Framework and laying down general provisions 
on the  European Regional Development Fund, the European Social Fund and the Cohesion Fund and 
repealing Regulation (EC) No 1083/2006 {COM(2011) 615 final} {SEC(2011) 1142 final} 

10 

 
 
                                                           
3.  HOW HAS THE SITUATION EVOLVED OVER THE EVALUATION PERIOD? 

One of the key outputs of the evaluation study, completed at the beginning of 2022, was 
the mapping of e-Cohesion systems. 

108 e-Cohesion systems were identified for the 2014-2020 ERDF and CF operational 
programmes, in all Member States. The breakdown of these systems by Member States 
sheds  light  on  the  approach  taken  on  the  implementation  of  ERDF  and  CF,  in  a 
centralised or decentralised manner. 

In most Member States (18) there is one single e-Cohesion system used by all operational 
programmes. The highest number of systems identified in a Member State was 32 (Italy), 
while in the remaining Member States the number of systems varied between 2 and 15. 
Figure 2 shows the overall mapping of e-Cohesion systems. 

Figure 2: The number of e-Cohesion systems in each Member States, plus Interreg 

Source: Evaluation of e-Cohesion 2014-2020, final report, (2022). 

Most  of  the  systems  identified  are  used  for  either  operational  or  for  cooperation 
programmes. A minority (11 systems) are used for both types of programmes. 

11 

 
 
 
Table 2: Breakdown of e-Cohesion systems by type 
National/regional 
systems 

National/regional 
systems covering 
Interreg also 

Interreg systems 

75 systems 

11 systems 

22 systems 

These systems are used only 
for national/regional 
programmes. 

These systems are primarily 
used for national/regional 
programmes, but also cover 
some Interreg ones. 

These systems are 
exclusively used for Interreg 
programmes. 

Source: Evaluation of e-Cohesion 2014-2020, final report, (2022). 

The evaluation study also revealed the coverage of programmes by e-Cohesion systems. 
Most  systems  (77)  cover  only  one  programme,  while  a  few  of  them  are  used  by  more 
than  20  programmes.  This  is  the  case  for  the  following  e-Cohesion  systems:  Synergie 
(used  in  France,  covering  37  programmes),  eMS  (used  by  and  covering  36  Interreg 
programmes), SL2014 (used in Poland, covering 25 programmes), MIS (used in Greece, 
covering 22 programmes), Fondos2020 (used in Spain, covering 21 programmes). 

33  programmes  (from  Greece,  Spain,  Italy,  Austria  and  Germany)  used  several  e-
Cohesion systems. This was the case especially when different intermediate bodies were 
involved in the management of the programme, each with its own specific projects which 
required tailored e-Cohesion systems. 

Despite an increase in the number of e-Cohesion systems, from 12 before 2013 to 108 in 
the 2014-2020 programming period, there are still a few programmes that do not have an 
e-Cohesion system: two in Germany, one each in Finland and France, and four Interreg 
ENI CBC programmes  (out  of 15 ENI CBC programmes). However, for  the ENI CBC 
programmes  there  was  no  obligation  under  the  2014-2020  legal  framework  to  set  up 
e-Cohesion  systems.  This  changed  in  the  2021-2027  programming  period.  It  is 
interesting to note that, even in the absence of a legal requirement, most of the ENI CBC 
programmes  have  already  started  using  e-Cohesion  systems  in  the  2014-2020 
programming period. 

12 

 
 
 
 
4.  EVALUATION FINDINGS 

This  section  presents  the  results  of  the  evaluation  study  in  relation  to  the  evaluation 
criteria.  It  is  divided  into  three  parts:  the  first  part  offers  insight  into  whether  the 
intervention  was  successful  and  the  reasons  behind  this.  It  covers  the  effectiveness, 
efficiency, coherence, as well as the user-friendliness evaluation criteria. The second part 
discusses whether the EU intervention made a difference (the EU added value criterion), 
while the last part examines the relevance criterion. 

4.1.  To what extent was the intervention successful and why? 

Effectiveness 

Under the effectiveness criterion, the evaluation aimed to establish: 

-  whether  the  introduction  of  e-Cohesion  has  resulted  in  simplification  and  a 

- 

reduction in the administrative burden; and 
the  extent to  which  the systems  in  place  in  the Member  States  comply  with  the 
requirements set out in the legislation in terms of the key principles and processes 
covered. 

Simplification and reduction of administrative burden 
The  reason  why  the  introduction  of  e-Cohesion  systems  was  mandated  by  the  CPR 
2014-2020  was  to  simplify  a  critical  aspect  of  the  delivery  of  the  cohesion  policy, 
reducing the administrative burden related to the exchange of information between 
the beneficiaries and the programme authorities. 

The results from the field research show a positive response from both beneficiaries and 
programme  authorities  on  the  reduction  of  administrative  burden  brought  about  by  the 
use of e-Cohesion systems. The answer to this question is positive, as the surveys show 
that close to 90% of both the beneficiaries and the authorities surveyed replied that the 
introduction  of  e-Cohesion  simplified  the  exchange  of  information  and  resulted  in  a 
reduction in the administrative burden. 

13 

 
 
 
Figure 3: the perception of e-Cohesion users on the impacts on simplification 

Authorities

Beneficiaries

Increased the transparency and
accessibility of relevant
information (N=395)

It has reduced the repeated
transmission of the same
information (N=387)

Resulted in a faster exchange of
information between
beneficiaries and authorities
(N=386)

Improved communication
between beneficiaries and
authorities (N=382)

42%

54%

3%

1%

39%

51%

7%

2%

38%

52%

9%

1%

37%

50%

11%

2%

Increased the transparency and
accessibility of relevant
information (N=4818)

It has reduced the repeated
transmission of the same
information (N=4910)

Resulted in a faster exchange of
information between
beneficiaries and authorities
(N=4891)

Improved communication
between beneficiaries and
authorities (N=4840)

27%

63%

8%

2%

27%

55%

15%

4%

27%

56%

13%

4%

23%

58%

16%

4%

Improved data quality and
integrity (N=390)

35%

56%

8%

1%

Improved data quality and
integrity (N=4600)

25%

65%

8%

2%

Improved data security and
privacy (N=345)

34%

58%

8%

1%

Improved data security and
privacy (N=4274)

27%

64%

7%

2%

Resulted in standardisation of
programme management
processes (N=358)

30%

58%

10%

2%

Reduced costs relating to the
management of projects (N=319)

29%

47%

20%

4%

Increased legal compliance (e.g.
reduced risk of double funding,
fraud, etc.) (N=340)

28%

54%

16%

2%

Reduced costs relating to the
management of projects
(N=4941)

Increased legal compliance (e.g.
reduced risk of double funding,
fraud, etc.) (N=4335)

29%

53%

13%

5%

24%

64%

10%

2%

Strongly agree

Agree

Disagree

Strongly disagree

Strongly agree

Agree

Disagree

Strongly disagree

0% 20% 40% 60% 80% 100%

0% 20% 40% 60% 80% 100%

Source: Evaluation of e-Cohesion 2014-2020, final report, (2022). 

The high percentage of  positive answers (‘strongly agree’ and ‘agree’), and the fact that 
they  represent  broadly  similar  proportions  of  the  replies  of  both  beneficiaries  and 
programme  authorities,  supports  the  conclusion  that  the  introduction  of  e-Cohesion  has 
been  successful  in  delivering  simplification.  The  same  conclusion  emerged  from  the 
discussions  in  the  webinar  with  programme  authorities.  Therefore,  considering  the 
assessment of the stakeholders’ responses, the introduction of e-Cohesion contributed 
to the simplification and a reduction in the administrative burden. 

Compliance with the legal requirements 
Another  aspect  analysed  by  the  evaluation  report  under  the  effectiveness  criterion  was 
the extent to which the e-Cohesion systems in place in the EU include the required 
key  principles,  processes,  and  functionalities.  The  surveys  and  desk  research  carried 
out in the context of the evaluation study provide a more nuanced picture regarding the 
implementation and compliance with the legal requirements. 

The two key principles to be embedded into e-Cohesion systems are: (1) interoperability 
and (2) ‘once-only’ encoding principle. 

The principle of interoperability states that ‘all the bodies involved in the implementation 
of  a  programme  should  work  together  at  the  organisational  and  technical  levels  in 
ensuring  effective  communication  between  computerised  systems,  as  well  as  the 

14 

 
 
exchange  and  re-use  of  information  and  knowledge’7.  In  relation  to  this  principle,  the 
legislation  provides  that  e-Cohesion  systems  should  facilitate  interoperability  with 
national  and  EU  frameworks.  Without  obliging  programmes  to  introduce  e-Cohesion 
systems, the legislation thus encouraged their development with a view to facilitating the 
reuse of information across different existing databases at national and at European level. 

The  evaluation  looked  at  the  interoperability  principle  from  two  perspectives:  between 
e-Cohesion  systems  and  registers/databases  at  national  level  and  between  e-Cohesion 
systems and management/monitoring systems at European level. 

The  management  and  monitoring  system  of  ERDF  and  CF  at  European  level  for  the 
2014-2020  programming  period  was  SFC2014.  Its  main  function  was  the  electronic 
exchange  of  information  concerning  shared  fund  management  between  Member  States 
and the European Commission. 

Looking at the national level, the desk research found that e-Cohesion systems are most 
commonly  linked  to  the  central  information  monitoring  systems  of  EU  funds  (67 
systems), and secondly to other national registers and databases (31 systems, of which 27 
were also linked to the central information monitoring systems). 

As  regards  the  interoperability  of  e-Cohesion  systems  with  European  databases  and 
monitoring  systems,  24  systems  were  identified  as  having  a  direct  interface  with 
SFC2014 (with one exception, all were also linked to the central information monitoring 
systems of the Member State or region in question)8. 

By  design,  SFC  is  expected  to  be  connected  to  central  monitoring  systems  for 
programme  implementation  (which  stores  the  information  to  be  sent  to  the  EC  from 
programme  level  reporting  purposes),  rather  than  directly  to  the  e-Cohesion  systems 
themselves.  In  fact,  the  e-Cohesion  systems  have  information  at  individual  beneficiary 
level,  while  the  information  sent  to  SFC  by  programme  authorities  is  aggregated  at 
programme  level.  This  aggregation  is  normally  carried  out  by  central  monitoring 
systems. 

The desk analysis concluded that more than 65% of the e-Cohesion systems (72 systems) 
were interoperable with  other databases at  national  or European level  in  the 2014-2020 
period. For 30% of the systems (30 systems) this information could not be verified using 
documentary  sources  or  via  other  interactions  with  stakeholders  and  therefore  their 
interoperability  status  is  unknown.  Three  systems  were  found  to  have  no  interoperable 
modules with other external databases. 

7  Questions  &  Answers  on  e-Cohesion  Programming  period  2014-2020  (ERDF,  Cohesion  Fund  and 
ESF), EGESIF_17-0006-00, 06/04/2017; Building Blocks for e-Cohesion: good practices from Member 
States, regions and programmes, Version 2, December 2013. 

8  In  one  case,  a  system  was  linked  to  the  keep.eu  European  project database  for  territorial  cooperation 

projects. 

15 

 
                                                           
The  once-only  encoding  principle  states  that  the  ‘submission  of  documents  and  data 
through the electronic data exchange systems shall be made only once (…) as regards the 
same  operation  for  all  authorities  implementing  the  same  programme’9.  The  evaluation 
found  that,  overall,  the  e-Cohesion  systems  comply  with  the  once-only  encoding 
principle.  Of  the  108  systems  identified,  only  one  was  non-compliant,  while  for  26 
systems the implementation of this principle could not be established with any certainty 
on the basis of the available documentation. When this principle is analysed on the basis 
of  whether  the  systems  can  reuse  information  provided  by  the  beneficiaries  during  the 
earlier stages of project implementation or even during the project application stage, the 
survey  shows  that  approximately  85%  of  beneficiaries  and  90%  of  authorities  gave  a 
positive  reply.  It  can  therefore  be  concluded  that  the  once-only  encoding  principle  was 
successfully integrated into most of the existing e-Cohesion systems. 

The key processes to be covered by e-Cohesion systems are: (1) reporting on progress, 
(2) payment claims and (3) exchange of information related to management verifications 
and audit. 

The  evaluation  found  that  75%  of  the  108  systems  identified  include  all  key  processes 
and 92% of the systems  include at least one process. When broken down by individual 
key  processes,  the  analysis  reveals  that  for  99  of  the  108  systems  identified,  payment 
claims  could  be  submitted  through  the  system,  while  only  74  systems  allowed  for 
modifications of the progress reports and/or payment claims. 

The analysis showed that the extent to which an e-Cohesion system is (exclusively) used 
to  exchange  information  for  all  key  processes  varies.  The  exchange  of  information 
relating  to  payment  claims  and  progress  reports  are  the  processes  supported  and  used 
most  extensively  in  e-Cohesion  systems;  the  exchange  of  information  relating  to 
management  verifications  and  on-the-spot  checks 
in  most  cases 
(97 e-Cohesion  systems)  but  less  used  in  practice.  Most  programmes  still  used  parallel 
channels  for  exchanging  information  with  beneficiaries,  especially  communications  via 
emails (to a limited extent for progress reports and payment claims, but extensively so for 
data  relating  to  audit  and  management  verifications).  This  finding  is  confirmed  by  the 
survey replies received from beneficiaries: 76% indicated that they used e-Cohesion for 
submitting payment claims, 70% used it for reporting of progress, while only 44% used it 
for providing documents related to management verifications and on-the-spot checks. 

is  available 

One interesting finding that emerged from the analysis is that almost 80% of the existing 
e-Cohesion systems had the possibility to support the application phase for the projects, 
even  if  this  was  not  explicitly  required  in  the  CPR.  Moreover,  according  to  more  than 
75% of the replies provided by beneficiaries, the e-Cohesion system was used (in some 
cases  with  other  parallel  channels  of  communication)  in  the  contracting  phase  of  the 
projects. 

9 See Article 10(4) of Commission Implementing Regulation (EU) No 1011/2014. 

16 

 
                                                           
The  key  functionalities  of  e-Cohesion  systems  are  as  follows:  (1)  interactive  forms 
and/or forms prefilled by the system; (2) automatic calculations; (3) automatic embedded 
controls; (4) system-generated alerts; (5) online status tracking; and (6) availability of all 
previous data and documents processed by the electronic data exchange system (i.e. this 
information can be retrieved from the system). 

The  evaluation  found  that  the  majority  of  e-Cohesion  systems  (more  than  80%) 
embedded  all  the  key  functionalities  listed  above.  When  looking  at  individual  key 
functionalities, the most present in e-Cohesion systems were the interactive forms (96% 
of the systems), while system-generated alerts were included in only 82% of the systems. 

Lastly,  one  essential  feature  of  all  e-Cohesion  systems  is  related  to  data  security.  The 
data security features  of  each  e-Cohesion  system  should  be in  line  with  data protection 
rules10  to  guarantee  the  security,  integrity  and  confidentiality  of  data  by  means  of  the 
features of (1) data encryption, role-based access control in the form of (2) authentication 
and  (3)  authorisation  and  a  (4)  defined  incident  management  process,  in  the  event  of 
technical  issues  or  disruptions.  Due  to  the  sensitivity  of  the  topic,  it  was  not  always 
possible to identify relevant data in the technical documentations of e-Cohesion systems 
(relevant information was identified for 84 out of 108 e-Cohesion systems). The analysis 
showed  that  in  most  cases  the  e-Cohesion  systems  presented  a  combination  of  three  or 
four of the key features listed above. Considering that the evaluation found no indication 
of data security infringements and, taking into account  the inputs  of the  respondents  to 
the survey (over 90% of survey respondents agreed that data integrity and quality, as well 
as  data  security  and  privacy,  had  improved  significantly  due  to  the  introduction  of  e-
Cohesion  systems),  it  can  be  stated  that  the  implementation  of  e-Cohesion  has  brought 
significant improvements in data security. 

To  conclude,  the  evaluation  found  that  the  majority  of  e-Cohesion  systems  complied 
with  the  legal  requirements,  although  in  certain  cases  some  of  the  key  requirements 
were still under development. 

Efficiency 

Under the efficiency criterion, the evaluation analysed whether: 

- 
- 

the introduction of e-Cohesion resulted in resource efficiency gains; and 
the  benefits  of  e-Cohesion  outweighed  its  costs,  for  both  categories  of  users 
(programme authorities and beneficiaries). 

In  addition,  a  series  of  challenges,  barriers  and  success  factors  that  have  a  direct 
influence of the efficiency of e-Cohesion systems were also analysed. 

10  Directive  95/46/EC  of  the  European  Parliament  and  of  the  Council  of  24 October  1995  on  the 
protection of individuals  with regard to the processing of personal data  and on the  free  movement of 
such data, repealed by Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 
April 2016 on the protection of natural persons with regard to the processing of personal data and on 
the free movement of such data. 

17 

 
                                                           
The administrative costs and burden were analysed through qualitative analysis based on 
the  relevant  stakeholder  responses.  Thus,  the  evaluation  of  the  efficiency  of  the 
e-Cohesion initiative examined whether the beneficiaries and relevant public authorities 
perceived  the  introduction  of  e-Cohesion  as  having  contributed  to  a  reduction  in  costs 
and administrative burden for the delivery of ERDF- and CF-funded investments. 

Resource efficiency 
The data collected via the survey showed that, compared with paper-based processes or 
email exchanges, both the beneficiaries and the programme authorities agreed that 
the  introduction  of  e-Cohesion  resulted  in  a  faster  exchange  of  information  (83% 
and 90% respectively), as well as in a reduction of the costs of  project management 
(82%  and  76%).  The  programme  authorities  who  participated  in  the  webinar  provided 
similar  responses  and  most  frequently  cited  the  reduction  in  costs  relating  to  project 
management as having been less influenced by the use of e-Cohesion systems. This can 
be  explained  by  the  fact  that  the  e-Cohesion  covered  only  few  processes  of  the  project 
lifecycle, while the costs for project management extend beyond these. 

Figure 4: Time and resource gains as a result of electronic data exchange (surveys of authorities 
and beneficiaries) 

Resulted in a faster exchange of 
information between 
beneficiaries and authorities 

Beneficiaries (N=4891)

27%

Authorities (N=386)

38%

52%

9%

1%

Reduced costs relating to the 
management of projects 

Beneficiaries (N=4941)

Authorities (N=319)

29%

29%

56%

47%

53%

13%

4%

20%

4%

13%

5%

0%

20%

Strongly agree

Agree

40%
Disagree

60%

80%

100%

Strongly disagree

Source: Evaluation of e-Cohesion 2014-2020, final report, (2022). 

In  general,  beneficiaries  were  the  group  of  stakeholders  that  reported  the  highest 
perceived  gains  in  terms  of  resources  and  time  (96%  of  respondents)  following  the 
introduction and use of e-Cohesion systems. 

Benefits compared with costs 
When analysing the benefits of e-Cohesion systems compared to their costs in relation to 
the  main  types  of  processes  in  the  implementation  of  projects,  the  results  are 
overwhelmingly  positive  for  both  beneficiaries  and  programme  authorities  (with  an 
average  of  90%  reporting  that  the  benefits  exceed  the  costs,  in  terms  of  the  time  and 
effort required to implement and use the e-Cohesion systems), as shown in the Figure 5 
below. 

18 

 
 
 
 
 
 
Figure 5: Benefits versus costs of e-Cohesion systems 

Authorities

Beneficiaries

Verifying and approving
payment claims (N=352)

Assessing and approving
progress reports (N=315)

Checking and approving
applications (N=312)

Planning and carrying out
audits (N=243)

53%

40%

5%

3%

46%

46%

7%

2%

46%

47%

5%

2%

42%

48%

7%

4%

Planning and implementing
management verifications…

37%

53%

8%

2%

Managing contract
amendments (N=252)

Preparing and signing
contracts for grants (or…

38%

50%

11%

2%

38%

52%

9%

1%

Creating, submitting or
modifying payment claims
(N=4706)
Creating, submitting or
modifying progress reports
(N=4595)
Creating, submitting or
modifying applications for
funding (N=4413)

Submitting documents
relating to management
verifications on-the-spot
checks (N=3783)

Processing contracts for
grants (or other forms of
support) (N=3626)

43%

45%

8%

4%

40%

48%

8%

4%

42%

47%

7%

4%

37%

48%

10%

5%

37%

51%

8%

3%

0% 20% 40% 60% 80% 100%

0% 20% 40% 60% 80% 100%

Strongly agree, benefits exceed costs

Agree, benefits exceed costs

Disagree, benefits do not exceed costs

Strongly agree, benefits exceed costs

Agree, benefits exceed costs

Disagree, benefits do not exceed costs

Strongly disagree, benefits do not exceed costs

Strongly disagree, benefits do not exceed costs

Source: Evaluation of e-Cohesion 2014-2020, final report, (2022). 

Among  the  key  processes  supported  by  the  e-Cohesion  systems,  both  beneficiaries  and 
programme authorities indicated the exchange of information related to payment claims 
as  requiring  the  most  effort.  Respondents  indicated  that  certain  missing  functionalities 
and  technical  features  (e.g.  incorrectly  applied  automatic  calculations,  no  prefilling  of 
information, file size and format limits for required attachments, etc.) required substantial 
additional time and effort. 

The evidence collected by the evaluation for the efficiency criterion shows that users of 
e-Cohesion  (both  beneficiaries  and  programme  authorities)  perceived  these  systems  as 
providing  a  faster  and  more  cost-effective  exchange  framework  throughout  project 
implementation. 

Challenges, barriers and external factors 
In  addition  to  the  simplification  and  reduction  in  the  administrative  burden,  under  the 
efficiency  criterion  the  evaluation  sought  to  collect  information  on  the  administrative 
effort  required by the introduction of  e-Cohesion, as well as on the external  barriers to 
the efficient functioning of e-Cohesion systems. 

Table 3 below shows how managing authorities and intermediate bodies assess the types 
of effort associated with the introduction of new IT systems. 

19 

 
 
 
 
Table 3: Level of effort required by the introduction and operation of e-Cohesion systems11 

Required a lot of 
resources / very 
extensive effort 

Required a 
manageable level of 
resources / 
manageable effort 

Required few 
resources / 
little effort 

IT procurement and tendering  
Human resources required to set up / 
deploy the system 
Administrative costs of setting up / 
deploying the system 
Administrative costs of 
operation/maintenance of the system 
Human resources required for 
operation/maintenance of the system 
Adaptation of legal and policy 
provisions  
Adaptation of organisational structures 
and processes  

42.6% 

38.2% 

38.1% 

21.0% 

20.1% 

18.8% 

16.4% 

48.8% 

57.9% 

52.5% 

60.1% 

62.3% 

55.5% 

58.2% 

8.5% 

3.9% 

9.4% 

18.8% 

17.5% 

25.8% 

25.3% 

Source: calculated on the basis of the Evaluation of e-Cohesion 2014-2020, final report, (2022). 

The  replies  show  that  the  most  resource-intensive  phase  is  that  of  the  setting  up  of 
e-Cohesion  systems,  while  their  operation  largely  involves  a  manageable  level  of 
resources. 

When it comes to external barriers to the efficient functioning of the e-Cohesion systems, 
the  number  one  factor  identified  is  a  lack  of  interoperability  (73%  of  the  replies).  It  is 
followed  by  a  lack  of  digital  skills  among  the  beneficiaries  (54%),  an  insufficient 
digitalisation  strategy  at  national/regional  level  (46%),  and  the  limitations  in  terms  of 
user-friendliness of the systems (44%). 

Figure 6: External barriers to the efficient functioning of e-Cohesion systems 

Lack of interoperability with other systems and registers
(N=360)

31%

42%

18%

10%

Lack of digital skills among beneficiaries (N=361)

14%

40%

34%

11%

26%

30%

30%

Lack of relevant legal frameworks (e.g. for electronic
signatures) (N=350)

Lack of (or insufficiently developed) overall digitalisation
strategy at regional/national level (N=344)

13%

13%

33%

Lack of user-friendliness of the system (N=386)

13%

31%

Lack of legal validity of verification documents (N=305)

12%

22%

Outdated IT infrastructure (N=356)

12%

27%

Cost of purchasing and renewing e-signature tokens and
the administrative procedures in place for obtaining…

11%

23%

34%

39%

38%

36%

38%

20%

17%

28%

26%

27%

Very relevant

Somewhat relevant

0%

20%
Not very relevant

40%

60%
Not relevant at all

80%

100%

Source: Evaluation of e-Cohesion 2014-2020, final report, (2022). 

11 The replies where the respondents indicated that they were not involved in these aspects were removed 

from the table. 

20 

 
 
 
 
                                                           
Interoperability emerges as  a key challenge  for e-Cohesion systems.  This  finding is 
further  analysed  in  light  of  the  external  coherence  between  e-Cohesion  systems  and 
national  and  European  systems  and  databases.  The  detailed  analysis  is  included  in  the 
‘Coherence’ section below. 

Coherence 

Coherence  can  be  defined  as  the  alignment  and  cooperation  between  different  policy 
frameworks, programmes and actions, leading to the better attainment of their objectives. 
The  evaluation  focused  on  coherence  between  the  different  authorities  and  systems  for 
the  electronic  exchange  of  information,  for  the  purposes  of  implementing  EU  cohesion 
policy. Thus, the coherence of the setting up of e-Cohesion systems was assessed at three 
distinct levels: (1) internal coherence at programme level; (2) coherence at national level; 
and (3) coherence with EU systems and databases. 

Internal coherence at programme level 
The analysis of coherence at programme level looked at whether all types of programme 
authorities  had  access  to  the  e-Cohesion  system  in  use  for  the  programme  they  were 
responsible for. The desk research found that for 81 systems (75%), all types of relevant 
authorities  had  access  to  their  e-Cohesion  system.  For  one  of  the  108  systems,  it  was 
found  that  access  was  not  granted  to  all  types  of  authorities,  while  for  26  systems  the 
evaluation  could  not  establish  with  certainty  whether  all  types  of  relevant  public 
authorities had access to the e-Cohesion system. 

Therefore,  it  can  be  concluded  that  internal  coherence  at  programme  level  was 
ensured for the majority of e-Cohesion systems. 

Coherence at national level 
The analysis of coherence at national level looked at whether the e-Cohesion systems are 
compatible  with  other  systems  and  databases,  including  the  central  monitoring  system 
established at national or regional level. 

The  data  collected  indicated  that  it  was  more  common  for  e-Cohesion  systems  to  be 
connected  to  a  central  monitoring  system  (67  systems)  than  to  national 
registers/databases  (31  systems).  A  possible  explanation  for  the  limited  uptake  of 
interconnectivity with other national registers/databases could be a lack of harmonisation 
and  integration  between  definitions  of  common  concepts  (for  example,  different 
terminology  between  European  and  national  legislation)  and  protocols  (for  example, 
different  sets  of  rules  and  guidelines  for  communicating  data)  at  programme/cross-
programme  and  national  level,  which  reduce  the  complementarity  and  compatibility  of 
the  various  systems.  For  example,  some  Member  States  do  not  have  an  integrated 
e-government  system12,  thus  preventing  e-Cohesion  systems  from  accessing  national 
registers  and  databases.  Authorities  also  mentioned  that,  in  certain  cases,  beneficiaries 

12 An integrated e-government system provides a wide range of public services to citizens and businesses 

digitally. It generally integrates services provided by various institutions in a country or a region. 

21 

 
                                                           
had to use alternative methods, or to submit parts of an application manually, because the 
e-Cohesion  system  had  not  integrated  certain  functionalities,  such  as  tendering, 
contracting, invoicing, etc., as these were built on other national IT systems. 

However,  for  40  e-Cohesion  systems  (37%),  the  degree  of  coherence  at  national  level 
could not be established by desk research or via the survey. 

The  difficulty  in  arriving  at  a  satisfactory  conclusion  on  the  coherence  of  e-Cohesion 
systems  at  national  level  is,  it  itself,  a  relevant  finding,  and  highlights  the  need  for 
stronger  integration  of  the  IT  systems  and  databases  at  national  level  to  facilitate  the 
management of EU funds. This conclusion is also borne out by the replies received to the 
large-scale survey,  in which  only  41% of respondents from  public authorities indicated 
that the e-Cohesion system they use is connected to a national or regional database. 

Coherence with EU systems and databases 
With  respect  to  coherence  at  EU  level,  the  evaluation  analysed  whether  e-Cohesion 
systems were linked to the relevant EU systems, such as SFC. 

The  desk  research  findings  suggest  that  only  a  limited  number  of  e-Cohesion  systems 
were connected to SFC (24 out of 108), and only one system was connected to another 
European database13. In the context of the case studies, when asked about the reasons for 
the  lack  of  connection,  representatives  of  the  managing  authorities  explained  that  this 
was not because of incompatibility or other technical reasons, but rather it had to do with 
the cost of developing this new functionality of the e-Cohesion system, as compared to 
transferring  the  data  to  SFC  manually.  The  results  of  the  research  showed  that  central 
monitoring  systems  were  more  often  connected  to  SFC  –  the  central  monitoring 
systems were outside the scope of this evaluation14. This specific finding may deserve a 
future  in-depth  analysis  which  should  examine  coherence  between  these  three  types  of 
systems (e-Cohesion, central monitoring systems and SFC). 

User-friendliness 

In addition to the five Better Regulation evaluation criteria, due to the technical nature of 
the examined subject, an extra evaluation criterion was included in the analysis: the user-
friendliness  of  e-Cohesion  systems.  The  reason  behind  this  choice  is  that  regardless  of 
the extent to which the various principles, functionalities and processes are implemented, 
the intended policy effects of information exchange systems can be enhanced or limited 
depending on how user-friendly those systems are. 

13 The European database in question is keep.eu which showcases information on Interreg projects. 

14 The external coherence between the central monitoring systems and SFC has already been analysed as 
part  of  the  ex post  evaluation  of  the  ERDF  and  CF  in  2014-2020  -  Work  package  2  -  Study  on  the 
monitoring data  on ERDF and Cohesion Fund operations, and on the  monitoring systems operated in 
the  2014-2020  period.  The  evaluation  concluded  that  interoperability  between  the  central  monitoring 
systems and SFC is a point for improvement which could generate efficiency gains and improve data 
consistency. 

22 

 
                                                           
The analysis looked at the level of agreement among the users of e-Cohesion in relation 
to  a  series  of  aspects  which  determine  the  user-friendliness  of  the  systems.  Figure 7 
presents  the  responses  of  stakeholders  that  participated  in  the  large-scale  survey  in 
relation to the user-friendliness of the e-Cohesion systems used. 

Figure 7: Perceived user-friendliness of e-Cohesion systems 

With time and more 
experience using the system, 
it helps me to carry out tasks 
more efficiently

The system’s user interface is 
easy to operate

The system helps users to 
understand which steps to 
follow

Using the system does not 
require extensive training

The system is clear and self-
explanatory

Authorities (N=398)

36%

Beneficiaries (N=5024)

27%

59%

63%

4%

1%

8%

2%

Authorities (N=395)

13%

66%

18%

3%

Beneficiaries (N=5015)

11%

Authorities (N=385)

13%

Beneficiaries (N=4950)

10%

Authorities (N=406)

11%

Beneficiaries (N=5143)

10%

58%

59%

56%

54%

55%

24%

8%

25%

3%

27%

29%

6%

6%

28%

7%

Authorities (N=407)

17%

62%

18%

2%

Beneficiaries (N=5149)

11%

61%

20%

8%

The response time and stability 
of the system is always 
adequate, even during times of 
high traffic

Authorities (N=367)

11%

Beneficiaries (N=4679)

10%

48%

49%

32%

9%

29%

12%

Strongly agree

Agree

Disagree

0%

20%

40%
Strongly disagree

60%

80%

100%

Source: Evaluation of e-Cohesion 2014-2020, final report, (2022). 

On average, the replies indicate that most users in both categories (73% of beneficiaries 
and 75% of authorities) agreed that the e-Cohesion systems they worked with were user-
friendly  (strongly  agree  or  agree).  However,  a  quarter  of  them  consider  that  the 
e-Cohesion  systems  they  used  were  not  user-friendly.  When  analysed  by  the  various 
aspects that make up the concept of user-friendliness, system response time and stability 
appeared  to  be  the  most  problematic  (over  40%  of  respondents  across  both  groups  of 
users disagreed or strongly disagreed that this aspect functioned properly). Other sources 
of  discontent  were  the  level  of  training  required  to  use  the  systems  and  the  logical 
sequence of steps included in each system. Nevertheless, the users tended to agree (90% 
of  beneficiaries  and  95%  of  authorities)  that,  with  time,  and  having  gained  more 
experience with the system, they were able to carry out their tasks more efficiently. 

An additional key aspect of user-friendliness is the help desk and the help functionalities 
embedded in the e-Cohesion systems. The overall perception of these help desk and help 
functionalities is positive (over 70% of users in both categories and for both aspects). The 
interviews with the users of e-Cohesion systems in the context of the case studies show 
to  well-developed  help  functionalities  and 
that 
comprehensive user documentation. The analysis also found that  the collection of user 

they  attach  significant  value 

23 

 
 
feedback and its incorporation into system developments generated higher levels of 
perceived user-friendliness. 

Based on the evidence presented in the evaluation report, it was found that the majority 
of e-Cohesion systems (58%) are moderately user-friendly and 40% are extensively user-
friendly, while 2% are considered to have limited user-friendliness. 

Figure 8: E-Cohesion systems by user-friendliness

2%

40%

58%

extensive user-friendliness (100%-75%)

moderate user-friendliness (75%-50%)

limited user-friendliness (50%-0%)

Source: Evaluation of e-Cohesion 2014-2020, final report, (2022). 

The  conclusion  of  the  analysis  of  the  user-friendliness  criterion  is  that,  while  the 
determinants  of  user-friendliness  may  vary,  the  e-Cohesion  systems  are  overall 
perceived  as  user-friendly.  Future  developments,  such  as  improvements  in  system 
response  time  and  stability,  improvements  to  the  user  interface  and  better  user 
documentation  to  reduce  the  need  for  training,  are  expected  to  improve  the  perceived 
user-friendliness of the systems. 

4.2.  How did the EU intervention make a difference? 

EU added value 

The  analysis  of  the  EU  added  value  criterion  was  carried  out  from  two  angles:  (1)  by 
examining  whether  the  legal  requirement  to  set  up  e-Cohesion  systems  was  the  main 
driver behind the introduction of the systems; and (2) by establishing if the introduction 
of e-Cohesion systems had any spillover effects in the Member States. 

To  determine  the  contribution  of  EU  legislation  to  the  setting  up  of  the  systems  in  the 
Member States, the evaluation assessed whether it was the legal provision that prompted 
the national authorities to set up e-Cohesion systems. Based on the desk research, before 
the entry into force of the CPR in 2014, only 12 systems were in place. During 2014, an 
additional 17 systems were deployed, while 51 systems became operational from 201515. 

15 For 28 systems, the starting date could not be established. 

24 

 
 
 
 
                                                           
No. of  systems that became operational 
during the period indicated 

Table 4: Year of start of operation of e-Cohesion systems 
The  year  when  the  system  was  made 
operational  
2013 and earlier 
2014 
2015  
2016 
2017 
2018 
2019 
2020 
2021 
Unknown 
Total 
Source: Evaluation of e-Cohesion 2014-2020, final report, (2022). 

12 
17 
24 
13 
7 
4 
0 
1 
2 
28 
108 

The time needed to develop and deploy such systems varies, but it can be assumed that 
the systems  that became operational  before 2014, as well as in  the first  two  years after 
that – i.e. up to end-2015 – were decided upon before the legal requirement was in place. 
This amounts to roughly half of the 108 systems. For a quarter of the systems – i.e. those 
developed  after  2015  –  the  legal  provision  was  most  probably  a  determining  factor  in 
their  development,  while  for  25%  of  the  e-Cohesion  systems,  the  year  of  deployment 
could  not  be  identified.  As  regards  these  later  cases,  and  the  systems  deployed  before 
2015  (75%  of  the  systems),  it  can  be  safely  assumed  that  the  development  of  certain 
functionalities and modules was influenced by the European Commission’s advocacy for 
the introduction of e-Cohesion systems.  In the overall context of the implementation of 
ERDF and CF 2014-2020, the Commission communicated, advocated for and promoted 
the e-Cohesion initiative to programme authorities, which might also have played a role 
in its mainstreaming across Member States. 

There  was  therefore  EU  added  value  in  the  e-Cohesion  initiative,  as  it  triggered 
either the deployment of new systems or the further development of existing ones in 
order to integrate the required functionalities and processes. 

The survey of authorities also aimed to establish whether the e-Cohesion systems had any 
spillover effects, e.g. by inspiring other non-cohesion policy systems, or by using lessons 
learned from e-Cohesion when setting up other new systems (in relation to other national 
or  EU  instruments).  Most  authority  representatives  assessed  the  spillover  effects  of  the 
e-Cohesion initiative positively (75% of respondents). This indicates that the e-Cohesion 
initiative supported innovation and the transfer of ideas between stakeholders. 

4.3. 

Is the intervention still relevant? 

Relevance 

When analysing the relevance criterion, the evaluation study (1) investigated whether the 
needs  of  the  different  user  groups  (programme  authorities  and  beneficiaries)  were  met 

25 

 
and  (2)  tried  to  identify  the  external  factors  that  may  impact  the  relevance  of  the  e-
Cohesion system. 

The different categories of user groups use the e-Cohesion systems to different extents –
for  example  the  beneficiaries  would  use  the  system  for  all  key  processes  listed  in  the 
legislation,  while  the  audit  authorities  only  for  one  key  process,  which  was  related  to 
exchange  of  information  for  audit  verifications.  Therefore,  the  systems  are  not  equally 
relevant to all user groups. To determine if the needs of the user groups are met, the field 
research  analysed  the  scope  of  the  use  of  e-Cohesion  systems  by  authorities  and 
beneficiaries by key stages and processes in a project’s life. 

The  analysis  showed  that  exchanges  of  information  in  relation  to  payment  claims  and 
reporting  of  progress  were  the  most  common,  while  the  preparation  and  signature  of 
financing contracts was the process that mostly took place outside the system. This is not 
surprising,  as  the  latter  was  not  a  process  listed  in  the  implementing  Regulation  for  e-
Cohesion  systems,  but  it  was  an  optional  module  which  was  developed  for  many  e-
Cohesion  systems.  The  e-Cohesion  systems  were  not  widely  used  for  communication 
between  programmes  authorities  and  beneficiaries  and  for 
the  planning  and 
implementation  of  audits,  even  if  the  latter  was  part  of  one  of  the  key  processes 
associated with e-Cohesion systems (i.e. exchange of information related to audits). 

In  terms  of  frequency  of  use,  the  analysis  showed  that  the  systems  were  used  most 
extensively  by  managing  authorities  and  certifying  authorities,  and  the  least  by 
audit authorities. 

E-Cohesion systems were less commonly used for the processes outside the scope of the 
legislative  provisions,  such  as  for  the  application  phase  of  the  projects,  including  the 
preparation  and  signature  of  contracts,  or  for  communications  between  the  programme 
authorities and beneficiaries. 

26 

 
 
 
Figure 9: e-Cohesion coverage of key processes 

Authorities

Beneficiaries

Preparing and signing of
contracts for grants (or other
forms of support) (N=245)

Submitting payment claims
for their projects/operations
(N=251)

Reporting on the progress of
their projects/operations (all
reports other than financial
data/payment claims)
(N=251)

Providing documents for
management verifications or
on-the-spot checks (N=240)

When communicating with
beneficiaries in relation to
various process outlined
above (N=359)

34%

40%

26%

76%

22%

2%

70%

29%

2%

38%

55%

8%

30%

51%

19%

0% 20% 40% 60% 80% 100%

All documents/reports are submitted only through the
system
Documents/reports are submitted both through the
system and via other channels (e-mail, paper or similar)
Documents/reports are submitted only using channels
other than the system (e-mail, paper or similar)

Preparing and signing
contracts for grants (or other
forms of support) (N=4827)

Submitting payment claims
for your projects/operations
(N=5079)

Reporting on the progress of
your projects/operations (all
other reports than financial
data / payment claims)
(N=4998)

Providing documents for
management verifications or
on-the-spot checks
(N=4496)

Communicating with
relevant authorities in
relation to various processes
outlined above (N=4892)

41%

38%

21%

76%

20%

4%

70%

25%

5%

44%

43%

12%

31%

53%

17%

0% 20% 40% 60% 80% 100%

All documents/reports were submitted only through the
system
Documents/reports were submitted both through the
system and via other channels (e-mail, paper or similar)
Documents/reports were submitted only using channels
other than the system (e-mail, paper or similar)

Source: Evaluation of e-Cohesion 2014-2020, final report, (2022). 

On  closer  analysis  of  the  communications  between  beneficiaries  and  authorities,  the 
evaluation found that e-Cohesion systems were used for this purpose by almost 90% of 
managing  authorities,  intermediate  bodies  and  certifying  authorities.  Only  40%  of  the 
audit authorities mentioned that they used the e-Cohesion system to a certain extent for 
communicating with the beneficiaries. 

Figure 10: Use of e-Cohesion for communication between authorities and beneficiaries 

32%

26%

MA (N=191)

IB (N=54)

CA (N=53)

AA (N=61)

10%

49%

31%

59%

59%

40%

59%

9%

15%

11%

Total (N=359)

30%

51%

19%

0%

20%

40%

60%

80%

100%

We exclusively use the system, as it gathers all necessary information (including the verification
documents) in electronic form

We partially use the system, as some of the necessary documentation is gathered outside of the system
(via e-mail, paper or similar)

We do not use the system, as it does not support this process

Source: Evaluation of e-Cohesion 2014-2020, final report, (2022). 

27 

 
 
 
From the evidence collected, it can be concluded that while, overall e-Cohesion systems 
were used by all types of users, the degree of relevance varied between the processes 
listed in the legislation (highly relevant) and other processes in the project lifecycle (less 
relevant), and between different types of users, with managing authorities, intermediate 
bodies and certifying authorities being the most frequent users. 

The  evaluation  also  examined  possible  external  factors  that  limited  the  relevance  of  e-
Cohesion systems. This aspect was explored at the level of the authorities, as they had a 
wider overview of these issues. The survey indicated the limited interoperability as being 
the  most  important  challenge  to  the  relevance  of  e-Cohesion  systems  (73%  of  the 
replies).  This  finding  should  be  corroborated  with  one  of  the  most  important  barriers 
identified  by  the  surveys,  more  specifically  the  lack  of  digitalisation  strategies 
sufficiently developed (46%). Another factor which influenced negatively the relevance 
of e-Cohesion systems was the lack of a nationally recognised, legally valid e-signature 
feature. In the absence of an e-signature feature the e-Cohesion system cannot function 
fully electronically, and beneficiaries are more inclined to use parallel channels for data 
exchange. 

5.  WHAT ARE THE CONCLUSIONS AND LESSONS LEARNED? 

5.1 Conclusions and lessons learned 

The main objective of introducing e-Cohesion in the 2014-2020 programming period was 
to  simplify  and  streamline  the  implementation  of  cohesion  policy  programmes,  by 
reducing  the  administrative  burden  for  beneficiaries  and  authorities.  The  evaluation 
found  that  this  objective  was  accomplished  –  both  beneficiaries  and  programme 
authorities  reported  that  e-Cohesion  systems  simplified  the  exchange  of  information 
and reduced the administrative burden for implementing ERDF and CF projects and 
programmes. 

The  ideas  above  emerged  from  the  in-depth  analysis  performed  for  six  evaluation 
criteria:  efficiency,  effectiveness,  coherence,  relevance,  EU  added  value  and  user-
friendliness. 

From  the  effectiveness  perspective,  these  systems  were  expected  to  facilitate  the 
interoperability  with  national  and  EU  frameworks  and  allow  for  the  beneficiaries  to 
submit the relevant information only once. 

The majority of the systems integrated the modules and features expected, in particular 
ensuring  availability  of  all  previous  processed  data  and  documents,  interactive  forms 
and/or forms prefilled by the system and online status tracking. The e-Cohesion systems 
were  extensively  used  for  (1)  submitting  and  managing  payment  claims  and  (2)  for 
reporting on progress. The systems also played an important role in the communication 
between  beneficiaries  and  programme  authorities  in  relation  to  these  two  processes. 
Alternative communication means, such as emails between beneficiaries and programme 

28 

 
 
authorities  were  still  used  in  certain  instances,  mostly  for  the  exchange  of  information 
related to audit verifications. In this respect, the evaluation found that one of the possible 
reasons for the use of alternative means in presence of e-Cohesion systems is the absence 
of e-signature features embedded in the systems. 

In terms of efficiency, users’ perception of e-Cohesion systems is overall positive, as the 
systems have provided faster and more economical information exchange frameworks for 
the  implementation  of  projects.  Beneficiaries  reported  the  highest  benefit  in  terms  of 
resources and time following the introduction of the e-Cohesion systems, especially for 
handling payment requests and reporting on progress. 

e-Cohesion  systems  were  reported  as  having  a  high  level  of  internal  coherence  -  all 
targeted categories of programme authorities (managing authorities, intermediate bodies, 
certifying  authorities  and  audit  authorities)  had  access  to  and  were  able  to  use  the 
systems. As regards, the external coherence, the e-Cohesion systems were more likely to 
be connected to the central monitoring system than to other national registries/databases 
or  European  systems,  such  as  SFC.  A  possible  explanation  for  the  limited  uptake  of 
interconnectivity with other national registers/databases could be a lack of harmonisation 
and  integration  between  definitions  of  common  concepts  (for  example  different 
terminology  between  European  and  national  legislation)  and  protocols  (for  example 
different  sets  of  rules  and  guidelines  for  communicating  data)  between  the  programme 
and national systems. 

As  regards  their  relevance,  e-Cohesion  systems  were  relevant  to  all  types  of  users 
(programme  authorities  and  beneficiaries).  The  analysis  showed  that  the  systems  were 
used  most  extensively  by  representatives  of  managing  authorities  and  certifying 
authorities,  and  least  by  representatives  of  audit  authorities.  As  highlighted  above,  this 
can be explained by the use of alternative means of communication (i.e. emails) for the 
audit verification process, which can be further linked to the possible limited accessibility 
to the e-signature feature for some of the beneficiaries. 

From the perspective of  EU added value, the e-Cohesion  initiative triggered either the 
deployment of new systems  or the further  development of  existing ones for integrating 
the  required  functionalities  and  processes.  The  EU  legislation  and  the  constructive 
dialogue  between  the  EC  and  programme  authorities  had  a  determining  role  in  the 
development  and  deployment  of  e-Cohesion  systems  in  the  2014-2020  programming 
period. 

e-Cohesion  systems  were  perceived  as  being  user-friendly.  The  help  desk 
functionalities,  the  collection  of  user  feedback  and  its  incorporation  into  system 
developments were highly appreciated by the users. 

In  conclusion,  during  the  2014-2020  programming  period,  e-Cohesion  systems  were 
successful in contributing to the EU’s digital progress through simplifying the exchange 

29 

 
of  information  and  reducing  the  administrative  burden.  In  the  period  2021-2027  the 
requirement was further consolidated in the legislation16. 

Areas  where  future  developments  could  deliver  further  simplification  and  reduction  of 
administrative burden include: 

-  The  continuous  developments  of  interoperability  between  e-Cohesion  systems 

and national databases and systems; 

-  The  integration  of  the  e-signature  feature  in  the  e-Cohesion  systems,  which 
would  further  incentivise  beneficiaries  to  use  this  electronic  exchange  channel 
and limit the parallel data exchanges; 

-  The  integration  of  additional  processes  in  the  e-Cohesion  systems,  such  as  the 
application  and  contracting  phases  of  projects,  which  were  already  included 
by some authorities in  their e-Cohesion  systems in  the 2014-2020 programming 
period, even though this was not a legal requirement; 

-  The improvement of response time, stability of the system, user interface and 
user documentation (guides, tutorials, questions and answer repositories) would 
reduce the training needs and increase user-friendliness; 

5.2. Good practices for improving the functioning of e-Cohesion systems 

The evaluation aimed also to collect and present the main lessons learned and challenges 
identified  in  setting  up  e-Cohesion  systems.  In  relation  to  this,  several  good  practices 
have been identified following the in-depth analysis of six e-Cohesion systems, which are 
listed in Table 5. 

Table 5: Good practices in relation to e-Cohesion systems 

Aspect 

Action 

Evolutionary  development  approach  -  characterised  by  a  high 
degree of prototyping, continuous improvements, and frequent releases 
of new versions 

Development 

User-centric approach – systematic collection of user feedback, user 
involvement  in  testing  prototypes  of  new  features,  consideration  of 
user needs 

Versatile  development  team  –  the  combination  of  IT  skills  (may 
involve procurement of private software developer) and knowledge of 
programme implementation  

16 Regulation (EU) 2021/1060 laying down common provisions on the European Regional Development 
Fund, the European Social Fund Plus, the Cohesion Fund, the Just Transition Fund and the European 
Maritime, Fisheries and Aquaculture Fund and financial rules for those and for the Asylum, Migration 
and Integration Fund, the Internal Security Fund and the Instrument  for Financial Support for Border 
Management and Visa Policy. 

30 

 
                                                           
Aspect 

Action 

Legal aspects 

Elimination of paper-based parallel processes  – by making the use 
of the system mandatory or the sole official solution, it eliminates the 
necessity  to  maintain  parallel  processes  and  oblige  authorities  to 
provide solutions of high usability.  

Support  of  the  exchange  of  structured  data  –  as  compared  to  the 
mere upload of unstructured data (e.g. forms as PDF files) that inhibits 
further information processing. 

Key 
Requirements 

Data centralisation – by supporting all key processes (including those 
not yet outlined in the minimum requirements, e.g. application, change 
requests, and communication features), all project-related information 
is centrally accessible in one place. 

Interoperability  beyond  programme  level  –  allows  for  once-only 
encoding  and  extraction  and  verification  of  information  on  a  wider 
scale.  

Provision of integrated e-signature feature – offers the advantage of 
fully  paper-free  processes  that  decrease  the  resources  required  for 
transport and storage. 

Usefulness 

Focus on processes that cause most effort – offers efficient support 
for  activities  that  otherwise  would  cause  most  of  the  administrative 
burden (capturing expenses, handling supporting documents). 

Flexibility  –  users  can  fulfil  tasks  according  to  their  preferences  and 
programmes to meet specific requirements. 

Self-descriptiveness  and  help  features  –  considering  that  most 
beneficiaries do not use the system often, support functionalities, such 
as tooltips, help users to navigate the system. 

User-
friendliness 

Automatically  embedded  validation  and  automatic  calculations  – 
helps  to  verify  the  information  and  reduce  error  rates,  which  reduces 
the administrative burden for both beneficiaries and institutional users. 

Performance  and  stability  –  server  capacity  is  a  requirement  to 
provide a sufficiently short response time.  

Source: Evaluation of e-Cohesion 2014-2020, final report, (2022). 

5.3 Implementation of e-Cohesion in the 2021-2027 programming period 

Under  Article 69(8)  of  the  CPR  2021-2027,  the  implementation  of  e-Cohesion 
requirements remains, in the first instance, the responsibility of the Member States. The 
European  Commission  will  continue  collecting  information  about  the  level  of 
implementation  of  the  e-Cohesion  requirements  in  the  2021-2027  programming  period 
for cohesion policy programmes, in particular using the methods outlined below: 

31 

 
 
1.  by  monitoring  and  checking  the  existence  of  e-Cohesion  systems  during 
programmes and the negotiation of partnership agreements. Where such a system 
has not yet been set up, the Commission will continue monitoring its setting-up; 
2.  at the monitoring committee and annual review meetings with Member States, the 
Commission representatives will enquire about e-Cohesion systems and follow up 
if needed; and 
the key requirements of the e-Cohesion systems will be checked by national audit 
authorities (in their audits of management and control systems) and by European 
Commission auditors throughout the entire programming period17. 

3. 

17 As laid out in Annex XI of the CPR 2021-2027, failure to comply with these requirements may lead to 
the  conclusion  that  this  key  requirement  ‘works  partially  and  substantial  improvements  needed’ 
(category  3)  or  ‘essentially  does  not  work’  (category  4).  Where  this  conclusion  is  associated  with 
deficiencies found in relation to other key requirements (also assessed in category 3 or 4), this may lead 
to interruption of the payment deadline, suspension of payments or financial corrections. 

32 

 
                                                           
ANNEX I: PROCEDURAL INFORMATION 

1.  Lead DG and Decide reference 

The  evaluation  was  managed  by  the  European  Commission’s  Directorate-General  for 
Regional and Urban Policy, DG REGIO Unit B2: Evaluation and European Semester. 

Decide entry: PLAN/2018/4898. 

2.  Derogations granted 

A derogation was  granted from  the obligation to carry out an open public consultation. 
The derogation was justified by the fact that the evidence underpinning the analysis could 
only be collected from the users of e-Cohesion (beneficiaries and programme authorities). 
The  general  public,  not  being  users  of  e-Cohesion  systems,  could  not  provide  any 
relevant input. 

3.  Organisation and timing 

The  evaluation  roadmap  was  published  on  11 September  2020  and  was  closed  on 
30 October 2020. The service contract started on 20 November 2020 and the final report18 
was delivered on 2 May 2022. 

An Interservice Steering Group (ISG) was set up comprising the following EC services: 
Secretariat-General, Budget, and Home Affairs. Five meetings of the ISG were held on: 
17 March  2021,  11 May  2021,  26 November  2021,  17 February  2022,  and  14 March 
2022. 

4.  Evidence and sources 

Evidence was gathered from several types of sources: surveys and interviews with users 
of  e-Cohesion  from  the  four  types  of  programmes  authorities,  as  well  as  beneficiaries, 
case studies, desk research, and a webinar with the representatives of the Member States. 
The main body of evidence consisted of the 6 248 replies to surveys by beneficiaries and 
455  replies  to  surveys  by  programme  authorities.  Although  the  respondents  are  not  a 
representative  sample  of  the  users  of  e-Cohesion,  the  large  number  of  replies  make  it 
possible to draw conclusions on the evaluation criteria. 

5.  External expertise 

The evaluation study was carried out by a consortium composed of PPMI Group 
(Lithuania), Rechenwerk (Germany) and Ismeri Europa (Italy). 

18 See: https://op.europa.eu/en/publication-detail/-/publication/5f2b4c00-e79c-11ec-a534-01aa75ed71a1 

33 

 
 
 
 
                                                           
ANNEX II. METHODOLOGY AND ANALYTICAL MODELS USED 

The  evaluation  of  e-Cohesion  posed  significant  challenges 
to  constructing  a 
representative sample of users from whom to collect evidence. To overcome this risk and 
to  collect  as  much  user  feedback  as  possible  to  underpin  the  analysis,  the  Commission 
services  and  the  contractor  decided  to  take  an  approach  based  on  large-scale  surveys, 
which were complemented by desk research, case studies, interviews, and a webinar with 
the representatives of the authorities. 

Desk research 

The desk research consisted in identifying the systems using open sources and analysis of 
any  relevant  information  that  was  readily  available.  The  desk  research  benefited  from 
input from the Member State level, collected via the network of country experts provided 
by the contractor. 

Large-scale surveys 

The main tools for  gathering evidence were the large-scale surveys. Given the different 
needs  of  the  beneficiaries  and  authorities  when  using  the  e-Cohesion  systems  and  to 
maximise  the  relevance  of  the  responses,  instead  of  a  single  survey,  the  contractor 
designed two different surveys, one for beneficiaries and one for authorities. The survey 
for authorities was further tailored to each of the four types of programme authorities. 

The survey questions were designed to both collect factual information on the e-Cohesion 
systems used by the respondents, as well as to obtain the opinion of the users regarding 
the various functionalities of the systems, their added value and the remaining challenges. 

To  ensure  the  relevance  and  ease  of  understanding  of  the  survey  for  the  various 
respondents,  the  surveys  were  first  piloted  on  a  small  group  of  representatives  of  the 
authorities,  beneficiaries  and  Commission  staff.  The  surveys  were  translated  into  the 
official languages of the EU. 

The  authorities  were  contacted  by  email  and  invited  to  take  the  survey,  while  the 
beneficiaries  were  contacted  via  the  managing  authorities,  given  that  the  latter  hold  the 
most  up-to-date  lists  of  beneficiaries.  The  contractor  sent  reminders  to  and  contacted 
several  authorities  by  phone  to  increase  the  number  of  replies.  When  the  surveys  were 
closed, 455 replies from authorities and 6 248 replies from beneficiaries were received. 

Although representativeness could not be established, responses were numerous enough 
to enable the analysis of the experience of users of e-Cohesion and address the evaluation 
criteria. 

34 

 
 
Case studies 

To  obtain  greater  analytical  depth  than  it  was  possible  to  achieve  via  the  large-scale 
surveys,  the  analysis  included  six  case  studies.  The  case  studies  were  carried  out  on 
systems  from  Estonia,  Greece,  Portugal,  Italy,  Poland,  as  well  as  the  eMS  system 
dedicated to  Interreg  cooperation programmes.  84 interviews were conducted  as  part of 
the case studies. 

Two case studies were run as pilots (Estonia and Greece). They were selected based on 
existing information at the Commission. 

The remaining systems were selected on the basis of the survey results (sufficiently high 
number of replies from both beneficiaries and authorities, high overall scores, with both a 
centralised and a decentralised approach to e-Cohesion). 

The case studies were prepared based on initial desk research and surveys and consisted 
of interviews with three groups of interviewees: 

  Policy:  representatives  of  the  authorities  who  have  designed  the  system  and  are 

responsible for operating the system. 

  Technical: representatives of the authorities and contractors who have set up the 

system. 

  Users: beneficiaries and institutional users. 

This three-pronged approach made it possible to gather sufficiently detailed information, 
which is presented in the case study reports and in the cross-case analysis report, which 
draws conclusions based on all the case studies. 

Webinar with the representatives of authorities 

Following the completion of the surveys and the initial analysis of the findings, a webinar 
was  organised  with  representatives  of  the  authorities  to  discuss  the  findings  and  gather 
additional information on certain aspects. The webinar was attended by 113 participants 
(of whom  18 were staff from  either the  contractor  or the Commission)  and 25 Member 
States were represented. 

Limitations 

  The  findings  under  the  efficiency  criterion  rely  on  qualitative  analysis  (i.e.  the 
perceptions  of  the  stakeholders  involved  in  the  field  research  via  the  large-scale 
survey, case studies and webinar) of the costs and benefits of the implementation 
of e-Cohesion.  

  The survey was experimental in the sense that it was the first time that ERDF and 
CF beneficiaries had been targeted on such a scale. The survey aimed to cover all 
ERDF  (including  Interreg)  and  CF  programmes,  and  targeted  authorities  and 
beneficiaries  from  all  EU  Member  States.  In  relation  to  this  attempt  to  target 
beneficiaries for the first time at scale, the survey has some limitations in terms of 
how  far  conclusions  may  be  drawn,  and  to  what  extent  certain  conclusions  are 

35 

 
substantiated by the  evidence.  In  addition,  the subject  of the survey  –  electronic 
data exchange systems (e-Cohesion systems) – is not a straightforward one. Based 
on the share of ‘Do not know / cannot answer’ answers, the survey results indicate 
that respondents found some of the survey questions difficult to answer. 

  To  reach  the  beneficiaries,  the  evaluation  team  relied  on  the  help  of  managing 
authorities, who were asked to disseminate the survey to their beneficiaries. This 
approach resulted in the evaluation team not having any opportunity to impact the 
scope of beneficiaries targeted, except for constant communication and reminders 
to  the  MAs  asking  them  to  disseminate  the  surveys  to  their  beneficiaries.  The 
evaluation  team  did  not  have  any  information  on  how  many,  or  what  kind  of 
beneficiaries  (i.e.  private,  public),  the  MAs  targeted.  This  approach  of  targeting 
beneficiaries  through  the  MAs  and  giving  the  MAs  full  control  of  this  process 
resulted in a rather uneven distribution of responses. For some countries, hundreds 
of  responses  were  received,  whereas  for  others  there  were  only  a  few.  For 
example,  40%  of  all  beneficiary  respondents  represent  Italy  and  Poland.  Some 
types  of  authorities  or  beneficiaries  are  thus  likely  to  be  overrepresented  in  the 
survey, which in turn places certain limitations on the interpretation of the data. 

 

It should also be noted that the evaluation was carried out during the preparation 
of  the  2021-2027  programmes,  the  Recovery  assistance  for  cohesion  and  the 
territories of Europe (REACT-EU)19 amendments to the 2014-2020 programmes, 
and the preparation of the Recovery and Resilience Facility (RRF)20 programmes, 
which put considerable pressure on the relevant stakeholders and limited the time 
they could devote to responding to the surveys. 

19 https://commission.europa.eu/funding-tenders/find-funding/eu-funding-programmes/react-eu_en  

20 

https://commission.europa.eu/business-economy-euro/economic-recovery/recovery-and-resilience-

facility_en  

36 

 
 
                                                           
ANNEX  III.  EVALUATION  MATRIX  AND,  WHERE  RELEVANT,  DETAILS  ON  ANSWERS  TO 
THE EVALUATION QUESTIONS (BY CRITERION) 

Effectiveness 

Evaluation question 
To what degree does the 
operation of the e-
Cohesion system 
implement the legal 
requirements? 

Judgment criteria 
Effectiveness is deemed 
high if the majority of the 
legal requirements are 
implemented in practice, as 
per the mapping 
framework: 

  principles 
  key processes; 
 
  data security 

functionalities; and 

requirements.  

Methods used 

Mapping  of  e-Cohesion 
systems; 

Surveys  of  authorities  and 
beneficiaries; 

Webinar with authorities; 

In-depth desk research and 
interviews under in-depth 
case studies. 

Surveys  of  authorities  and 
beneficiaries. 

Does the use of the e-
Cohesion system lead to 
(perceived) simplification 
(differentiated by type of 
user and process)? 

Effectiveness is deemed 
high if all of the different 
types of users/respondents 
report that e-Cohesion has 
simplified the way in 
which they handle 
information in at least 
some of their key 
processes; 
Effectiveness is high if 
most users of different 
types report improvements 
in terms of: 

Findings 
The fulfilment of the key 
requirements on principles, 
processes, functionalities, 
and data security 
requirements is mixed; the 
requirements with regard 
to the principles, 
functionalities and data 
security are met by most e-
Cohesion systems 
(although the last of these 
is difficult to assess); 
However, most e-Cohesion 
systems still use parallel 
channels for their key 
processes (to a limited 
extent for progress reports 
and payment claims, but 
extensively so for the 
exchange of data relating 
to audit and management 
verifications); 
To further increase the 
effectiveness of e-
Cohesion, the features and 
functionalities necessary to 
facilitate wholly electronic 
exchanges of information 
must be implemented (e.g. 
integrated e-signature 
features). 
The introduction of e-
Cohesion systems has led 
to simplifications from the 
perspectives of both 
beneficiaries and 
authorities; 
The most significant 
improvements relate to 
simplified data 
management, accessibility, 
transparency, quality and 
integrity. 

 

reduced 
repeated 
transmission  of  the 
same information; 
  data and information 
as 
quality, 
lower error rate, data 
being  more  up-to-
consistency, 
date, 
volume 
and 
granularity of data; 

such 

  communication 

(such 
speed, 

as 

greater 
accuracy, 

37 

 
 
 
 
 
 
 
Evaluation question 

Judgment criteria 

Methods used 

Findings 

Does the use of the e-
Cohesion system lead to a 
(perceived) reduction of 
administrative burden 
and cost (differentiated 
by type of user and 
process) in a longer 
term? 

Efficiency 

Evaluation question 
For each user type and 
process for which e-
Cohesion is used: where 
did e-Cohesion lead to 
improvements or make 
things worse? 

To what extent are the 
benefits of e-Cohesion 
systems higher or lower 
than its costs (per type of 
user)? 

and 

the 
clarity, 
of 
avoidance 
misunderstandings); 
and 
transparency 
accessibility 
of 
relevant information. 

 

e-Cohesion has led to a 
reduction in administrative 
burden if: 

  beneficiaries report 
(and provide valid 
examples) that e-
Cohesion has 
reduced 
administrative 
burden in their key 
processes in the long 
term; 
  users from 
programme 
authorities report 
(and provide 
examples) that e-
Cohesion has 
reduced 
administrative cost 
in their key 
processes in the long 
term. 

Identification of other key 
results for beneficiaries 
and authorities produced 
by e-Cohesion systems. 

Judgment criteria 
Efficiency is deemed high 
if most users of different 
types report gains in terms 
of resources or time in 
most of their relevant 
processes (such as faster 
entry, sharing and retrieval 
of data). 
Efficiency is deemed high 
if most users of different 
types think the benefits of 
the e-Cohesion system 
significantly outweigh its 
costs or burden on them, 
and in comparison to the 
previous paper-based 
processes. 

Surveys  of  authorities  and 
beneficiaries. 

According to both 
beneficiaries and 
authorities, the reduction of 
administrative burden has 
been reduced as a result of 
the introduction of e-
Cohesion systems; 
For beneficiaries, the 
provision of a single point 
of data exchange, 
interactive forms 
(especially prefilled forms) 
and automatic calculations 
and verifications has 
contributed most to the 
reduction of administrative 
burden 
For institutional users, the 
elimination of paper-based 
processes, and the e-
Cohesion system as a 
single point of data 
exchange has contributed 
most to the reduction of 
administrative burden. 

Methods used 
Surveys of authorities and 
beneficiaries; 
In-depth desk research and 
interviews under in-depth 
case studies. 

Findings 
E-Cohesion has resulted in 
significant gains in 
resources and time for the 
majority of users, 
compared with previous 
paper-based processes. 

Surveys of authorities and 
beneficiaries; 
In-depth desk research and 
interviews under in-depth 
case studies. 

Across all processes, the 
benefits of introducing e-
Cohesion systems 
significantly outweigh the 
associated costs compared 
to previous paper-based 
processes. This indicates a 
high degree of efficiency 
when assessing the impact 
of e-Cohesion systems 
during project application 
and implementation. 
While the evaluation could 
not clearly identify one key 
process that causes the 
most effort for users of e-
Cohesion systems, 
payment claims are 
considered a core process 

For each user type and 
process: which actions 
within the workflow 
cause the most effort 
(data capturing, 
checking, searching, 
coordinating)? 

Processes identified by 
different types of 
users/respondents as 
causing the most effort and 
constituting the biggest 
administrative cost or 
(necessary/ unnecessary) 

Surveys of authorities and 
beneficiaries; 
In-depth desk research and 
interviews under in-depth 
case studies. 

38 

 
 
 
Evaluation question 

Judgment criteria 
burden. 

Methods used 

Methods used 
Mapping of e-Cohesion 
systems; 
Surveys of authorities and 
beneficiaries; 
In-depth desk research and 
interviews under in-depth 
case studies. 

Relevance 

Evaluation question 

To what extent do the 
different types of e-
Cohesion systems and 
key functionalities 
available meet the needs 
for exchange of data, 
documents and 
information of the 
different types of users of 
these systems? 

Judgment criteria 
Meets the needs if a large 
majority of different 
categories of users agree 
that the relevant key 
elements of the e-Cohesion 
systems meet their needs; 
if no major categories of 
(potential) users have been 
excluded from using the e-
Cohesion systems; 
Identify whether e-
Cohesion systems are used 
by the relevant 
stakeholders throughout 
the various key processes; 
A list of functionalities by 
type of user for which the 
e-Cohesion systems are 
mostly used / 
functionalities that are the 
most important; 
Identify needs that are 
currently not being met. 

39 

Findings 
for both applicants and 
beneficiaries. Therefore, 
maximising systems’ 
support for this process 
could further increase their 
value and efficiency. 
The introduction of e-
Cohesion systems requires 
notably more effort from 
authorities than their 
operation/maintenance. 
Among the barriers 
affecting the efficient 
functioning of e-Cohesion 
systems, limited 
interoperability represents 
a key issue. In addition, a 
lack of harmonisation and 
simplification can be 
highlighted as an 
overarching challenge 
affecting several areas of e-
Cohesion and resulting in 
burdens for both 
authorities and 
beneficiaries. 
An approach of continuous 
evolutionary development 
represents an overarching 
success factor relevant to 
the efficient functioning of 
e-Cohesion systems that 
can minimise efforts 
during their introduction 
and operation. 

Findings 
While e-Cohesion systems 
are relevant to all 
institutional user groups, 
the extent of their 
relevance varies between 
different types of users and 
system processes; the 
systems are most 
extensively used to 
exchange information 
relating to payment claims 
and progress reports. 
Beneficiaries still use other 
channels (e.g. email) for 
data exchange relating to 
key processes such as 
signing contracts and 
providing documents for 
controls/verifications, as 
well as ad hoc 
communication 
The most important 
functionalities to ensure 
the relevance of e-
Cohesion systems are 

 
 
Findings 
automatic calculations, the 
availability of previously 
submitted data, and 
automatic embedded 
controls. 
Most e-Cohesion systems 
collect user feedback in 
order to continue adapting 
to the evolving needs of 
their stakeholders. The 
findings suggest that there 
is a correlation between the 
attention paid to user 
feedback and the perceived 
user-friendliness of 
systems. 
Lack of interoperability 
with other electronic 
systems and registers 
constitutes a key challenge 
to ensuring the relevance 
of e-Cohesion systems. 
Similarly, the lack of 
relevant legal frameworks 
(e.g. for legally valid e-
signature) further increase 
reliance on parallel 
channels of data exchange. 

Findings 
Across the e-Cohesion 
systems identified, there is 
a high level of internal 
coherence, defined as the 
extent to which programme 
authorities have access 
rights to the system and 
can share data among 
themselves, once submitted 
by beneficiaries. 
Coherence at national level 
is not uniformly developed 
across e-Cohesion systems, 
and the results here are 
varied. It is slightly less 
common for e-Cohesion 
systems to be connected to 
national registers/databases 
than to a central 
monitoring system. 

Evaluation question 

Judgment criteria 

Methods used 

Identify evidence that 
users’ feedback is being 
collected by authorities; 
Identify whether the 
systems are being further 
developed and improved. 

Survey of beneficiaries; 
In-depth desk research and 
interviews under in-depth 
case studies. 

How did the e-Cohesion 
systems adapt to the 
evolving needs of the 
relevant stakeholders? 

What external factors 
make an e-Cohesion 
system (more or less) 
relevant for different 
types of users? 

A list of contextual factors 
influencing relevance that 
were mentioned by 
different types of 
respondents. 

Survey of authorities. 

Methods used 
Mapping of e-Cohesion 
systems; 
Survey of authorities. 

Mapping of e-Cohesion 
systems; 
Survey of authorities; 
In-depth desk research and 
interviews under in-depth 
case studies. 

Judgment criteria 
The systems are internally 
coherent in terms of the 
reuse of information if 
information only needs to 
be encoded once by 
beneficiaries and is shared 
between different 
authorities. 

The systems are 
compatible and 
complementary if e-
Cohesion systems are 
compatible with other 
public electronic systems, 
registers and databases in 
the Member States, and 
can source and exchange 
information with them. 

Coherence 

Evaluation question 
To what extent do 
authorities of the 
programme have access 
rights to the system and 
share data among 
themselves? 

To what extent are the e-
Cohesion systems 
introduced and/or 
developed for the period 
2014-2020 compatible 
and/or complementary 
with relevant national 
register databases and 
other systems of 
electronic exchange for 
the administration of 
other EU funds in the 
Member States? 
To what extent are the e-
Cohesion systems 
compatible with and/or 
complementary to the 
System for Fund 
Management (SFC) and 
other Commission 
systems for the electronic 

The systems are 
compatible and 
complementary if e-
Cohesion systems are or 
could be linked to SFC 
2014 (as well as to any 
other relevant systems such 
as keep.eu (Interreg)). 

Mapping of e-Cohesion 
systems; 
Survey of authorities; 
In-depth desk research and 
interviews under in-depth 
case studies. 

Coherence at EU level was 
limited during the 2014-
2020 programming period; 
only a minority of systems 
were connected to 
European management 
and/or monitoring systems 
for ESIF, such as SFC or 

40 

 
 
 
 
Evaluation question 
exchange of data, 
documents and 
information (e.g. keep.eu 
(Interreg))? 

EU added value 

Evaluation question 
To what extent has the e-
Cohesion initiative (as 
defined in the CPR) 
contributed to the 
development of electronic 
data exchange systems in 
the Member States? 

To what extent has the 
introduction of e-
Cohesion systems 
contributed to the 
dissemination of good 
practice and policy 
learning to other policy 
areas in the Member 
States? 

User-friendliness 

Evaluation question 
Is the e-Cohesion system 
self-descriptive (clear 
structure, feedback via 
tool tips, etc.) and 
intuitively useable? 

Judgment criteria 

Methods used 

Findings 
keep.eu (the latter is only 
relevant for Interreg). 

Judgment criteria 
Identify whether the 
electronic data exchange 
systems already existed 
between authorities and 
beneficiaries, or were 
being developed prior to 
the e-Cohesion initiative; 
Opinion of the authorities 
as to whether the e-
Cohesion initiative 
provided the decisive 
impetus for the 
development/ improvement 
of electronic data exchange 
systems; 
Opinion of the authorities 
as to whether the audits of 
functioning of the 
management and control 
systems carried out by the 
Commission provided 
valuable recommendations 
for improving e-Cohesion 
systems. 
The extent is deemed large 
(high added value) if there 
is compelling evidence that 
e-Cohesion led to: 
similar 
national 
systems being  set up 
for  other  policies 
due  to  learning  from 
e-Cohesion 
the 
promoted  by 
the 
Commission; 

 

common business 
processes and standards 
created and implemented 
in managing similar public 
policy interventions. 

Methods used 
Survey of authorities; 
In-depth desk research and 
interviews under in-depth 
case studies; 
Mapping of e-Cohesion 
systems. 

Findings 
The key aspects of EU 
added value include: the 
introduction of some e-
Cohesion systems in 
Member States where 
similar systems did not 
previously exist, as well as 
contributing to the 
continuous improvement of 
existing systems. The latter 
is, according to the 
findings of the survey, the 
most common outcome of 
e-Cohesion. 

Survey of authorities; 
In-depth desk research and 
interviews under in-depth 
case studies. 

Additional dimensions on 
EU added value range from 
the increased use and 
coverage of e-Cohesion 
systems to positive 
spillover effects into other 
policy areas with the 
development of electronic 
data exchange systems to 
accommodate 
national/regional as well as 
other EU-level funds and 
schemes. 

Judgment criteria 
User-friendliness is 
deemed high if most 
users/respondents of 
different types: 

  characterise 
system 
as 
descriptive 
clear; 

the 
self-
and 

  agree  that  using  it 

Methods used 
Surveys of authorities and 
beneficiaries; 
In-depth desk research and 
interviews under in-depth 
case studies of good 
practice systems. 

Findings 
Overall, e-Cohesion 
systems exhibit a high 
degree of clarity, ease of 
use and self-
descriptiveness. However, 
there remains notable 
variation between different 
systems; 
While not all systems meet 

41 

 
 
 
Evaluation question 

Judgment criteria 

Methods used 

Does the e-Cohesion 
system have the main 
functionalities, as per e-
Cohesion requirements, 
that facilitate user-
friendliness? 

Does the e-Cohesion 
system provide help 
functionality and a help 
desk service? 

 

the 

users 

user 
appealing 

does 
require 
not 
(extensive) training;  
  agree that the system 
to 
what 
to 

helps 
understand 
operating  steps 
follow; 
find 
interface 
and easy to use; 
  are  happy  with  the 
clarity  and  level  of 
the 
complexity  of 
system. 
User-friendliness is 
deemed high if: 
  e-Cohesion 
systems 
support 
key 
functionalities closely 
related 
user-
to 
friendliness; 

  most 

users/respondents  of 
types 
different 
consider 
key 
functionalities  to  be 
useful  (they  simplify 
their  key  processes 
and the way in which 
the 
they 
handle 
exchange 
of 
information). 

User-friendliness is 
deemed high if: 

  most 

users/respondents  of 
types  are 
different 
the 
happy  with 
system’s 
help 
functionality and user 
documentation 
(software features); 

  most 

users/respondents  of 
different  types  agree 
that 
the  help  desk 
service 
(organisational 
function) 
helpful assistance. 

provides 

Mapping of e-Cohesion 
systems; 
Surveys of authorities and 
beneficiaries; 
In-depth desk research and 
interviews under in-depth 
case studies of good 
practice systems. 

Mapping of e-Cohesion 
systems; 
Surveys of authorities and 
beneficiaries; 
In-depth desk research and 
interviews under in-depth 
case studies. 

Findings 
all of their users’ needs 
consistently, users 
overwhelmingly agree that, 
with time and more 
experience, e-Cohesion 
systems help them to carry 
out tasks more efficiently. 

Nearly all e-Cohesion 
systems support all key 
functionalities closely 
associated with user-
friendliness, and users are 
highly satisfied with them 
overall; 
The provision of e-
signatures can greatly 
reduce administrative 
burden by enabling fully 
paper-free processes; 
The incomplete 
implementation of 
functionalities and absence 
of various technical 
features can result in 
significant burdens for 
users and diminish the 
user-friendliness of e-
Cohesion systems. 
Satisfaction with the 
support features of e-
Cohesion systems is 
widespread despite some 
caveats – overall, help 
functionalities and help 
desk services are well 
implemented and largely 
meet the needs of both 
beneficiaries and 
institutional users; 
Both types of features can 
serve needs beyond 
support, such as improving 
communication between 
authorities and 
beneficiaries and 
contributing to system 
development.  

42 

 
 
 
 
ANNEX  IV.  OVERVIEW  OF  BENEFITS  AND  COSTS  AND  TABLE  ON  SIMPLIFICATION  AND 
BURDEN REDUCTION 

Given  the  scope  of  the  e-Cohesion  initiative  (hundreds  of  thousands  of  people  at  beneficiary 
and  programme  authority  level  use  these  systems),  a  quantitative  analysis  of  the  costs  and 
benefits would have required significant resources in terms of time and budget which were not 
available  for  this  evaluation.  Due  to  the  lack  of  quantitative  data  on  the  costs,  benefits  and 
savings  associated  with  the  introduction  of  e-Cohesion  are  described  in  the  following  tables 
from a qualitative perspective, as the perceived reduction of costs and reduced administrative 
burden by beneficiaries and programme authorities. 

Table 1 Overview of the costs and benefits identified in the evaluation 

Comment 

Project beneficiaries 
Quantitative 
Cost associated with the transcription and aggregation of financial, 
monitoring data on paper and with the submission of documents on 
paper 
n/a 

Programme authorities 
Quantitative 

Comment 

n/a 

Direct 
compliance 
Enforcement cost 

Recurrent  

Direct 
compliance 
Enforcement cost 

Recurrent  

Direct 
compliance 
Enforcement cost 

Recurrent  

Direct 
compliance 
Enforcement cost 

Recurrent  

The 
exchange of 
information 
related to 
payment 
claims was 
found to  
require the 
most effort  

88% of 
beneficiary 
respondents 
to the survey 
indicated 
that benefits 
exceeded 
costs in 
relation to 
this process.   

88% of 
beneficiary 
respondents 
to the survey 
indicated 
that benefits 
exceeded 
costs in 
relation to 
this process.   

85% of 
beneficiary 
respondents 

Benefits related to the introduction of e-Cohesion systems for 
payment claims 
n/a 

n/a 

Benefits related to the introduction of e-Cohesion systems for 
progress reports 
n/a 

n/a 

Benefits related to the introduction of e-Cohesion systems for 
management verifications 
n/a 

n/a 

43 

The 
exchange of 
information 
related to 
payment 
claims was 
found to  
require the 
most effort  

93% of 
programme 
authorities 
respondents 
to the survey 
indicated 
that benefits 
exceeded 
costs in 
relation to 
this process.   

92% of 
programme 
authorities 
respondents 
to the survey 
indicated 
that benefits 
exceeded 
costs in 
relation to 
this process.   

90% of 
programme 
authorities 

 
 
 
 
 
 
 
 
 
 
Direct 
compliance 
Enforcement cost 

Recurrent  

Project beneficiaries 
Quantitative 

Comment 
to the survey 
indicated 
that benefits 
exceeded 
costs in 
relation to 
this process.   

Programme authorities 
Quantitative 

Comment 
respondents 
to the survey 
indicated 
that benefits 
exceeded 
costs in 
relation to 
this process.   

Benefits related to the introduction of e-Cohesion systems for the 
application and contracting phase of projects 
n/a 

n/a 

89% of 
beneficiary 
respondents 
to the survey 
indicated 
that benefits 
exceeded 
costs in 
relation to 
this process.   

93% of 
programme 
authorities 
respondents 
to the survey 
indicated 
that benefits 
exceeded 
costs in 
relation to 
this process.   

Table 2 Simplification and burden reduction  
Part 1 – 
Already 
achieved 

Project beneficiaries 
Quantitative 

Comment 

Programme authorities 
Quantitative 

Comment 

Recurent 

Recurent 

Part 2 - 
Potential 

Recurent 

n/a 

Faster exchange of information 
n/a 

83% of beneficiary 
respondents to the 
survey indicated 
that the e-Cohesion 
systems resulted in 
a faster exchange of 
information with 
programme 
authorities.  

Reduced costs related to the management of projects 
n/a 

n/a 

82% of beneficiary 
respondents to the 
survey indicated 
that the e-Cohesion 
systems resulted in 
reduced costs 
relating to the 
management of 
projects. 

90% of programme 
authority 
respondents to the 
survey indicated 
that the e-Cohesion 
systems resulted in 
a faster exchange 
of information 
with beneficiaries.  

76% of programme 
authority 
respondents to the 
survey indicated 
that the e-Cohesion 
systems resulted in 
reduced costs 
relating to the 
management of 
projects.  

Programme authorities 
Quantitative 

Comment 

n/a 

n/a 

Project beneficiaries 
Quantitative 
Faster exchange of information 
n/a 

Comment 

Having the e-
signature feature 
embedded in the e-
Cohesion system 
could incentivise 
the beneficiaries to 
use even more often 

44 

 
 
 
 
 
 
 
 
the e-Cohesion 
systems as 
compared to 
alternative means of 
communication. 

Recurent 

Reduced costs related to the management of projects 
n/a 

n/a 

Positive outcomes 
have been indicated 
for using e-
Cohesion for the 
application and 
contracting phase of 
projects 

Positive outcomes 
have been 
indicated for using 
e-Cohesion for the 
application and 
contracting phase 
of projects 

45 

 
 
 
 
ANNEX V. STAKEHOLDERS CONSULTATION - SYNOPSIS REPORT 

1.  Strategy 

Objectives 

The general objectives of the consultation activities were to: 

  disseminate the activity and encourage stakeholder participation in the evaluation 

process; 

  ensure transparency of the process; 

  enhance accountability and transparency of European Commission activities; 

  encourage  stakeholders  to  respond  to  the  enquiries  and  the  other  knowledge-

gathering activities to fill information gaps; 

  gather general public end expert stakeholder opinions on the principle evaluation 

findings; and to 

  publicise the main evaluation findings. 

Tools 

The  stakeholders  targeted  by  the  consultation  activities  were  first  mapped  by  and  then 
matched to different consultation tools. In particular,  four sets of activities, each with a 
different purpose, were organised: 

  publication of the evaluation roadmap; 

 

large-scale survey of all users of e-Cohesion systems 

  semi-structured  and  in-depth  interviews  carried  out  by  the  contractor  for  use  in 

case studies; and 

  webinar  organised  by  the  contractor  to  discuss  preliminary  findings  of  the 

evaluation and collect additional input from programme authorities. 

A derogation was  granted from  the obligation to carry out  an open public consultation. 
The derogation was justified by the fact that the evidence underpinning the analysis could 
only  be  collected  from  the  users  of  e-Cohesion  systems  (beneficiaries  and  programme 
authorities).  The  general  public,  not  being  users  of  e-Cohesion  systems,  could  not 
provide any relevant input. 

46 

 
 
 
 
Stakeholder participation 

A summary of the final stakeholder participation figures is provided in Table 1 below. 

Table 1 – Stakeholder reach by activity 

Activity 

Key numbers 

Roadmap 

6  stakeholders  sent  contributions  within  the  deadline  (5 
from  Germany, 1 for Spain; 3 public authorities, 1 non-
governmental  organisation,  1  business  association,  1 
enterprise) 

Large-scale survey 

6 248 responses from beneficiaries 

455 responses from programme authorities 

Interviews 

interviews  conducted  with 
84 
programme authorities and beneficiaries 

representatives  of 

Webinar 

113 participants  

TOTAL 

6 906  stakeholders  reached  (individual  stakeholders 
might have taken part in more than one activity) 

Source: Evaluation of e-Cohesion 2014-2020, final report, (2022). 

2.  Delivery and results 

Roadmap 
The roadmap of the evaluation was published in September 2020 on the ‘Have Your Say’ 
website of the European Commission21 for four weeks. 

The  six  replies  received  from  stakeholders  revolved  around  the  functionalities  of 
e-Cohesion  systems,  including  accessibility,  some  issues  relating  to  data  import  from 
different  file  sources,  the  relevance  of  the  information  collected  in  the  context  of 
evaluation,  the  interoperability  between  e-Cohesion  systems  and  SFC,  and  the 
requirements for using the systems. 

Large-scale survey 
The key findings of the large-scale survey are presented below. 

Respondents are highly satisfied with the functioning of e-Cohesion systems 

21 

https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/12519-Evaluation-of-e-

Cohesion-2014-2020/feedback_en?p_id=8509725  

47 

 
 
                                                           
The survey results indicate the high overall satisfaction of both the authorities and the 
beneficiaries  with  the  functioning  of  e-Cohesion  systems.  This  high  satisfaction  rate 
relates  to  a  broad  range  of  factors  that  were  explored  in  the  surveys  of  authorities  and 
beneficiaries,  including  the  benefits  of  having  the  ability  to  exchange  information 
electronically throughout the key processes and the user-friendliness of the systems. 

When  asked  whether  the  benefits  of  the  e-Cohesion  system  outweigh  their  costs,  the 
results  were  overwhelmingly  positive  in  both  target  groups  (approximately  90%  of 
respondents agree or strongly agree), with only minor differences between the perception 
of efficiency gains brought by e-Cohesion across different processes. This indicates that 
e-Cohesion  systems  help  to  efficiently  support  the  operational  processes  of  authorities 
and beneficiaries. 

Figure 1 - % of respondents who strongly agree or agree that systems’ benefits outweigh 
their cost for tasks related to … 

With  regard  to  the  e-Cohesion  initiative’s  objectives,  the  survey  results  show  that 
authorities  and  beneficiaries  alike  agree  that  e-Cohesion  systems  have  delivered 
improvements and simplification  in  a wide  range  of areas. The  areas in  which both 
groups  of  respondents  say  that  electronic  data  exchange  has  yielded  the  greatest 
improvements  are  data  security  and  privacy,  data  quality  and  integrity,  and  the 
transparency and accessibility of relevant information. 

In  terms  of  user-friendliness,  the  responses  from  authorities  and  beneficiaries  suggest, 
overall,  that  e-Cohesion  systems  are  highly  user-friendly  and  effective,  though  the 
responses do also indicate some areas of weakness.  

The factors relating to user-friendliness that were rated most positively are as follows: 

  With time and more experience using the system, it helps users to carry out their 

tasks more efficiently. 

  The  use  of  the  e-signature  and  its  associated  reduction  of  costs  and  increased 

security. 

 

(Especially  among  authorities)  the  usefulness  of  the  assistance  provided  by  the 
help desk service. 

Users  are,  however,  comparatively  less  satisfied  overall  with  the  responsiveness  and 
stability of e-Cohesion systems. In addition, authorities, in particular, were more likely to 
perceive systems as less flexible regarding their preferred order of steps in the workflow. 

48 

 
 
 
 
Lastly,  around  35%  of  respondents  said  that  the  use  of  e-Cohesion  systems  requires 
extensive training (see Figure 2).  

Figure 2 - % of respondents who strongly agree or agree that … 

e-Cohesion systems provide extensive support for the key processes but the parallel 
exchange of information still takes place outside the systems 

The  findings  of  the  surveys  show  that  the  systems  cover  all  key  processes  (i.e. 
application, implementation, auditing), albeit to a varying extent. The experience of the 
survey  respondents  reveals  that  the  application  process  is  extensively  supported,  even 
though  this  is  not  legally  required  (89%  of  respondents  say  that  e-Cohesion  systems 
support  the  application  process).  In  terms  of  project  implementation,  the submission  of 
payment claims and progress reporting are the core e-Cohesion processes, with 96% of 
beneficiary  respondents  stating  that  those  core  processes  are  supported  by  the  system. 
Auditing, however, is less standardised and thus a considerable amount of data exchange 
takes place outside the system. 

Even though the respondents indicate that the key processes are supported by e-Cohesion 
systems,  including  the  application  phase,  parallel  data  exchanges  outside  e-Cohesion 
systems  have  not  been  eliminated  altogether  and  remain  commonplace,  especially  in 
relation to less standardised auditing activities: 

  Application process: 26% of authorities answered that only up to 50% of all data 
exchanges related to  application take place via the system; 32% of beneficiaries 
answered  that  they  could  apply  through  the  system,  but  some  documents  are 
submitted through other channels. 

 

Implementation:  10%  of  authorities  answered  that  only  up  to  50%  of  all  data 
exchanges  related  to  project  implementation  take  place  via  the  system;  20%  of 
beneficiaries answered that information related to payment claims were submitted 
both  through  the  system  and  via  other  channels;  25%  of  beneficiaries  answered 
that information related to reporting on the progress were submitted both through 
the system and via other channels. 

  Auditing: 26% of authorities answered that only up to 50% of all data exchanges 
related to auditing take place via the system; 43% of beneficiaries answered that 

49 

 
 
information  related  to  management  verifications  or  on-the-spot  checks  were 
submitted both through the system and via other channels. 

Individual  responses  by  authorities  to  open  questions  highlight  that  the  elimination  of 
paper-based processes and the closely linked centralised access to all information is one 
of the most relevant improvements resulting from the electronic data exchange and can 
significantly  contribute  to  the  simplification  of  their  key  processes.  The  elimination  of 
additional  paper  files  is  also  among  the  key  areas  for  development  for  the  upcoming 
programming period mentioned by authorities in open-ended answers. 

Interoperability is a key challenge 

Regarding barriers relevant to the efficient functioning of e-Cohesion systems, the survey 
results show that authorities perceive the lack of interoperability with other systems and 
registers as the most significant challenge (73% of authority respondents indicated it as a 
very  or  somewhat  relevant  challenge),  with  all  other  challenges  being  indicated  as 
significantly  less  relevant.  This  finding  is  also  supported  in  answers  provided  by 
respondents  to  several  open-ended  questions,  showing  that  even  without  prompts, 
interoperability  is  mentioned  as  one  of  the  most  important  challenges  indicated  by 
authorities (see Figure 3). 

Figure 3 - Key challenges for the efficient functioning of systems (% of authorities that 
selected very relevant or relevant) 

A  closer  look  at  the  results  of  the  surveys  shows  that  it  is  much  more  common  for 
systems  to  reuse  and  prefill  information  that  was  previously  entered  in  e-Cohesion 
systems  rather  than  prefill  information  based  on  information  available  from  external 
databases  or  registers:  82%  of  beneficiaries  and  89%  of  authorities  answered  that  the 
system  prefills  information  based  on  what  was  entered  in  previous  steps;  81%  of 
beneficiaries and 88% of authorities answered that the system prefills information based 
on  project  application;  56%  of  beneficiaries  and  46%  of  authorities  answered  that  the 
system prefills information based on information available from various national/regional 
registers. 

50 

 
 
Figure 4 - % of respondents who selected that the e-Cohesion system pre-fills information 
for beneficiaries 

Improvement of e-Cohesion systems is continuous process 

The findings of the surveys also show that the improvement of e-Cohesion systems is a 
continuous process, requiring significant input from the authorities: most authorities say 
that their systems  will either be further developed (55%) or replaced (36%)  in  the next 
programming period. 

The  ongoing  improvement  of  the  systems  is  also  noticed  and  appreciated  by  the 
beneficiaries:  68%  reported  that  the  various  functionalities  and  the  overall  operation  of 
the  systems  are  continuously  improving,  a  finding  which  also  provides  a  strong 
indication of their overall satisfaction with those systems. 

The  introduction  and  maintenance  of  such  systems  is  nevertheless  a  resource-intensive 
activity  for  the  authorities.  Survey  findings  suggest  that  the  initial  deployment  of 
e-Cohesion systems requires more effort from authorities than their subsequent operation, 
with  IT procurement and the tendering process  requiring  the most  effort and resources, 
followed  by  the  setting-up  and  deployment  of  the  systems  and  the  associated 
administrative costs and human resource requirements. 

Interviews 
The  interviews  were  integrated  into  the  six  case  studies  conducted  as  part  of  the 
evaluation. The six case studies are listed in Table 2. 

Table 2. e-Cohesion systems analysed in the case studies 

Case study  e-Cohesion system analysed 
1 
2 
3 

MIS 
e-Toetus 
eMS 

4 
5 

Balcão2020 
SFINGE2020 

51 

Country 
Greece 
Estonia 
Used  by  several  Interreg 
programmes 
Portugal 
Italy 

 
 
 
6 

SL2014 

Poland 

Source: Evaluation of e-Cohesion 2014-2020, final report, (2022). 

The  in-depth  analysis  of  the  six  selected  e-Cohesion  systems  complemented  the 
information  previously  gathered  during  the  initial  desk  research-based  mapping  and 
further in-depth desk research of additional data sources acquired from relevant national 
authorities or the European Commission. 

In  many  cases,  the  functionality  of  the  e-Cohesion  systems  is  an  integral  part  of  the 
transactional monitoring system. Transactional monitoring (activities related to checking, 
approving applications, progress reports and payment claims) in such cases is supported 
by  a  single,  unified  system  sharing  a  common  user  interface  and  database.  This  is  the 
approach  of  systems  like  eMS  (Interreg),  MIS  (Greece),  SFINGE2020  (Italy)  and 
SL2014 (Poland). Consequently, the terms ‘e-Cohesion’ and ‘monitoring’ are often used 
interchangeably. 

There  are  also  situations  in  which  the  e-Cohesion  system  is  a  separate  entity.  In  these 
cases,  the  front-office  functionality  is  independent  of  the  back-office  functionality. 
‘Independent’ in this context means that if even one system is down, the other can still 
operate as normal. Moreover, one system can be individually developed or even replaced 
if technical progress or additional requirements make this favourable. Both systems have 
their own database but they are connected by a bidirectional interface. In such cases, the 
transactional monitoring system pre-dates the e-Cohesion system, which was developed 
later to collect and exchange information, e.g. in the case of Balcão2020 and e-Toetus. 

In the 2014-2020 CPR, e-Cohesion is described as a task of the Member States, while the 
monitoring  system  is  described  as  a  task  of  the  managing  authority.  Except  for 
Balcão2020, all the systems analysed were developed in the same organisational context 
as the monitoring system. 

e-Cohesion  is  a  complex  environment  that  comprises  various  different  aspects.  Thus, 
similarities and differences exist at different levels. 

The  context  in  which  e-Cohesion  systems  operate,  especially  the  fact  that  they  may  be 
used  for  only  one  or  for  several  programmes,  or  by  only  one  Member  State  or  across 
multiple Member States, is also significant. Equally, it is important to take into account 
whether the system is used only for programmes under a single fund or for programmes 
under multiple funds, e.g. for both ERDF and ESF programmes. There are certain fund-
specific aspects, such as the handling of participant data, that affect the performance and 
complexity of the corresponding  IT solutions. Since ESF programmes process sensitive 
data, essential measures are needed to ensure data security and meet the requirements of 
EU  data  protection  rules:  such  data  security  measures  will  also  affect  ERDF  financed 
projects  if  they  are  managed  together  with  ESF  programmes  in  a  single,  integrated  IT 
solution. Only two of the six systems examined – eMS and SFINGE2020 – support only 
ERDF programmes. All the other systems support several different programmes. 

52 

 
The  same  is  true  if  different  groups  of  beneficiaries  use  the  same  IT  solution.  It  is 
different if only public organisations from a single Member State use a system (as is the 
case  for  MIS,  which  is  used  for  both  national  and  regional  programmes)  or  if  a  wide 
range of different types of public and private beneficiaries located in different countries 
use the same system (as is the case for eMS). Another important aspect is if programmes 
using  the  e-Cohesion  system  adhere  to  a  common  or  at  least  similar  management  and 
control  system,  as  these  heavily  define  the  structures,  processes  and  rules  that  apply  to 
the system. 

Webinar 

The main purpose of the webinar was to present the survey findings and gather additional 
feedback from participants on a few selected themes to complement those findings. Thus, 
during  the  webinar,  the  evaluation  team  collected  reactions  and  thoughts  on  the  survey 
findings  and obtained additional input on the key selected themes that  would later feed 
into the final evaluation reports. 

The target group for the webinar consisted of MAs and any other authorities representing 
the  Member  States  (intermediate  bodies,  certifying  authorities,  audit  authorities).  The 
webinar took place on 23 November 2021 via the Zoom platform.  

After a presentation of the findings of the survey on the scope of the use of e-Cohesion 
systems, webinar participants were invited to rank the processes they believed were most 
burdensome  for  beneficiaries  during  the  application  and  implementation  phases.  The 
purpose  of  this  exercise  was  to  establish  whether  their  responses  corresponded  to 
beneficiaries’  views  on  which  system  processes  are  the  most  burdensome,  and  to 
complement  beneficiaries’  views  with  those  of  the  authorities,  as  the  survey  findings 
indicate that the authorities are more frequent users of e-Cohesion systems. According to 
participants, the exchange of information relating to payment claims required by far the 
most effort. This process was ranked highest by over half of the webinar participants who 
voted on this question. Interestingly, the survey findings indicate that the payment claims 
process  has  benefited  most  from  the  introduction  of  the  e-Cohesion  system22. 
Furthermore,  the  survey  findings  show  that  the  payment  claims  process  is  the  one  for 
which data exchange is most likely to be carried out exclusively in the system. 

The  survey  findings  indicate  that  the  key  processes  are,  to  a  large  extent,  being 
implemented via the e-Cohesion systems. However, results also indicate that information 
is  still being exchanged  in  parallel  via other channels  (e.g. email, paper  or similar). To 
explore the topic of parallel data flows, webinar participants were asked if any exchange 
of data still takes  place outside the system.  Almost  a fifth  of authorities in  the webinar 
said  that  all  data  exchange  takes  place  within  the  system,  while  most  said  that  data 

22  Question  for  beneficiaries:  ‘Please  assess  the  following  statement:  the  benefits  (e.g.  reduced 
administrative  burden,  simplified  procedures)  of  the  introduction  of  the  electronic  data  exchange 
system  between  beneficiaries  and  authorities  exceeds  the  associated  costs  (e.g.  the  time  and  effort 
required to use it) for the following processes:…’. 

53 

 
                                                           
exchange  still  takes  place  through  other  channels  as  well.  When  asked  for  further 
comments,  one  MA  representative  mentioned  that  participant  information  required  for 
ESF  is  gathered  outside  the  system.  Another  authority  representative  commented  that 
information  relating  to  procurement  is  still  exchanged  outside  of  the  system,  as  these 
documents are large, and the system is limited in terms of the size of documents it allows 
users to upload. 

Most  webinar  participants  value  the  advantages  of  interoperability  and  the  ‘once-only’ 
encoding  principle. However, a  greater proportion of participants  felt that  the ability to 
reuse  information  entered  by  beneficiaries  in  previous  steps  was  more  useful  than  the 
ability to reuse information from external databases. These findings can be explained by 
the  perceived  level  of  difficulty  associated  with  the  latter:  while  around  one  third  of 
webinar  participants  felt  the  reuse  of  previously  entered  information  is  very  difficult, 
almost  90%  of  them  rated  the  reuse  of  information  from  external  databases  as  very 
difficult. 

The  survey  findings  indicate  that  e-Cohesion  systems  brought  simplification  in  a  wide 
range  of  areas.  All  of  those  areas  were  assessed  very  positively  and  none  stood  out  in 
particular. For this reason, webinar participants were asked to name a single key area in 
which authorities perceived the introduction of the e-Cohesion system had had the least 
impact.  Most  webinar  participants  who  voted  chose  the  reduction  of  costs  relating  to 
project  management,  closely  followed  by  improvements  in  data  quality  and  integrity. 
However, it is important to bear in mind that these areas were assessed positively in the 
survey. 

As  part  of  the  survey,  the  authorities  were  asked  one  question  regarding  the  main 
challenges of and significant barriers to developing an efficient and effective e-Cohesion 
system,  and  one  question  on  the  factors  that  are  essential  for  success  in  developing  an 
efficient and effective e-Cohesion system. On the whole, the challenges were not rated as 
particularly  significant,  except  for  the  lack  of  interoperability  with  external 
applications  and  databases,  which  73%  of  authorities  considered  to  be  a  significant 
issue. All options concerning success factors, however, were rated as significant (at least 
90% of respondents agreed, to varying degrees, with all the success factor statements). 

Clearly, the survey findings are more complex when it comes to the challenges. For this 
reason,  webinar  participants  were  asked  several  questions  designed  to  obtain  a  better 
understanding  of  the  authorities’  perspective  on  this.  Participants  were  asked  to  further 
specify  which  aspects  of  interoperability  represent  a  challenge  to  an  efficient  and 
effective e-Cohesion system. Most webinar participants reported that a lack of common 
definitions  (of  concepts,  information  needs,  data  fields,  forms,  etc.)  at  programme  and 
cross-programme level,  was  the main  barrier to  interoperability with  other systems  and 
registers. 

To  assess  the  preparedness  of  e-Cohesion  systems  for  the  2021-2027  programming 
period, webinar participants were asked  about their future plans for the development of 

54 

 
their respective systems. The planned improvements to e-Cohesion systems cited by the 
authorities can be grouped into three main categories: 

 

Interoperability  –  facilitate  data  exchange  with  other  systems  and/or  national 
registers/databases. 

  User-friendliness  and  usefulness  –  introduce  new  features  such  as  dashboards, 
automatic controls  (validation checks), better communication with  beneficiaries, 
standardisation and simplification. 

  Harmonisation  on  a  national  level  –  use  e-Cohesion  functionalities  for  national 
fund management, provide overviews of fund performance on national level. 

55