COMMISSION STAFF WORKING DOCUMENT EVALUATION Mid-term evaluation of the JASPERS initiative in 2014-2020
Date: 2021-08-31
Institutions: Directorate-General for Regional and Urban Policy, European Commission
Subjects: EU Member State, EU financing, EU initiative, European Commission, European Investment Bank, European Region, accession to the European Union, consultancy, investment project, investment promotion

EN 

  EN 

   EUROPEAN COMMISSION  Brussels, 31.8.2021  SWD(2021) 236 final  COMMISSION STAFF WORKING DOCUMENT  EVALUATION  Mid-term evaluation of the JASPERS initiative in 2014-2020 {SWD(2021) 237 final}  
 
 
Table of contents 

1. 

INTRODUCTION ....................................................................................................... 3 

Purpose and scope ............................................................................................................ 3 

2.  BACKGROUND TO THE INTERVENTION ........................................................... 4 

Description of the intervention and its objectives ........................................................... 4 

Baseline and points of comparison ................................................................................ 10 

3. 

IMPLEMENTATION / STATE OF PLAY .............................................................. 12 

Description of the current situation ............................................................................... 12 

4.  METHOD .................................................................................................................. 16 

Short description of methodology.................................................................................. 16 

Limitations and robustness of findings .......................................................................... 17 

5.  ANALYSIS OF THE EVALUATION CRITERIA .................................................. 19 

Effectiveness .................................................................................................................. 19 

Efficiency ....................................................................................................................... 25 

Coherence ...................................................................................................................... 28 

Relevance ....................................................................................................................... 29 

EU added value .............................................................................................................. 31 

6.  CONCLUSIONS ....................................................................................................... 32 

Effectiveness .................................................................................................................. 33 

Efficiency ....................................................................................................................... 33 

Coherence ...................................................................................................................... 34 

Relevance ....................................................................................................................... 34 

EU added value .............................................................................................................. 35 

7.  DEVELOPMENTS  MADE  DURING  THE  LAST  2  YEARS  OF  THE  2014-2020 
PROGRAMMING PERIOD AND THE OUTLOOK FOR 2021-2027 ................... 36 

ANNEX 1: PROCEDURAL INFORMATION ................................................................ 38 

ANNEX 2: STAKEHOLDER CONSULTATION ........................................................... 44 

ANNEX 3: METHODS AND ANALYTICAL MODELS ............................................... 48 

ANNEX 4: INFOREGIO REPORT “ANALYSIS AND ASSESSMENT OF SERVICES 
AND PERFORMANCE OF JASPERS INITIATIVE 2014-2018” .......................... 52 

1 

 
 
Glossary 

Term or acronym 

Meaning or definition 

CEF 

CF 

CPR 

DG MOVE 

DG NEAR 

Connecting Europe Facility 

Cohesion Fund 

Common Provisions Regulation, Regulation (EU) 
1303/2013 

Directorate General for Mobility and Transport 

Directorate General for European Neighbourhood 
Policy and Enlargement Negotiations 

DG REGIO 

Directorate General of Regional and Urban Policy 

EC 

EIB 

ERDF 

ICT 

IPA 

IQR 

JASPERS 

MoU 

MS 

PSA 

SGA 

SWD 

European Commission 

European Investment Bank 

European Regional Development Fund 

Information and communication technology 

Instrument for Pre-accession Assistance Fund 

Independent quality review 

Joint Assistance to Support Projects in European 
Regions 

Memorandum of Understanding 

Member State 

Post-submission appraisal 

Specific Grant Agreement 

Staff Working Document 

2 

 
 
 
1. 

INTRODUCTION 

Purpose and scope 

The Joint Assistance to Support Projects in European Regions (JASPERS) is an initiative 
of the European Commission (EC) and the European Investment Bank (EIB). It provides 
independent and free-of-charge technical advisory services to all Member States and pre-
accession  countries  in  all  sectors  relevant  for  EU  funded  investments.  JASPERS  was 
created in 2005 and launched in 20061. Since then, it has grown into one of the largest 
initiatives for technical assistance supported by the EC.   

In  line  with  Article  2(4)  of  the  2014-2020  framework  partnership  agreement  for  the 
management  of  the  JASPERS  technical  assistance  facility  concluded  between  the  EIB 
and the EC, which regulated the functioning of JASPERS in the 2014-2020 programming 
period,  the  parties  agreed  to  undertake  a  mid-term  evaluation  of  the  implementation  of 
the agreement. 

This evaluation aims to assess the services provided by JASPERS in 2014-2018 and how 
they  contributed  to  the  development  of  high-quality,  mature  infrastructure  projects  as 
well as to the technical capacity of managing authorities and beneficiaries to prepare such 
projects. The previous evaluation commissioned by the EC on JASPERS was finalised in 
2012  and  covered  2006-2011.  The  current  evaluation  is  intended  to  inform  decision-
makers  at  European  and  national  level  and  to  provide  useful  insights  for  steering  the 
future organisation of and demand for the services concerned.  

The  evaluation  covers  all  JASPERS’  activities  during  the  2014-2018  period  in  all 
countries  that  made  use  of  its  advisory  services  (i.e.  Member  States  and  pre-accession 
countries2). 

It  follows  the  five  evaluation  criteria  defined  by  the  EC’s  better  regulation  guidelines: 
effectiveness, efficiency, coherence, relevance and EU added value. 

The  main  analytical  sources  of  the  staff  working  document  are  the  Inforegio  report 
‘Analysis  and  assessment  of  JASPERS’  services  and  performance  2014-2018’3  and  the 
final  report  of  the  external  study  ‘Mid-term  evaluation  of  the  JASPERS  initiative  in 
2014-2018’4.  The  findings  of  the  European  Court  of  Auditors  special  report  on 
JASPERS5, published in 2018, have also been considered in the scope of this evaluation, 
as have the results from the previous evaluation of the JASPERS initiativecompleted in 
2012. The findings and conclusions of this evaluation are drawn from the cross analysis 
of these documentary sources.  

1 

2006 MoU signed between the EC, the EIB, the European Bank for Reconstruction and Development (EBRD) and 
later the Kreditansalt fur Wiederaufbau (KfW), covering the 2007-2013 programming period. For the 2014-2020 
programming period, the initiative has been continued on the basis of a framework partnership agreement (FPA) 
signed between the EC and the EIB.  

2  Albania, Bosnia and Herzegovina, Kosovo*, the Republic of North Macedonia, Montenegro, Serbia, Turkey. 

*  This  designation  is  without  prejudice  to  positions  on  status,  and  is  in  line  with  UNSCR  1244  and  the  ICJ 
Opinion on the Kosovo declaration of independence. 

3  Violeta Piculescu (DG REGIO), 2021. 
4 
5 

COWI, Ecorys, CSIL (2021). 
https://www.eca.europa.eu/Lists/ECADocuments/SR18_01/SR_JASPERS_EN.pdf 

3 

 
 
                                                 
2.  BACKGROUND TO THE INTERVENTION 

Description of the intervention and its objectives 

The legal documents establishing the functioning of JASPERS (i.e. the Memorandum of 
Understanding for 2007-2013, and the framework partnership agreement for 2014-2020) 
set  out  JASPERS’  overall  objective,  which  was  generally  and  widely  stated  as 
contributing to the improvement of the quality of investments supported by EU funds.  

The  idea  behind  JASPERS  was  first  described  in  a  concept  paper  issued  by  the  EC  in 
20056. At the time, the provision of technical advisory services was justified by: 

- 

- 

the  increased  financial  resources  available  to  fund  certain  types  of  investment, 
which received relatively small amounts of funding in the past; 
a  relative  lack  of  experience  and  capacity  in  the  new  Member  States7  and 
acceding countries8 to develop large-scale projects. 

Consequently, JASPERS’ general aim, as defined by the 2006 MoU, has been to support 
cohesion  policy  implementation  by  increasing  the  resources  available  for  preparing 
projects, and by improving the quality of technical advice available to project promoters, 
thus contributing to the preparation of high quality projects. The expected impact was a 
more rapid  absorption of European Regional Development Fund (ERDF) and Cohesion 
Fund  (CF)  budgets  by  Member  States  and  acceding  countries  finally  contributing  to 
economic growth and environmental development.  

In 20119, JASPERS conducted a needs analysis in preparation of the 2014-2020 period, 
based on inputs from the Member States and the EC. The analysis showed that the types 
of  service  provided  by  JASPERS  would  continue  to  be  relevant  for  at  least  seven 
Member States that had 'high' priority need for support in project preparation in at least 
one of the sectors examined10.  

Therefore,  in  the  2014-2020  programming  period,  JASPERS’  general  objective 
continued to be improving the quality of investments supported by EU funds. However, 
due in part to the new regulatory framework for approval of major projects, its scope was 
extended significantly in terms of Member States covered, the types of service provided 
and the EU funds targeted.  

6 
7 

8 

9 
10 

Final version of 6 September 2005. 
Cyprus, Czechia, Estonia, Hungary, Latvia, Lithuania, Malta, Poland, Slovakia and Slovenia which joined the EU 
on 1 May 2004. 
Bulgaria  and  Romania,  which  joined  the  EU  on  1  January  2007,  as  of  then  having  access  to  ERDF  and  CF 
financing. 
JASPERS (2011), JASPERS Strategy post-2013 needs assessment and preliminary approach. 
14  MS  participated  in  the  survey.  The  full  list  of  analysed  sectors  was:  roads,  public  transport,  water  and 
wastewater,  flood  risk,  climate,  solid  waste,  energy,  knowledge  economy.  More  details  on  the  needs  analysis 
performed  by  JASPERS  services  are  available  in  Section 4.3.1  of  the Final  Report of  the  external  study  on  the 
Mid-term Evaluation of the JASPERS initiative in 2014-2020. 

4 

 
 
 
 
 
                                                 
Figure 1. JASPERS beneficiary countries 

Source: JASPERS website (15.06.2020) 

In  the  2014-2020  programming  period,  JASPERS  assistance  covered  advisory  services 
for  potential  investments  supported  by  the  following  EU  funding  instruments  (i.e. 
mandates): 

-  ERDF and CF: mandate of the Directorate-General of Regional and Urban Policy 

(DG REGIO)11; 

-  Connecting  Europe  Facility  (CEF),  since  2015:  mandate  of  the  Directorate-

- 

General for Mobility and Transport (DG MOVE); 
Instrument  for  Pre-accession  Assistance  Fund  (IPA  II),  since  2016:  mandate  of 
the  Directorate-General  for  European  Neighbourhood  Policy  and  Enlargement 
Negotiations (DG NEAR). 

From  a  procedural  and  operational  perspective,  in  the  2014-2020  programming  period, 
JASPERS activities were regulated by the FPA signed in 2014 between the EC and the 
EIB for the ESIF mandate. This was subsequently amended in 2015 to include the CEF 
mandate, and in 2016 for the IPA II mandate. The FPA is based on the broader Financial 
and  Administrative  Framework  Agreement  (FAFA)  signed  by  the  EC  and  the  EIB  in 
2014. The two documents fix the overall roles and responsibilities of JASPERS as well 
as the roles and responsibilities of other parties involved in managing and implementing 
JASPERS. 

Since  its  creation,  JASPERS  was  managed  by  the  EIB  and  supervised  by  a  Steering 
Committee of representatives of its partner institutions, the EC and the EIB.  

In accordance with the FPA, the role of the Steering Committee was to: 

11  Referred to as the ESIF (European Structural and Investment Funds) mandate. 

5 

 
                                                 
a.  review  and  decide  on  all  significant  policy  issues  related  to  the  operation  of 
JASPERS  including,  but  not  limited  to,  eligibility  of  a  new  country  for  JASPERS 
support; joining of JASPERS by a new partner; issues involving national authorities 
at a high level; communication policy;  

b.  review  and  approve  significant  operational  outputs  and  provide  guidance  on  any 

significant operational issues arising; 

c.  at the request of any of the partners, review the implementation of JASPERS with a 

view to proposing improvements to its supervision or management  

d.  review and approve the annual reports and annual financial statements of JASPERS, 

as well as any other publications reporting on the operations of JASPERS. 

Further on, annual or multiannual specific grant agreements (SGAs) were signed between 
the EIB and the three DGs governing the mandates, as follows: 

-  The ESIF mandate was  based on annual SGAs signed between DG  REGIO and 
the  EIB,  which  defined  the  scope,  priorities,  and  conditions  of  implementation 
and the annual resources allocated to JASPERS; 

-  The CEF mandate was based on multiannual SGAs12, signed between DG MOVE 

and the EIB;  

-  The  IPA  II  mandate  was  based  on  a  multiannual  SGA13,  signed  between  DG 

NEAR and the EIB. 

The  SGAs  reiterate  the  roles  and  responsibilities  of  JASPERS  and  further  specify  the 
types of services that JASPERS can provide under the different mandates. Further to this, 
regular  Steering  Committees  with  the  participation  of  the  EC  and  JASPERS,  provide 
guidance on the development of JASPERS. 

JASPERS  was  organised  in  seven  divisions.  Five  of  them  are  thematic  (for  roads;  rail, 
air,  and  maritime;  water  and  wastewater;  smart  development;  and  energy  and  solid 
waste), the remaining ones being the Independent Quality Review (IQR division) and the 
Networking and Competence Centre (NCC division)14. In addition, JASPERS included a 
Quality Management Unit.  

Over  the  2014-2020  programming  period,  JASPERS  has  operated  its  advisory  services 
on  a  demand  basis,  in  response  to  specific  requests  made  by  Member  States,  with  the 
subsequent agreement of the EC15. Each task implemented by JASPERS is referred to as 
an ‘assignment’. JASPERS formalised its work on assignments in rolling country action 
plans  maintained  throughout  the  period.  According  to  JASPERS’  quality  manual,  the 
country  action  plans  included  working  arrangements  signed  with  the  beneficiary 

12  Signed in 2015 (CEF 1.1) and 2018 (CEF 1.2). The first CEF mandate had the objective of providing JASPERS’ 
technical assistance for the preparation of eligible projects, with the intention of being submitted, to CEF calls for 
the  cohesion  envelope  of  EU  financing.  Countries  and  projects  to  be  assisted  were  identified  based  on  a  list of 
priority projects annexed to the first SGA (Annex I) from 2015. The second mandate expanded the geographical 
scope of assistance to all Member States and focused more on innovative forms of financing (blending). The CEF 
mandate was initiated in the period of the evaluation as a test pilot initiative with a limited budget. 

13  Signed in 2016, divided into IPA II  mandate covering all pre-accession countries for 2016-2021 and the Serbia 
National Programme  Agreement for 2016-2022. For readability, both agreements are referred to in the SWD as 
the IPA II mandate. The IPA II mandate aimed at providing technical support for all phases of preparatory work 
for the projects which might be potentially financed via EU funds, as well as capacity building activities. 

14  The  JASPERS  IQR  unit  was  created  in  2014  and  was  staffed  to  appraise  all  categories  of  major  projects 
developed by the advisory units. The details regarding the activity of Independent Quality Review are explained 
in the following paragraphs.  
In the case of the IPA II mandate, the assignments are undertaken directly by JASPERS, the latter informing DG 
NEAR when a new assignment has started.  

15 

6 

 
                                                 
countries  and  a  list  of  projects  and  areas  where  JASPERS  was  likely  to  be  asked  for 
support.  These  plans  were  updated  in  line  with  requests  for  support  received  from  the 
beneficiary countries.  

As defined in the FPA, the services provided by JASPERS were: 

a)  Technical advisory services (relevant for all mandates, available from 2006); 
b)  Review  function  (relevant  for  the  ESIF  mandate,  introduced  in  the  2014-2020 

programming period); 

c)  Capacity building and horizontal and strategic support (relevant for all mandates, 

available from 2006). 

The  specific  objectives  for  each  type  of  service  were  developed  and  explained  in  the 
quality manual produced by JASPERS16.  

a)  The technical advisory services were designed primarily to improve the projects 
intended  for  EU  financing,  indirectly  contributing  to  their  timely  approval.  The 
advisory services focused on the project preparation phase, but they could also be 
available  during  project  implementation,  with  a  view  of  achieving  a  more  rapid 
absorption of EU funds. These services were provided for all three mandates.  

For the ESIF mandate, technical advisory services concerned both major and non-
major  projects  as  defined  by  the  Common  Provisions  Regulation  (CPR)  (EU) 
1303/201317,  18.  In  the  case  of  the  CEF  and  IPA  II  mandates,  the  distinction 
between major and non-major projects did not apply.  

b)  The review function was relevant solely for the ESIF mandate.  It depended only 
in part on the demand from Member States as JASPERS received a clear mandate 
from  the  EC  to  review  all  new  major  projects  (except  of  phased  projects  under 
Article 103 of the CPR).  This review was linked to the EC’s assessment of major 
projects in the 2014-2020 programing period carried out under Articles 101 and 
102 CPR. The review services were designed to help Member States and the EC 
assess whether or not major projects fulfil the quality regulatory criteria required 
for  EU  funding  on  a  sound,  economic  and  technical  basis,  by  providing  clear 
statements on a major project’s feasibility and economic viability. 

16  Version 1.4 has been in force since June 2019. The first version was produced in November 2010.  
17  Major projects were defined as projects for which the total eligible costs exceed EUR 50 million, except for the 
transport sector for which the threshold is EUR 75 million – see Article 100 of the CPR. Non-major projects were 
projects  which  do  not  fulfill  the  regulatory  requirements  for  a  major  project.  Regulatory  requirements  for  the 
application for EU funding for a major project were laid down in Article 101 CPR. 

18  Applications for EU financing for a major project were subject to EC approval, which was not requested for non-
major projects approved at national level during the 2014-2020 programming period. Applying for EU funding for 
a major project entailed detailed preparation of project technical specifications, a  feasibility study, a cost-benefit 
analysis  and  an  environmental  impact  assessment.  The  project  application  had  to  summarise  not  only  the 
technical,  economic  and  financial  feasibility  of  the  project  documented  in  the  above  studies,  but  also  aspects 
relating  to  the  alignment  of  the  project  with  EU  cohesion policy  objectives,  and  with  the  EU  requirements  and 
standards  regarding  environmental  impact,  adaptation  and  mitigation  measures  for  climate  change,  and  EU 
competition rules.   
The  technical  advisory  services  targeting  non-major  projects  were  organised  in  a  similar  manner.  Although  the 
CPR did not mention regulatory requirements for non-major projects, most of the beneficiaries chose to apply a 
similar approach as the one for major projects (i.e. development of a feasibility  study, cost-benefit analysis). In 
addition,  the  non-major  projects  were  expected  to  comply  with  the  EU  requirements  and  standards  regarding 
environmental impact, adaptation and mitigation measures for climate change, and EU competition rules. 

7 

 
 
  
 
                                                 
For  the  2014-2020  programming  period,  as  proposed  by  the  EC,  legislators 
provided a new system for appraising and approving new major projects funded 
under EU cohesion policy. It suggested two alternative systems. 

1.   In line with Article 102(1) CPR (notification procedure), Member States 
could notify a selected major project to the EC, accompanying it with an 
independent  quality  review  (IQR)  report  performed  by  independent 
experts19.  

2.  Under  Article  102(2)  CPR  (submission  procedure),  Member  States  had 
the  possibility  to  submit  the  major  project  application,  together  with  the 
full  project  documentation,  directly  to  the  EC  for  assessment  and 
approval20,  21.  This  is  similar  to  the  procedure  applied  during  the  2007-
2013 programming period. 

Unlike in the 2007-2013 procedures, in the 2014-2020 programming period, the 
EC  had  the  possibility  when  assessing  a  major  project  to  request  a  post-
submission appraisal (PSA) report from independent experts.  

Under  the  notification  procedure  (Article  102(1)),  Member  States  submitted  all 
project  documentation  directly  to  JASPERS’  IQR  division  for  assessment. 
JASPERS  IQR  then  issued  its  opinion  as  to  whether  or  not  the  project  met  the 
required  quality  criteria,  and  confirmed  the  project’s  feasibility  and  economic 
viability.  The  IQR  report  included  an  assessment  covering  all  the  requirements 
and was sent to the Member State upon completion. If the IQR conclusions were 
positive, the Member State notified the project application, together with the IQR 
report,  to  the  EC  for  approval.  Under  the  notification  procedure  the  Member 
States needed to  wait  until JASPERS’ assessment was complete and the  project 
application  was  submitted  to  the  EC  before  being  able  to  certify  expenditure. 
With the entry into force of the Omnibus Regulation (EU, Euratom) 2018/1046, 
even  projects  subject  to  the  notification  procedure  could  have  had  certified 
expenditure  as  soon  as  JASPERS’  IQR  division  confirmed  the  project 
documentation and application were complete. 

Under  the  submission  procedure  (Article  102(2)),  Member  States  submitted  the 
full  project  documentation  to  the  EC,  together  with  the  application  for  funding,  
since, in this case, it was the EC that was responsible for assessing the quality of 
the  project  submitted.  Upon  receiving  the  project  documentation  and  the 
application for EU financing, the EC asked JASPERS’ IQR division to perform a 
PSA  and  to  assess  all  aspects  of  the  project,  except  for  the  alignment  of  the 
project  with  the  operational  programme;  State  aid  rules;  environmental  issues; 
and the completeness of the project application (which were assessed directly by 
the EC). As regards climate change aspects, the PSA report assessed the project’s 
alignment  with  climate-proofing  requirements  established  in  a  Memorandum  of 

19  The requirements for the independent experts carrying out the quality review and the steps involved in this are set 
out  in  Commission  Delegated  Regulation  (EU)  480/2014.  The  template  for  the  notification  of  a  major  project 
based on Article 102(1) CPR is included in Commission Implementing Regulation (EU) No 1011/2014. 

20  Article 101 CPR lists the documents necessary for the approval of a major project submitted under the procedure 

based on Article 102(2). 

21  A third procedure based on Article 103 CPR refers to phased projects. These are projects approved in 2007-2013 
and  which  are  continued  with  a  distinct  phase  in  2014-2020.  The  evaluation  focuses  on  project  preparation  in 
2014-2020; it therefore focuses less on phased projects as they were already prepared in the previous period, and 
are therefore not comparable with the new major projects.  

8 

 
 
 
 
 
                                                 
Understanding agreed by DG REGIO and DG CLIMA in 2015. JASPERS’ IQR 
division  issued  its  opinion  on  project  quality  in  the  form  of  a  PSA  report  sent 
directly to the EC. As this review had a more limited scope than the IQR report, a 
PSA  report  was  less  comprehensive;  it  distinguished  only  between  critical  and 
non-critical issues identified for project quality and documentation for the topics 
covered22. Information provided in the PSA report was not the only source the EC 
used to assess a major project. Member States could certify expenditure for major 
projects  submitted  under  Article  102(2)  as  soon  as  the  EC  had  determined  the 
completeness of project documentation and application in SFC2014.  

c)  Capacity  building  and  horizontal  strategy  support  was  available  for  all  three 
mandates. The aim was to strengthen Member States’ capacity to identify, prepare 
and  implement  projects,  and  to  provide  support  in  preparing  national/regional 
strategies  for  the  implementation  of  EU  cohesion  policy.  From  an  operational 
perspective, 
implementation  of 
assignments  for  capacity  building  was  similar  to  that  for  the  technical  advisory 
function.  

the  process  of  assignment  creation  and 

In the context of this type of service, in 2012 JASPERS established the JASPERS 
networking  platform23.  The  platform  complemented  JASPERS’  project  advisory 
operations  by  implementing  knowledge  sharing  and  capacity  building  activities, 
as well as fostering dissemination of good practices and exchange of experiences 
among  all  Member  States,  pre-accession  countries  and  other  JASPERS 
stakeholders. Its website included information on JASPERS’ past and upcoming 
events,  annual  programmes  from  2012/2013  to  the  present,  and  a  library  of 
JASPERS’ guidance and capacity building documents.  

Figure 2 showcases (in a simplified manner) the intervention logic of JASPERS covering 
all types of services, i.e. technical  advisory services to  projects,  horizontal  and strategy 
support, capacity building, and review services (IQR and PSA). 

22  A critical issue is defined as an instance of not fulfilling the minimum criteria for quality review of major projects 
listed in the Commission Delegated Regulation (EU) No 480/2014, or of non-compliance with relevant EU legal 
obligations. A non-critical issue applies when the quality standards for the project, or information requirements, 
are  not  fully  met  but  this  does  not  affect  the  overall  positive  assessment  of  the  project.  Based  on  JASPERS’ 
quality manual (2018). 
23  www.jaspersnetwork.org  

9 

 
 
 
 
                                                 
Figure 2. Intervention logic of the JASPERS initiative for the 2014-2020 period

Source: Mid-term evaluation of the JASPERS initiative in 2014-2020 

Since  the  2007-2013  programming  period,  the  performance  of  JASPERS  services  has 
been  measured  through  key  performance  indicators  (KPI),  such  as  numbers  of 
completions,  cumulative  investment  volumes  of  approved  projects  supported  by 
JASPERS24, and EC approval times for major projects25. The KPIs were complemented 
for  the  2014-2020  period  by  a  result  management  framework  with  four  dimensions 
(institutional; customer; financial & risk; internal processes; learning & people) and the 
European Foundation for Quality Management (EFQM) model. 

Baseline and points of comparison  

The baseline for this evaluation is the 2007-2013 period, when JASPERS was operating 
with  a  smaller  quantitative  scope  and  geographical  coverage.  The  legal  provisions 

24  The EU amount and cumulative investment linked to the project. 
25  The time elapsed between the submission of the major project to the EC and its approval by the EC.  

10 

 
 
                                                 
regarding major projects have also evolved between the two programming periods26. The 
baseline was chosen to ensure comparability and consistency with regard to the concept 
of  ‘programming  period’  and  the  number  of  years  covered  by  the  evaluation.  The 
previous evaluation of the JASPERS initiative, completed in 2012, informs the points of 
comparison.  The  detailed  numerical  information  for  both  programming  periods  is 
available in Tables A.3.1 – A.4.6 of Annex 4 to the SWD.  

This section provides the comparison point used in the analysis included in Section 5 and 
informs the findings under the different evaluation criteria used to assess the performance 
of  the  instrument  over  the  2014-2018  period.  The  points  of  comparison  are  used 
primarily for the effectiveness and relevance criteria. A proxy analysis was performed for 
the  efficiency  criterion.  It  refers  to  a  new  type  of  service  (i.e.  the  review  service) 
introduced in the 2014-2020 period and the analysis of the overtime trend of human and 
financial resources. The coherence and EU added value criteria are not influenced by the 
points of comparison as the analysis is self-standing in relation to the 2014-2018 period 
of the evaluation. 

Over the period 2007-2013, the EC invested EUR 132 million in the JASPERS initiative, 
representing  71%  of  the  total  costs.  The  remaining  29%  was  covered  primarily  by  the 
EIB with in-kind support (i.e. for additional technical expertise, logistics of the operating 
offices, etc.)  27. During this period, JASPERS contributed to mobilising more than EUR 
32  billion  ERDF  and  Cohesion  Fund  resources  by  assisting  383  major  projects  which 
were subsequently approved by the EC28. 

As mentioned earlier, during the 2007-2013 period, JASPERS’ work consisted of solely 
the ESIF mandate for DG REGIO. JASPERS completed around 790 assignments for EU 
cohesion  policy  projects  and  six  assignments  for  IPA-funded  projects.  82%  of  these 
activities  were  related  to  the  advisory  function  for  preparing  major  and  non-major 
projects  and,  to  a  lesser  extent,  for  project  implementation.  The  remaining  activities 
consisted  primarily  of  horizontal  and  strategic  support.  The  water  and  wastewater 
treatment sector was predominant in JASPERS’ portfolio, accounting for one third of all 
assignments  for  the  period,  followed  by  the  knowledge  economy,  energy  and  waste 
management combined. Capacity building activities and horizontal and strategic support 
focused largely on the same sectors as well as on multi-sector assignments.  

As  regards  the  distribution  of  assignments  between  countries,  76%  of  JASPERS’ 
activities  in  2007-2013  were  concentrated  in  five  Member  States:  Poland,  Romania, 
Bulgaria, Czechia and Hungary. The remaining assignments were distributed across nine 
other Member States and three IPA countries.  

The  highest  number  of  assignments  (160  accounting  for  20%  of  all  assignments  of  the 
period) was completed in 2010 (the fourth  year of the programming period). Moreover, 
by the fifth  year of the period (i.e. the  year equivalent to the reference end-date for the 
evaluation of the 2014-2020 period), JASPERS had carried out 68% of all assignments 
completed in 2007-2013. 

26  For example, the 2014-2020 legal framework introduced a more rigorous and comprehensive cost-benefit analysis 
methodology for major projects that defined new elements such as content of option analysis, and climate change 
proofing - Annex II to Commission Implementing Regulation (EU) No 207/2015. 

27  The European Bank for Reconstruction and Development (EBRD) also provided a small number of experts.  
28  The total number of major projects (assisted and non-assisted for preparation) adopted for the period 2007-2013 is 
767, mobilising close to EUR 65 billion from the ERDF and Cohesion Fund. This, however, understates the total 
amount  of  EU  funding  for  the  projects  assisted,  as  it  does  not  include  the  support  for  non-major  projects,  for 
which the EC does not have an estimate (non-major projects are approved directly at the level of MAs). 

11 

 
                                                 
3. 

IMPLEMENTATION / STATE OF PLAY 

Description of the current situation  

During  the  2014-2018  period,  the  EC  planned  to  invest  close  to  EUR 150 million  in 
JASPERS29,  representing  around  81%  of  the  total  costs  of  the  initiative  (EUR  175 
million in total). The remaining amount was to be covered primarily by the EIB30.  

The  technical  advisory  support  provided  by  JASPERS  helped  to  mobilise  almost 
EUR 23 billion  of  ERDF  and  Cohesion  Fund  resources  through  142  major  projects 
assisted during preparation and which were adopted or approved by the EC31. A further 
EUR 2.2 billion from the CEF cohesion budget was mobilised, with the projects assisted 
by  JASPERS  and  approved  for  CEF  funding  under  the  CEF1.1  mandate,  and  EUR  82 
million for 10 IPA projects assisted and approved for IPA funding by the end of 2018. 

Over  this  reference  period,  JASPERS’  portfolio  included  1096  assignments  for  all 
mandates  and  services.  Figure  3  illustrates  the  distribution  of  these  assignments  by 
mandate and type. 

Figure 3. JASPERS’ portfolio by mandate and assignment type, 2014-2018 

Note: CB = capacity building, HSS = horizontal and strategic advice ; data covers 1096 assignments 
Source:  Inforegio  report  “Analysis  and  assessment  of  JASPERS’ Services  and  Performance  2014-2018”,  JASPERS’ 
JADE database, December 2018 

The ESIF mandate was the largest in JASPERS’ portfolio, with close to 960 assignments, 
followed by the IPA mandate with around 100 assignments.  

JASPERS’  traditional  function  of  technical  advisory  services  –  primarily  project 
preparation for the three mandates – represented almost two-thirds of its portfolio.  

The activities for capacity building and horizontal and strategic support mostly focused 
on  cross-cutting  topics  relevant  for  different  sectors  (e.g.  environmental  requirements, 
cost  benefit  analysis,  climate  proofing),  and  country-level  issues.  They  were  linked 
primarily to the ESIF and the IPA mandates. 

Finally,  the  review  function  was  specific  to  the  ESIF  mandate  and  included  project 
assessment services provided to Member States (for IQR assessments – 60 assignments) 
and to the Commission (for PSA assessments – 150 assignments). All new major projects 

29  Calculation based on the SGAs. Following decommitments, the EC’s investment was close to EUR 140 million.  
30  The European Bank for Reconstruction and Development (EBRD) also provided a small number of experts.  
31  As noted above, the EU resources mobilised by ESIF projects assisted by JASPERS advisory are understated, as 
they do not include the EU funding for the non-major projects assisted and approved at national level, for which 
the EC does not have an estimate of the total funding involved. The EC had approved 210 new major projects by 
the end of October 2019. 

12 

 
 
 
 
                                                 
submitted  by  Member  States  in  the  period  evaluated  were  reviewed  by  JASPERS  IQR 
division using one of the two procedures mentioned above.  

Figure 4. Distribution of JASPERS’ assignments by sector, 2014-2018 

Note: CB = capacity building, HSS = horizontal and strategic advice ; data covers 1096 assignments. 
Source:  Inforegio  report  “Analysis  and  assessment  of  JASPERS’ Services  and  Performance  2014-2018”,  JASPERS’ 
JADE database, December 2018 

Assignments for the rail, air and maritime transport sector accounted for the largest share 
of JASPERS’ portfolio (33%). The majority of these assignments (70%) were related to 
technical advisory services for project preparation. The water  and wastewater treatment 
sector  was  the  second  largest,  with  a  substantial  number  of  assignments  for  technical 
advisory  services.  The  multi-sector  assignments  related  primarily  to  capacity-building 
activities, including technical support papers, workshops and training (both multi-country 
and  country-specific)  and  were  on  topics  of  general  relevance,  such  as  cost-benefit 
analysis, climate adaptation and mitigation, and environmental requirements. 

13 

 
 
 
 
 
Figure 5. Distribution of JASPERS’ assignments by country, 2014-2018 

Note: CB = capacity building, HSS = horizontal and strategic advice ; data covers 1096 assignments 
Source:  Inforegio  report  “Analysis  and  assessment  of  JASPERS’ Services  and  Performance  2014-2018”,  JASPERS’ 
JADE database, December 2018 

Until  end-2018,  JASPERS  initiated  assignments  in  21  Member  States  and  5  IPA 
countries. The countries with the largest number of assignments were Poland, Romania, 
and  Croatia,  accounting  together  for  52%  of  all  assignments  in  JASPERS’  portfolio. 
Poland  also  stood  out  for  the  relatively  large  number  of  assignments  for  the  review 
assignments related to major projects.   

As  regards  timing,  76%  of  assignments  were  started  in  2014  and  the  following  years, 
while  the  remaining  24%  were  started  before  2014,  in  preparation  for  the  2014-2020 
programming period. By December 2018, 51% of assignments had been completed and 
9% administratively closed32, while the rest were still ongoing.  

Figure 6. Ongoing assignments by main type by year, 2014-2018 

Note: CB = capacity building, HSS = horizontal and strategic advice ;  data covers 1066 assignments 
Source:  Inforegio  report  “Analysis  and  assessment  of  JASPERS’ Services  and  Performance  2014-2018”,  JASPERS’ 
JADE database, December 2018 

The  yearly  data  for  ongoing  assignments  indicate  that,  already  by  the  first  year  of  the 
2014-2020  programming  period,  JASPERS’  portfolio 
than  300 
assignments for technical advisory, capacity building and horizontal and strategic support 

included  more 

32  Administratively closed assignments are assignments that  were  closed before their objective could be achieved. 
The timeline and reasons for administrative closure of assignments are analysed in detail in Section 5, in the sub-
section on efficiency. 

14 

 
 
 
 
 
                                                 
services33.  The  assignments  for  the  review  function  were  first  requested  in  2015  and 
increased over time as project preparation was completed and projects were submitted for 
EU co-financing.  

The  number  of  ongoing  assignments  increased  steadily  each  year  until  2017  and 
decreased for the first time in 2018. By end-December 2018, there were 435 assignments 
still ongoing.  

Figure 7. Completed assignments by main type by year, 2014-2018 

Note: CB = capacity building, HSS = horizontal and strategic advice ; data covers 535 assignments 
Source: Inforegio report “Evaluation of JASPERS’ Services and Performance 2014-2018”, JASPERS’ JADE database, 
December 2018 

The  pace  of  assignment  completion  accelerated  significantly  from  2017.  For  the  first  2 
years,  JASPERS  completed  assignments  relating  only  to  technical  advisory  or  capacity 
building and horizontal/strategic support services. From 2016, it also reported completing 
assignments linked to the review function (IQR and PSA).  

State of implementation – ESIF mandate 

The financial resources mobilised for JASPERS’ services under the ESIF mandate over 
the  2014-2018  period  amounted  to  EUR  175  million,  representing  84%  of  the  ESIF 
mandate  budget  for  2014-2020.  The  EC  contribution  covered  81%  of  the  total  cost  of 
these services (EUR 141 million).  

Considering  the  overall  EC  financial  contribution  to  JASPERS’  services  for  the  three 
mandates over the 2014-2018 period, the ESIF mandate provided the main share of the 
budget (97%). 

JASPERS’  portfolio  for  the  ESIF  mandate  in  the  2014-2020  period  up  until  December 
2018 included close to 960 assignments. These were primarily for technical assistance for 
project  preparation  and,  to  a  much  lesser  extent,  implementation  (59%  combined), 
followed  by  horizontal  activities  and  capacity  building  (23%)  and  its  new  review 
function (18%). 

In terms of the distribution of the ESIF assignments across countries, 57% were carried 
out in three main beneficiary countries (Romania, Croatia, and Poland), 5% were multi-
country assignments, and the rest were distributed across 19 other Member States. In the 
previous  period,  59%  of  JASPERS’  activities  for  this  mandate  were  similarly 

33  As explained, many of these assignments were started prior to 2014, in preparation for the period.  

15 

 
 
 
                                                 
concentrated  in  three  main  beneficiary  countries  (Poland,  Romania,  Bulgaria),  with  the 
rest of activities distributed across 11 other Member States. 

State of implementation – CEF mandate 

EUR  1.4  million  was  allocated  for  JASPERS  technical  assistance  services  under  the 
CEF1.1 mandate in 2015-2018, of which the EC contribution covered 90%.   

According to the final annual report of the CEF1.1 mandate34, JASPERS carried out 31 
assignments in support of the preparation of 29 CEF projects35, with a large majority of 
them  in  the  rail  sector.  At  country  level,  the  initial  agreement  for  the  first  CEF  pilot 
(CEF1.1) was that JASPERS would support projects chosen from the list of 58 projects 
in  seven  cohesion  countries  included  in  Annex  I  to  the  Special  Grant  Agreement.  The 
actual  number  of  projects  supported  was  determined  by  the  budget  available  for  the 
JASPERS assistance pilot for the CEF1.1 mandate.   

State of implementation – IPA mandate 

The resources allocated under the IPA mandate amounted to just over EUR 4 million in 
2015-2018. This was 64% of the total IPA budget allocated36.  

JASPERS’  portfolio  for  IPA  included  102  assignments  up  to  end-December  2018.  Of 
these,  38%  had  been  completed,  50%  were  still  ongoing,  and  the  rest  had  been 
administratively closed. The IPA assignments were not only for investment projects, but 
they were also used to help national authorities in guiding and supervising contracts for 
technical assistance. 

As  regards  the  horizontal  and  strategic  support  for  the  IPA  mandate,  80%  of  the  20 
assignments  in  the  portfolio  at  end-2018  had  been  completed  or  were  still  ongoing  in 
Montenegro, Republic of North Macedonia and Serbia. Half of these were on workshops 
and advisory support on cost-benefit analysis and on assistance with  national  strategies 
and  master  plans.  The  remaining  assignments  had  a  variety  of  objectives,  such  as 
preparing a project pipeline and analysis and advice at sector level. 

More  information  on  the  state  of  implementation  of  each  JASPERS’  mandate  can  be 
found in Section 5 of Annex 4 to this SWD. 

4.  METHOD 

Short description of methodology 

This evaluation is based on an extensive analysis of documentary sources and qualitative 
and  quantitative  data  on  the  activities  performed  by  the  JASPERS  services  during  the 
2014-2018 period.  

The documentary sources include the strategic agreements governing the functioning of 
JASPERS, the documents produced by JASPERS in relation to the assignments, activity 

34  The CEF1.2 mandate started in 2018, so there is only a  few  months of overlap with the period covered by this 

evaluation and, accordingly, limited scope to assess it. 

35  An additional three projects supported during CEF1.1 were postponed for CEF1.2 and are so not included in the 

CEF1.1 mandate. 

36  Of these, the budget execution rate is highest for the Serbia mandate (75% as compared with 59% for IPA II). 

16 

 
                                                 
reports  for  the  CEF  mandates,  annual  JASPERS  activity  reports,  country  action  plans, 
and documents produced by the EC.  

The quantitative data were collected from JASPERS’ dataset of assignments (the JADE 
database),  the  EC’s  database  which  includes  information  on  the  major  projects 
(SFC2014),  EC  databases  with  information  on  CEF  and  IPA  projects,  and  JASPERS’ 
annual implementation reports.   

The  qualitative  data  were  collected  via  an  online  targeted  consultation,  a  seminar 
organised in Brussels in September 2019 with relevant stakeholders, and interviews with 
MA representatives and EC officials.  

The analytical comparisons in this evaluation cover the following aspects: 

-  For  the  comparisons  between  the  2007-2013  and  2014-2020  programming 
periods,  the first 5 years are used as the basis for the calculations, as this is the 
number of years covered by the current mid-term evaluation. 

-  Projects  assisted  by  JASPERS  and  those  not  assisted  are  compared  using 
econometric  models  in  order  to  estimate  JASPERS’  contribution  to  project 
preparation,  the  subsequent  performance  of  the  assisted  projects  in  the  approval 
process for EU funding and later in implementation. 

Limitations and robustness of findings 

This  SWD  relies  only  to  a  limited  extent  on  the  final  report  produced  by  the  external 
evaluators  commissioned  by  the  EC  to  assess  JASPERS.  The  data  of  the  evaluation 
report, on which this SWD is based, relate to the description of JASPERS services, the 
six  country  case  studies  and  the  qualitative  data  collected  from  stakeholders.  The 
evidence  compiled  in  the  final  report  was  not  considered  sufficiently  robust  and 
representative,  and  so  the  conclusions  were  rather  general  in  nature37.  It  was  therefore 
decided  to  carry  out  a  new  round  of  data  collection  and  analysis  in  order  to  reach 
conclusions that are backed by sufficiently strong evidence and representative of all the 
assignments undertaken. This new and complete analysis is annexed to the SWD as the 
Inforegio report ‘Analysis and assessment of JASPERS’ Services and Performance 2014-
2018’  –  Annex  4.  This  analysis  is  used  to  inform  all  the  findings  related  to  the 
quantitative analysis. References throughout the document indicate the different sources 
of information where the detailed analysis can be consulted.  

Most  of  the  findings  of  this  evaluation  can  be  considered  robust  as  they  are  based  on 
evidence collected mostly for the relevant population as a whole. Tables A.2.1.1, A.2.1.2, 
and A.2.1.3 of Annex 4 present a complete picture of the information that was collected 
and used for the analysis under each of the evaluation criteria, making a link between the 
investigated  topic,  the  source  of  information  and  the  percentage  of  the  relevant 
population  covered  by  the  collected  data.  As  shown  in  these  tables,  in  the  majority  of 
cases  the  entire  population  was  covered  by  the  collected  data,  which  ensures  the 
reliability of the findings put forward by the evaluation.  

In some instances, the findings are affected by the following limitations.  

37  The analysis and findings for the effectiveness, efficiency and EU added-value criteria were mostly supported by 

circumstantial evidence or statements of a qualitative nature collected from stakeholders. 

17 

 
 
                                                 
a)  The  objectives  of  the  intervention  were  stated  generally  in  the  original 
documents, a weakness which was also documented by the Court of Auditors 
in the Special Report on JASPERS (2018).  The specific objectives that were 
linked  to  each  type  of  service  provided  by  JASPERS  were  listed  in  the 
Quality  Manual  drafted  by  the  JASPERS  services,  but  no  operational 
objectives were defined.    

b)  At the end of 2018, it was planned to update the monitoring system in place to 
better  reflect  the  performance  of  JASPERS  services,  but  this  update  was 
scheduled for 2019 (outside the period covered by this evaluation).  

c)  The  needs  analysis  performed  before  the  start  of  the  2014-2020  period  used 
mostly qualitative data collected directly from the stakeholders concerned (the 
Member States and the EC).  

d)  Because  of  the  need  for  the  EC  to  carry  out  a  new  analysis  and  data 
collection, there was limited time for deepening  the analysis of the need  for 
JASPERS’ services and of the objectives of the intervention.  

e)  The  analysis  of  JASPERS’  advisory  for  non-major  projects  remains  partial 
because of the lack of follow-up data on what happened to these projects once 
JASPERS’  advisory  was  completed.  JASPERS  does  not  collect  this  type  of 
data,  and  only  two  of  the  six  Member  States  sampled  for  the  evaluation 
responded to the EC’s data requests.   

f)  As regards the efficiency of the time allocation for JASPERS’ activities, the 
SWD  analyses  only  the  assignments  completed  in  2017-2019.  Before  2017, 
data  on  the  actual  person-days  the  JASPERS  experts  spent  on  each 
assignment  were  not  recorded.  A  more  complete  dataset  for  annual  actual 
work per assignment over the whole period would have been needed for a full 
estimation of the scope for a more optimal allocation of resources.  

g)  The  analysis  of  the  effectiveness  of  the  IQR  review  service  is  also  limited 
because  of  a  lack  of  data.  The  IQR  process  was  less  formal  than  the  PSA 
assessment  made  for  the  EC,  in  the  sense  that  much  of  the  information 
relating to the IQR review of projects was conveyed by email from JASPERS 
to the national authorities, as such in a non-structured form. As a result, it was 
not  possible  to  carry  out  a  systematic  analysis  of  the  effectiveness  and 
coherence of this service relative to the advisory services (i.e. analysis of the 
issues  signalled  during  IQR  review  versus  the  issues  signalled  during 
technical advisory).  

h)  The analysis of EU added-value is limited to the interpretation of qualitative 
data. The information used to assess this criterion was collected in an online 
targeted  consultation  addressed  to  stakeholders.  However,  the  opinions 
collected from stakeholders make reference  more to the general  added-value 
of  the  JASPERS  services  as  compared  to  similar  services  available,  rather 
than to the perceived EU added-value. 

18 

 
Given the limitations listed above, a number of proxy  analyses38 were undertaken in 
order  to  assess  the  effectiveness,  efficiency  and  relevance  of  the  intervention.  The 
details of these are presented in the corresponding sections on the evaluation criteria 
in  Chapter  5  of  the  SWD.  A  comparative  analysis  was  also  made  between  projects 
assisted by JASPERS and those not assisted in order to judge the performance of the 
intervention.  

More  details  relating  to  the  data  sources  and  methods  used  are  listed  in  Section  4  of 
Annex 4 to the SWD. 

5.  ANALYSIS OF THE EVALUATION CRITERIA 

The evaluation carried out in the current SWD follows the five evaluation criteria listed 
by  the  better  regulation  guidelines.  It  was  decided  to  present  the  analysis  by  criterion 
without  referring  to  individual  evaluation  questions.  This  decision  was  justified  by  the 
fact  that  the  SWD  relies  primarily  on  the  analysis  performed  internally  by  the  EC. 
Because  of  the  limitations  indicated  in  the  previous  chapter,  the  SWD  relies  only  to  a 
limited extent on the external evaluation report.  

The analysis by evaluation  criterion is  mainly focused on the ESIF mandate. The ESIF 
mandate  encompasses  almost  90%  of  the  assignments  in  the  reference  period  and 
accounts  for  97%  of  the  EC’s  financial  contribution  to  the  functioning  of  JASPERS  in 
2014-2018. The CEF mandate was initiated during the evaluation. The evaluation was a 
pilot  initiative  aimed  at  testing  the  relevance  of  the  technical-assistance  services 
proposed, so the evaluation findings are therefore limited. For the IPA mandate, most of 
the  assignments  were  still  ongoing  at  the  end  of  2018,  and  therefore  the  evaluation 
findings are based on a limited number of completed assignments. 

Effectiveness 

The analysis of effectiveness aims at assessing how successful the intervention has been 
in achieving, or progressing towards, its objectives.  

The  main  objective  of  JASPERS  was  to  help  Member  States  prepare  large  investment 
projects  (mainly  major  projects)  to  meet  EU  standards  and  comply  with  EU  regulatory 
requirements.  The  advisory  function  of  JASPERS  was  intended  to  contribute  to  the 
preparation  of  good  quality  projects,  which  could  then  be  proposed  by  Member  States 
and succeed in a timely  way to  receive EU funding.  Ultimately,  well prepared projects 
should  reach  the  stage  of  implementation  as  quickly  as  possible,  contributing  to 
achievement of cohesion policy goals.  

Under  the  review  function,  JASPERS  provided  an  independent  expert  assessment  of 
major projects presented for EU financing.  

Under the horizontal advice and capacity building function, JASPERS: (i) assisted MS in 
preparing  strategies  and  national  plans  for  investments  supported  by  EU  funds;  (ii) 
helped to identify viable project pipelines for operational programmes; and (iii) helped to 
strengthen national technical capacity in preparing and implementing projects.  

To assess effectiveness, an analysis was carried out based on the following indicators:  

38  A  proxy  analysis  aims  to  provide  an  indirect  measure  of  relevant  outcomes.  It  is  commonly  used  when  direct 

measures of outcomes are unobservable and/or unavailable. 

19 

 
                                                 
- 

- 

- 
- 

the amount of EU resources mobilised and the number of major projects approved 
compared to the previous programming period; 
the number of issues signalled on the quality of the projects submitted for funding 
– i.e. the rate of interruptions in the approval of major projects; 
compliance with the regulatory deadlines (only for review services); 
the  development  of  technical  capacity  at  Member  State  level,  understood  as  a 
reduction  in  the  requests  for  advisory  services  between  the  two  programming 
periods. 

On  the  ESIF  mandate,  JASPERS’  portfolio  of  assignments  for  technical  advisory  in 
2014-2018 included 414 assignments for major projects. Of these assignments, 178 were 
completed by the end of 2018 and an additional 48 were completed in 2019. Of the 178 
major projects  covered by  the  completed  assignments,  158 were adopted  by the  EC  by 
October 2019. The adopted projects assisted by JASPERS involved a total EU funding of 
EUR 23.9  billion  (ERDF  and  Cohesion  Fund  combined),  representing  65%  of  EU 
resources mobilised for major projects adopted by the EC during the reference period of 
the evaluation.  

Compared with the first 5 years of 2007-2013, in 2014-2018 JASPERS completed fewer 
advisory assignments for major projects in the period under review (178 compared to 343 
in the first 5 years of the previous programming period). Despite this smaller number of 
assignments,  JASPERS helped  mobilise  22%  more  EU  resources  in  2014-2018  than  in 
2007-2013.  The  variations  in  the  number  of  assignments  between  Member  States  and 
sectors is detailed on page 35 of Annex 4 of the SWD.  

On the rates of interruption, there has been a significant reduction (from 82% to 46%) in 
interruptions of major projects  assisted by JASPERS advisory services during  approval 
procedures39 compared to the previous programming period. 

Comparing projects assisted and not assisted by JASPERS advisory services, the rate of 
interruption  for  assisted  projects  is  48%  as  against  43%  for  the  non-assisted  ones. 
However,  the  higher  rate  of  interruption  for  projects  assisted  by  JASPERS  was  not 
attributable to any failings of JASPERS advisory services, but rather to: (i) the features of 
the projects concerned (such as a higher level of complexity, which underpinned the need 
to seek external advice); and (ii) country-specific factors (such as the level of technical 
administrative capacity). For example, projects assisted by JASPERS advisory tended to 
have lower rates of interruption than non-assisted projects for all sectors, except for the 
roads  sector.  For  the  water  and  waste  water  sector,  all  projects  submitted  had  been 
assisted by JASPERS advisory, and therefore a comparison was not possible. Within the 
group of assisted projects across sectors, the rate of interruption tended to be higher for 
energy and solid waste projects and for water and wastewater projects.  

When  comparing  rates  of  interruption  for  assisted  and  non-assisted  projects  at  country 
level, it was found that the projects assisted by JASPERS advisory tended to have zero or 
lower rates of interruption in 4 countries (Austria, Czechia, Portugal, Slovakia), similar 

39  This  analysis  covers  all  150  major  new  projects  submitted  to  the  Commission  under  the  submission  procedure 
(with PSA review) during the reference period (whether assisted or not), with a focus on the projects interrupted 
in the approval process in the Commission.  
The approval process of a major project is interrupted when the Commission identifies critical issues in the project 
documentation. If this happens, an observation letter is sent to the Member State signalling the issues. The process 
is resumed once the reply of the Member State is received.   

20 

 
                                                 
rates  of  interruption  for  3  countries  (Spain,  Italy,  Poland),  and  a  higher  rate  for  only  1 
country (Greece). Table A4.6 of Annex 4 to the SWD has more detail on this. 

An  analysis  was  made  of  documents  produced  when  providing  technical  advisory 
services.  This  analysis  showed  that  JASPERS’  experts  signalled  that  changes  were 
needed  in  the  documentation  in  over  57%  of  the  projects  that  later  received 
‘observations’  from  the  EC40.  This  suggests  that,  even  though  the  JASPERS’  experts 
included significant issues and comments in their reports, in some cases the issues were 
not  resolved  by  the  Member  State  before  submitting  the  project  to  the  EC.  There  are 
many  possible  reasons  why  the  comments  by  the  JASPERS  experts  were  not  fully 
resolved by the Member States. One possible explanation is that the experts’ message on 
issues requiring further attention might not have been sufficiently strong for the Member 
State to understand that changes were needed. Another possible reason is that there may 
have been time constraints on the side of the Member States for submitting the project. 
Yet  another  possible  reason  is  that  political  decisions  could  have  been  taken  not  to 
address  the  issues  signalled  by  JASPERS  at  that  point.  There  were  also  cases  where 
major  projects  were  interrupted  and  received  observations  from  the  EC,  even  though 
JASPERS advisory provided a positive assessment of the project, with no suggestions for 
improvement. This is the case, for instance, as regards the assessment of environmental 
issues, indicators, and project descriptions, for approximately 90% of the major projects 
interrupted for issues on these topics. 

By comparison, and as noted above, in the 2007-2013 programming period, the overall 
rate  of  interruption  of  major  projects  assisted  by  JASPERS  advisory  was  much  higher 
(82%  of  all  projects  submitted),  with  similar  rates  of  interruption  for  both  assisted  and 
non-assisted projects. Because the extent of the reduction in interruptions between these 
two  periods  is  roughly  the  same  for  both  assisted  and  non-assisted  projects,  this 
improvement  can  be  seen  as  a  consequence  of  the  changes  in  the  administrative  set-up 
for  assessing  and  approving  projects  instituted  by  the  EC  for  2014-2020,  the  latter 
implicitly integrating the involvement of JASPERS services for the review of all major 
projects.  

Finally,  on  interruptions,  an  analysis  was  carried  out  to  assess  whether  the  probability 
that a project was not interrupted was linked to the stage in which JASPERS’ advisory 
support was requested for project preparation. This analysis took explicit account of other 
factors  such  as  project  size,  sector  and  country-specific  factors.  The  analysis  covered 
95%  of  all  new  major  projects  approved  by  the  EC  under  the  submission  procedure 
(entailing  the  PSA  review)  up  until  October  201941.  The  analysis  indicates  that  major 
projects  were  more  likely  not  to  be  interrupted  during  the  approval/adoption  process  if 
they had been supported by JASPERS advisory during the preparation phase. This effect 
is  stronger  and  more  statistically  significant  when  JASPERS  advisory  provided 
assistance from the very beginning of the preparation of the project – or at least from half 
way through its preparation – rather than at a later stage.  

40  The most common reasons for interruption related to significant issues which are: cost-benefit analysis, feasibility 
studies,  state  aid,  risk  and  sensitivity  analysis,  and  capacity  of  the  beneficiary.  The  analysis  covers  37  projects 
assisted  by  JASPERS  advisory,  interrupted  by  the  Commission,  and  for  which  the  complete  JASPERS  support 
and project documentation was available. In total, 69 major projects were interrupted or withdrawn until October 
2019. 
142 major projects were included in the analysis (with complete data sets of the necessary variables), out of 150 
major projects adopted after the submission procedure (with PSA review). 

41 

21 

 
                                                 
In addition, JASPERS’ advisory support was positively correlated to a shorter duration of 
the preparation of the project relative to the overall project lifecycle, but only when the 
support  was  requested  at  the  very  start  of  the  preparation  stage.  At  the  same  time,  the 
evaluation  found  that  later  involvement  of  JASPERS  advisory  in  project  preparation 
might have been correlated with a longer duration of project preparation.  

For non-major projects,  the effectiveness  of JASPERS’ advisory  cannot  be established, 
because data on the progress of projects to approval and implementation is not available. 
However,  preliminary  evidence  for  a  third  of  completed  assignments  indicates  an 
approval rate no higher than 60%. 

The  detailed  analysis  underpinning  the  above  findings  can  be  consulted  in  Chapter  5.2 
(Technical assistance for major projects) of Annex 4 to the SWD. 

On  the  review  functions,  the  two  strands  of  work  were  different  in  their  scope  and 
purpose as set out in the two bullet points below. 

  Under Article 102(1) (notification procedure, entailing the IQR review),  the full 
documentation of the major project was assessed by JASPERS before submission 
to  the  EC.  The  EC  then  received  the  IQR  report  issued  by  JASPERS  and  a 
notification template filled in by the Member State. 

  Under Article 102(2) (submission procedure, entailing the PSA review),  the full 

documentation of the major project was submitted to the EC.  

The choice of one of the two procedures for assessing the major projects was in the remit 
of  the  Member  State.  An  important  difference  between  the  two  procedures  is  that  the 
Member States were permitted to make changes to the project documentation during the 
drafting of the IQR report. This was not the case for the submission procedure entailing 
the  PSA  report  (where  changes  were  possible  only  based  on  an  official  letter  of 
interruption sent by the EC). Implicitly, depending on whether the Member State took on 
the  advice  of  JASPERS,  the  IQR  process  had  the  potential  to  lead  to  faster  processing 
and  a  higher  approval  rate  of  the  major  projects,  and  therefore  a  faster  start  to 
implementation on the ground. 

Moreover,  the  median  total  duration  of  the  process  for  assessing  and  approving  major 
projects notified under Article 102(1) (notification procedure, entailing the IQR review) 
was 7 months42, of which the median duration of the IQR assessment was 5 months. By 
contrast,  the  median  total  duration  of  the  assessment  and  approval  process  for  major 
projects  submitted  under  Article  102(2)  (submission  procedure,  entailing  the  PSA 
review) was 3 months for projects not interrupted and 6 months for those interrupted.  

In  this  process,  JASPERS  was  highly  effective  in  delivering  timely  PSA  reports. 
Compared  to  2007-2013,  there  was  a  significant  reduction  in  the  overall  time  taken  by 
the EC to assess and approve major projects in 2014-201843. This reduction was greater 
for the submission procedure with PSA review (50%) than for the notification procedure 
with  IQR  review  (13%).  By  comparison,  in  2007-2013,  the  average  duration  of  the 
approval  process  for  a  major  project  was  around  9  months,  and  the  minimum  duration 
recorded was 6 months.  

42  Starting with the entry into force of the Omnibus Regulation (EU, Euratom) 2018/1046 in July 2018, a regulatory 

43 

deadline of 6 months was introduced for the IQR assessment.  
JASPERS  services  were  not  involved  in  the  approval  process  of  major  projects  in  the  previous  programming 
period of 2007-2013. 

22 

 
                                                 
Details on the ability of JASPERS IQR services to signal issues related to project quality 
can be consulted in Chapter 5.2 (Review function) of Annex 4 to the SWD. 

For capacity building and horizontal and strategic advice, JASPERS’ portfolio included 
220  assignments,  75%  of  which  were  for  horizontal  and  strategic  advice  and  25%  for 
capacity building.  

The  objective  of  the  capacity-building  assignments  was:  (i)  to  provide  training, 
workshops  and  online  fora  on  topics  relevant  for  preparing  and  implementing  projects; 
and  (ii)  to  help  disseminate  guides  and  studies.  The  training  and  events  organised  for 
capacity building could have been either multi-country or tailored to the specific needs of 
an  individual  country.  In  the  period  analysed,  most  assignments  for  capacity  building 
were  multi-country  assignments,  and  the  most  common  topics  of  training/workshops 
were climate change, environmental requirements and state aid. 

During  the  2014-2018  evaluation  period,  most  activities  for  capacity  building  were 
completed  and  achieved  their  objective  of  delivering  training  to  the  Member  States. 
However, these activities did not prove sufficient to trigger a visible knock-on effect on 
the  technical  capacity  of  Member  States  that  would  allow  them  to  effectively  address 
issues  in  their  preparation  of  projects  such  as:  (i)  cost-benefit  analysis;  (ii)  climate 
change; and (iii) environmental requirements.  

It  is  helpful  to  compare  the  main  issues  addressed  by  capacity-building  activities 
completed up until end-2018 on the one hand with the quality gaps identified in project 
preparation during the advisory  work of JASPERS and during the assessment of major 
projects by the EC on the other. Two main issues were identified in this comparison, both 
of which are addressed in the paragraphs below.  

Firstly, despite the relatively frequent focus of capacity-building activities on cost-benefit 
analyses, climate change and the environment,  shortcomings in  these aspects  of project 
applications  were  still  among  the  most  frequent  reasons  for  interruptions  issued  by  the 
EC  in  the  approval  process  of  major  projects.  This  suggests  that  the  capacity-building 
activities organised on these topics did not succeed in having a significant effect on the 
technical capacity of Member States to address these issues themselves without support.  

Secondly,  the  documentation  of  advisory  work  and  the  EC’s  assessments  of  major 
projects showed that there were still many projects in need of advisory support on topics 
such as feasibility studies, risk analysis, and sensitivity analysis. Capacity building is not 
that  frequent  in  these  three  areas.  During  a  consultation,  stakeholders  suggested  that 
capacity building is better strengthened with hands-on experience while working directly 
with  JASPERS  on  the  projects,  rather  than  through  training  and  workshops.  While,  in 
principle, there might be merit in this argument, there is little evidence that this hands-on 
experience is reflected in better preparation of projects across sectors and countries. This 
may  be  due  to  high  turnover  of  staff  in  national  authorities’  administrations,  a 
phenomenon also referred to in the ECA report. It may also be because JASPERS often 
interacted  with  consultants  contracted  by  Member  States 
to  prepare  project 
documentation, and therefore the learning effect for the staff of national authorities was 
limited.  Moreover,  the  evaluation  highlighted  that  countries’  reliance  on  JASPERS 
advisory  assistance  for  project  preparation  has  not  significantly  decreased  compared  to 

23 

 
the  previous  period.  This  suggests  there  has  only  been  a  modest  capacity  increase  in 
project preparation at national level between the two periods44.  

A pragmatic approach implemented by JASPERS for technical capacity building was to 
‘train  the  trainers’.  This  focused  training  on  local  trainers  who  could  subsequently 
disseminate  the  know-how  they  had  acquired  to  larger  audiences  on  many  occasions 
within  and  across  countries.  This  approach  was  used  by  JASPERS  for  some  capacity-
building assignments over the period, but it had not yet been mainstreamed at the level of 
the portfolio, within the timeframe of this evaluation.  

On horizontal and strategic support for capacity building, these assignments were highly 
diverse in  their specific  objectives (including support for national  strategies on specific 
issues, river basin flood management plans or integrated urban development plans). For 
this reason, it was far more difficult to assess the overall effectiveness of the support. An 
analysis of 10 sampled capacity-building assignments indicated that JASPERS’ support 
helped  compensate  for  weaker  local  technical  capacity  to  guide  and  supervise  tasks  on 
specialised topics of policy relevance. However, this finding cannot be generalised for all 
166 training assignments for horizontal and strategic support from the reference period of 
the evaluation. 

The detailed analysis underpinning the findings from above can be consulted in Chapter 
5.2 (Capacity building and horizontal and strategic advice) of Annex 4 to the SWD. 

On the CEF mandate, most45 of the projects supported by JASPERS during its first pilot 
mandate  were  successful  in  CEF  competitions  up  to  August  2019.  Seven  of  the 
investment projects assisted by JASPERS at the application stage for CEF competitions 
had also been assisted by JASPERS technical advisory in their preparation in the 2007-
2013  programming  period  under  the  ESIF  mandate.  Because  of  its  pilot  nature,  a 
comparative  analysis  of  the  effectiveness  of  JASPERS’  intervention  under  the  CEF 
mandate  compared  to  a  previous  existing  baseline  cannot  be  undertaken.  It  is  also  not 
possible to conduct a comparative analysis of the effectiveness of the intervention for the 
ESIF mandate. Nevertheless, the objectives stated in the strategic documents have been 
followed, with JASPERS providing technical assistance to prepare eligible projects, with 
the intention of them being submitted in response to CEF calls. 

On the IPA mandate, by the end of 2018, only 18% of JASPERS’ assignments resulted in 
IPA projects being approved for EU funding (17 projects). Overall, by the end of 2018, 
38%  of  the  assignments  had  been  completed,  50%  were  still  ongoing,  and  the  rest  had 
been  administratively  closed.  Based  on  an  analysis  of  a  sample  of  six  assignments46, 
there is an indication that the effectiveness of JASPERS’ advisory support was limited in 
cases  where  the  requests  for  support  arrived  late  in  the  project  preparation  cycle.  For 
these late-arriving projects, it can be concluded that JASPERS advisory could have made 
a  difference  to  project  quality  if  its  support  had  been  requested  earlier  in  the  project-

44  However, because of a lack of data it was not possible to establish whether those participating in capacity-building 
events were also the ones working on project preparation. In addition, the requirements for  major projects were 
somewhat different between the two programming periods, so the expertise gained by the Member States in the 
past required updating. More information on the requirements linked to major projects in the 2007-2013 period 
can be found in the JASPERS evaluation report (AECOM, 2012).  

45  The number of assignments undertaken (29) was directly dependant on the budget available, which was limited 
since the initiative was at a pilot stage. The combined estimated total costs of the projects linked to the JASPERS 
assignments supported though CEF1.1 mandate amounts to over EUR 8 billion. 

46  Two  of  these  projects  were  approved  for  IPA  financing,  with  the  remaining  4  either  not  being  submitted  (2 

projects) or not being approved due to cuts in IPA financing. 

24 

 
                                                 
preparation  process.  When  JASPERS  support  was  requested  in  the  final  stages  of 
preparation  (as  was  the  case  for  three  completed  assignments  by  2018),  despite 
JASPERS’  critical  assessment  of  project  quality,  no  substantial  changes  were  made  by 
the national authorities concerned. 

Finally,  there  is  a  finding  that  is  linked  to  the  consultation  activities  and  valid  for  all 
mandates. This  finding is  that JASPERS had a significant,  and even decisive, effect  on 
project  technical  specifications  and  documentation.  A  large  majority  of  stakeholders 
(80% of 210 respondents) rated highly JASPERS’ assistance in aligning projects with EU 
and national policies and in achieving faster project preparation and approval. 

The state of play of the JASPERS initiative was assessed by the ECA in a performance 
audit  report  published  in  201847.  The  audit  covered  the  first  10  years  (2006-2016)  of 
JASPERS’  activities  and  four  Member  States:  Croatia,  Malta,  Poland,  and  Romania48. 
On  the  effectiveness  of  JASPERS’  activities,  the  ECA  auditors  concluded  that  these 
activities contributed to faster project approval and had a positive effect on the quality of 
projects  and  their  documentation.  For  the  implementation  of  projects  assisted  by 
JASPERS,  the  ECA  auditors  found  that,  while  JASPERS’  comprehensive  assistance 
could  generally  not  have  an  impact  on  the  financial  implementation  of  EU  funds,  the 
projects assisted were found to be less frequently affected by legal and regularity errors 
during the compliance audit exercises carried out by ECA in 2014-2015.  

Efficiency  

The  evaluation  of  efficiency  aims  at  assessing  the  relationship  between  the  resources 
used  by  an  intervention  and  the  changes  generated  by  that  intervention  (the  costs  and 
benefits).  Because  data  on  the  actual  costs  incurred  by  the  work  on  individual 
assignments  is  limited  (see  Section  4),  the  analysis  considers  JASPERS  efficiency  by 
looking at: 

- 
- 
- 

the administrative closure of assignments;  
the work intensity distributed across assignments49; 
the correlation between human resources, financial resources and the number of 
assignments. 

The  analysis  below  covers  only  the  ESIF  mandate,  which  provided  97%  of  the  total 
financial resources allocated to JASPERS during the 2014-2020 programming period.  

Overall, more than 90% of JASPERS’ assignments were completed in the period 2014-
2018,  with  9%  being  administratively  closed  (i.e.  assignments  cancelled  before 
completion), usually long time after being initiated. Half of those administratively closed 
assignments were concentrated in three countries50.  

A  quarter  of  the  cancelled  assignments  had  a  duration  of  more  than  50  months, 
highlighting  the  long  time  taken  before  closure.  Moreover,  based  on  the  analysis  of 
country plans, it can be concluded that many of the administratively closed assignments 
had no initial expected date for completion or, if they had such a date, this date was much 
later  than  set  out  in  the  initial  plan.  On  the  reasons  for  administrative  closure,  the 

https://www.eca.europa.eu/Lists/ECADocuments/SR18_01/SR_JASPERS_EN.pdf  

47 
48  The  assessments  for  Croatia  and  Poland  were  carried  out  in  cooperation  with  the  national  supreme  audit 

institutions.  

49  Only starting from 2017, as this is when JASPERS began recording the number of hours worked per assignment.  
50  More than half of the administratively closed assignments were in Croatia, Poland, and Romania. 

25 

 
                                                 
evaluation  identified  three  main  reasons:  (i)  an  agreement  between  JASPERS  and  the 
country  concerned  to  combine  the  assignment  with  other  ongoing  assignments;  (ii) 
closure of the assignment because of a change in context (for example, projects prepared 
for  the  ESIF  but  eventually  submitted  for  CEF);  and  (iii)  a  decision  by  the  country 
concerned  to  remove  the  assignment  from  its  plan  because  it  was  deemed  no  longer 
feasible. 

On  the  intensity  of  work  in  relation  to  the  duration  of  assignments51,  the  evaluation 
concludes that there is little correlation between these two variables in most cases. This 
means that assignments that last longer tend to have less intensity of work. For technical 
advisory support, this can be explained by the fact that the advice during various steps in 
the preparation and implementation of the project depends on a number of factors (time 
needed  to  receive  the  reply  from  the  Member  State,  contractual  arrangements  with 
external consultants at Member State level, etc.). This finding also holds true for review 
services.  However,  it  is  important  to  differentiate  the  two  strands,  as  the  PSA  services 
had  a  high  concentration  of  short  assignments  (with  most  being  finalised  within  a 
month52), while the IQR services had a wider range of durations of assignments (up to 10 
months53). PSA assignments remained the most work intensive. 

Comparing  the  resources  allocated  over  the  two  periods,  it  is  clear  that  JASPERS  has 
grown significantly, increasing from 56 people in 2007 to 140 professional and support 
staff in 2018. This reflects adaptation to the demand for JASPERS’ services, which grew 
and became more diversified in the 2014-2020 period. 

Moreover,  the  change  in  the  structure  of  the  budget  indicates  a  reorientation  in 
JASPERS’ staff policy. In the period 2014-2018, the initiative relied on its own staff to a 
larger extent, as opposed to the more extensive use of external consultancy services in the 
previous  period.  This  means  that  most  of  JASPERS’  resources  were  invested  in  its 
people,  with  the  share  of  staff  costs  increasing  from  82%  to  91%  between  the  two 
periods. 

Figure 8. Human and financial resources, and assignments over time, ESIF mandate 

51  Data available starting from 2017. 
52  The  PSA  assignments  had  a  clear  deadline  of  30  days  agreed  between  JASPERS  and  the  Commission.  This 
deadline was introduced since the Commission itself is bound by regulatory deadlines for issuing a decision on 
major  projects  submitted,  and  therefore  there  was  a  need  for  predictability  as  regards  time  delivery  of  PSA 
reports. 
In  relation  to  the  IQR  function,  there  was  no  regulatory  deadline  up  until  July  2018,  and  therefore  the  process 
tended to be prolonged for some assignments. This is also because JASPERS IQR division needs to undertake an 
interactive process of assessment with Member States. This happens, for instance, because of the need to clarify 
certain aspects of projects with Member States during the assessment. In the future, however, it is expected the 
IQR process will become more limited in time because of the regulatory deadline of 6 months introduced in the 
Omnibus Regulation in 2018. 

53 

26 

 
                                                 
Source:  Inforegio  report  Analysis  and  assessment  of  JASPERS’  Services  and  Performance  2014-2018, 
based on data reported in JASPERS’ annual reports 2007-2018. 

The left panel in Figure 8 illustrates the number of annual full-time equivalent workers 
(FTEs) supported in JASPERS and the annual monetary contribution by the EC over the 
period  2011-2018.  These  data  indicate  that,  starting  in  2014,  there  was  a  continuous 
increase in staff every year over the period, while the EC’s annual contribution increased 
at  the same time. Compared to  2013, the number of JASPERS’ personnel  in  2018 was 
55% larger and the EC’s annual contribution was 65% larger. 

For  ongoing  assignments,  there  has  been  an  almost  constant,  linear  increase  each  year 
until 2017, while staff numbers continued to increase throughout the period. The increase 
in the annual number of assignments up until 2017 can be explained by both the rate at 
which  assignments  were  accepted  and  their  median  duration  of  roughly  2  years.  The 
completion  or  closure  of  assignments  mostly  began  to  increase  in  2016.  However,  the 
number of ongoing assignments is an imperfect proxy for the actual work involved. This 
is  because,  as  explained  earlier,  the  work  intensity  per  assignment  varied  significantly, 
and work intensity was not very high for most assignments other than those involving the 
PSA review54. This preliminary evidence therefore indicates a need for further in-depth 
analysis of the topic to help identify possibilities to optimise the allocation of work and 
rationalise costs.  

In addition, the ECA audit also identified scope for improvement in a number of aspects 
in the administrative set-up and approach of JASPERS’ activities. These included:  

- 

- 

the  role  and  responsibilities  of  the  main  stakeholders,  which  the  ECA  believes 
should be defined better by calibrating more precisely the working arrangements 
between  JASPERS  and  the  Member  States  for  all  types  of  service  (including 
capacity-building activities and horizontal and strategic support);  
the need to implement a better system for monitoring the efficiency of JASPERS’ 
operations. 

The detailed analysis underpinning the above findings can be consulted in Chapter 5.3 of 
Annex 4 to the SWD. 

54  A more detailed analysis of this aspect of efficiency is not feasible at present, as it would require data on hours 

worked per assignment and calendar year for all assignments – data which are not available. 

27 

 
  
 
 
                                                 
Coherence 

The analysis of the coherence criterion was undertaken from both an internal and external 
perspective.  The  internal  perspective  refers  to  both:  (i)  the  coherence  of  the  different 
types  of  service  provided  by  JASPERS  with  each  other;  and  (ii)  the  coherence  of 
JASPERS’  services  with  EU  priorities.  The  external  perspective  assesses  coordination 
between  JASPERS  and  other  advisory  services  at  EU  level  providing  similar  technical 
assistance to Member States.  

The evaluation finds that JASPERS’ services were coherent with EU requirements set by 
cohesion  policy  legislation  and  the  EU-level  objectives  in  the  area  of  climate,  the 
environment  and  competition.  The  services  helped  to  guide  project  preparation  and 
supported the assessment of projects in line with these objectives. 

Applying  for  EU  co-financing  of  major  projects  required  the  Member  States  to  ensure 
their  projects:  (i)  were  aligned  with  both  cohesion  policy  objectives  and  the  EU-level 
objectives  of  mitigating/adapting 
their 
environmental impacts; and (iii) met all regulatory requirements related to state aid55. As 
a  consequence,  all  JASPERS  services  were  organised  so  as  to  align  the  advisory  and 
review work with these EU priorities.  

to  climate  change;  (ii)  accounted  for 

In the course of the advisory work, depending on the scope of the assignment requested 
by Member States, JASPERS provided an assessment (or, at a minimum, a description) 
of  the  project’s  alignment  with:  (i)  EU  policy  objectives  and  priorities;  and  (ii)  EU 
requirements  on  the  environment,  climate  change,  and  competition.  In  this  process, 
JASPERS identified changes needed in project documentation to ensure this alignment, 
or  made  suggestions  for  improvement.  The  evaluation  found  that  JASPERS  provided 
advice  on  climate  and  environmental  issues  on  54-60%  of  the  projects  assisted,  while 
JASPERS addressed state aid issues in about a quarter of these projects. 

From an internal coherence perspective, while acting independently during the course of 
providing the services, JASPERS review services and JASPERS technical advisory were 
found to be generally coherent with each other in the way they applied EU requirements 
to  project  preparation  on  the  one  hand  and  to  project  assessment  on  the  other.  Some 
scope for further alignment remained, although this was limited to around 10% or less of 
projects and concerned minimum regulatory requirements for project quality. 

The  evaluation  finds  that  for  a  large  majority  of  projects  (above  90%),  there  was  full 
coherence between the two services. Firstly, for projects with no critical issues identified 
in the PSA report, there were cases for which JASPERS advisory had raised significant 
issues  during  project  preparation  and  the  Member  State  addressed  these  issues  before 
submitting the project to the EC. These issues were therefore no longer present when the 
project  reached the PSA assessment. The two outputs (of  the technical  advisory and of 
the  review  services)  were  therefore  coherent  even  if  they  were  different.  Secondly,  on 
many other projects on which the PSA report did raise critical issues, JASPERS advisory 
also signalled significant issues during project preparation, but these issues had not been 
addressed by the Member State by the time the project reached the EC. In these cases, the 
two outputs were also coherent, as the outstanding critical issues were due to the Member 
State failing to heed JASPERS’ warnings on the shortcomings of the application.  

55  The  templates  for  submitting  information  on  major  projects  for  the  ERDF  and  Cohesion  Fund  are  included  in 
Commission Implementing Regulation (EU) 2015/207 Annex II (for the procedure based on Article 102(2)), and 
in Commission Implementing Regulation (EU) 1011/2014 Annex II (for notification based on Article 102(1)). 

28 

 
                                                 
On external coherence, the evaluation explored the possibility of assessing the coherence 
and  synergies  between  JASPERS  services  and  other  EC  initiatives  for  technical 
assistance.  Under  the  2014-2020  multiannual  financial  framework,  there  were  several 
centrally managed initiatives for technical assistance, most of them focused on capacity 
building  and  project  preparation.  These  initiatives  include:  the  European  Investment 
Advisory Hub (EIAH); European Local Energy Assistance (ELENA); and the European 
Energy  Efficiency  Fund  (EEEF).  JASPERS  was  one  of  the  largest  of  these  initiatives, 
with  30%  of  the  total  cumulated  budget  of  more  than  EUR  700  million  for  centrally 
managed technical assistance. 

The evaluation concludes that, until the end of 2018, the potential for synergies between 
JASPERS  and  other  technical  assistance  initiatives  were  only  partly  exploited. 
JASPERS’ Quality Manual mentions the possibility for JASPERS to cooperate in sharing 
expertise with  other  technical  assistance services.  In 2015, following the  publication of 
the European Fund for Strategic Investments (EFSI) Regulation56, JASPERS assessed the 
scope  for  cooperation  with  the  EIAH  and  even  set  up  procedures  to  promote  this 
cooperation.  During  the  reference  period  of  this  evaluation,  JASPERS  cooperated  with 
the EIAH on three assignments to construct and equip regional hospitals in Romania. 

The detailed analysis underpinning the above findings can be consulted in Chapter 5.4 of 
Annex 4 to the SWD. 

Relevance 

Because  of  the  limitations  described  in  Section  4  of  this  SWD,  the  relevance  of 
intervention  is  assessed  mainly  in  terms  of  its  scope  as  compared  with  the  previous 
programming period. The SWD only examines the connection of JASPERS’ work with 
identified  needs  in  a  limited  way.  More  specifically,  the  needs  analysis  performed  to 
inform the 2014-2020 working arrangements was based mostly on direct feedback from 
the  Member  States,  which  were  also  the  direct  beneficiaries  of  the  technical-assistance 
services provided by JASPERS. 

The main approach for  evaluating the  relevance criterion  was to  assess the structure of 
JASPERS’ portfolio compared with the previous programming period.  

For the ESIF mandate, JASPERS remained relevant for project preparation in the 2014-
2020 period, and provided more services for more countries than it did in 2007-2013. In 
addition,  JASPERS  became  relevant  for  the  new  function  of  quality  reviewing  ESIF 
major projects. 

Similarly,  JASPERS  was  relevant  for  the  objectives  set  out  in  the  CEF  and  IPA 
mandates, which called for the provision of technical assistance for projects that would 
be eligible for EU funding.  

During the programming period, JASPERS’ assistance became available for all Member 
States  and  IPA  countries  and  was  mandated  by  three  EC  services  (DG  REGIO,  DG 
MOVE, and DG NEAR). Its services included: (i) technical advice for the preparation of 
major  and  non-major  projects;  (ii)  capacity-building  activities  and  horizontal  and 
strategic  support;  and  (iii)  a  new  review  function  for  projects  supported  by  cohesion 
policy. For the first 5 years of the period, JASPERS registered around 1 100 assignments 
in  26  countries.  By  comparison,  in  the  previous  period  2007-2013,  JASPERS  provided 

56  Regulation (EU) 2015/1017. 

29 

 
                                                 
technical  assistance  for  14  Member  States  and  IPA  countries  through  just  under  800 
completed assignments.  

Most  of  JASPERS’  activities  for  the  ESIF  mandate  were  concentrated  in  3  main 
beneficiary countries (Romania, Croatia and Poland) – 2 of which (Romania and Poland) 
were consulted for the needs assessment to prepare the 2014-2020 period and indicated 
high needs of support from JASPERS services.  When looking at  the distribution of the 
ESIF assignments at country level, 57% of these assignments were carried out in 3 main 
beneficiary  countries  (Romania,  Croatia,  and  Poland),  5%  were  multi-country 
assignments,  and  the  rest  were  distributed  across  19  other  Member  States.  By 
comparison, during the previous period 2007-2013, 59% of JASPERS’ activities for this 
mandate  were  also  concentrated  in  3  main  beneficiary  countries  (Poland,  Romania, 
Bulgaria), with the rest of the activities distributed across 11 more Member States. 

JASPERS’ new review function was highly relevant, because it assessed the quality of all 
new major projects submitted or notified to the EC under the CPR legislative framework 
for  the  period  2014-2020  involving  either  Article  102(1)  entailing  the  IQR  review  or 
Article  102(2)  entailing  the  PSA  review.  The  evaluation  finds  that  71%  of  new  major 
projects  were assessed under the submission  procedure (with  PSA report) during 2014-
2018. 

On  training  support  to  build  administrative  capacity,  the  share  of  training  activities  for 
multi-country  capacity  building  on  topics  of  high  relevance  for  all  countries  increased. 
Nevertheless,  horizontal  and  strategic  support  tailored  to  country-specific  needs 
accounted for most of the support. 

Based on JADE data, the evaluation finds that JASPERS carried out 220 assignments for 
capacity  building  and  horizontal  and  strategic  support  up  to  December  2018,  75%  of 
which were for horizontal and strategic advice. By comparison, in 2007-2013 JASPERS 
completed close to 140 such assignments, 96% of which were for horizontal and strategic 
advice. The relative share of assignments for capacity building, therefore, increased from 
4% to 25% between the two periods. These assignments can be tailored to the needs of a 
specific country or can be organised on a multi-country basis for topics of high relevance 
to several or even all Member States.  

For  capacity-building  training,  in  contrast  to  the  previous  period  when  almost  all 
assignments were country specific, there was an increase in 2014-2020 in the frequency 
of multi-country assignments. The most frequent topics for these training activities were 
climate change, the environment and state aid.  

The  evaluation  finds  that  the  demand  for  country-specific  capacity-building  activities 
was not high during the period examined. 

Only 19 country-specific assignments for capacity building had been completed or were 
ongoing  at  the  end  of  December  2018  in  10  Member  States.  More  than  60%  of  these 
training  assignments  were  devoted  to  climate-change  adaptation  and  the  environment, 
with the remainder mostly focused on horizontal aspects, planning and implementation. 

Finally, the  evaluation  finds that JASPERS’ support was relevant  for meeting cohesion 
policy objectives, with JASPERS’ general aim of supporting Member States during the 
project-preparation process being maintained across the last two programming periods. In 
practical terms, Member States first sent requests for assistance to JASPERS, which then 

30 

 
communicated these requests to the EC57 in order to receive the green light to proceed, 
which  was  generally  done  by  tacit  agreement.  This  approach  tended  to  ensure  that 
Member States’ needs for support were addressed, but it did not mean that there was a 
strategic  analysis  or  prioritisation  of  requests.  As  highlighted  by  ECA  auditors, 
JASPERS has not targeted its activities sufficiently, as it has not decided on clear priority 
areas for providing assistance to Member States, but has simply responded to demands as 
they arose. 

EU added value 

This  section  includes  the  main  findings  from  the  analysis  of  EU  added  value  of  the 
JASPERS  initiative.  First,  it  should  be  noted  that  the  nature  of  the  support  concerned 
implicitly embeds the European dimension, as the technical assistance services provided 
are supported financially by EU institutions (EC and EIB).   

The  evaluation  of  EU  added  value  relies  primarily  on  the  feedback  received  from 
stakeholders responding to an online targeted consultation during summer/autumn 2019, 
in which stakeholders’ views primarily referred to the general added value of JASPERS. 
Of the respondents to this consultation, 75% were either project promoters/beneficiaries 
or  managing  authorities,  while  the  remaining  25%  were  consultancies,  companies,  EC 
officials,  and  other  types  of  institutions  at  national  level.  As  they  had  first-hand 
experience with JASPERS services, the project promoters and managing authorities were 
able  to  provide  informed  opinions  on  the  topics  related  to  services  provided  by 
JASPERS.  This  distribution  of  respondents  to  the  consultation  inherently  includes  a 
possible bias in assessing involvement with JASPERS services, as project promoters and 
managing  authorities  were  directly  benefiting  from  the  free-of-charge  support  provided 
by JASPERS. 

The  information  obtained  from  stakeholders  indicates  that  JASPERS  services  were 
appreciated because of the extensive methodological and sectoral technical expertise that 
it provided, thanks to the experience JASPERS gained from working with many Member 
States.  According  to  stakeholders,  this  expertise  cannot  easily  be  replaced  as  it  is  not 
readily available on local markets.  

In  addition,  a  large  majority  of  stakeholders  considered  that,  relative  to  other  advisory 
services,  JASPERS’  added  value  was  higher  especially  in:  (i)  its  support  for  aligning 
projects  with  EU  priorities;  (ii)  the  potential  for  knowledge  transfer  among  Member 
States;  and  (iii)  its  hands-on  experience.  Different  opinions  were  expressed  on  aspects 
such as flexibility in responding to needs and expertise on administrative and procedural 
issues. 

A large majority of managing authorities and project beneficiaries/promoters were of the 
view that JASPERS’ services  will still be needed in  the future,  even if  EC  approval  of 
major  or  strategic  projects  was  no  longer  required.  According  to  this  majority  of 
managing authorities, JASPERS support is necessary for aligning national strategies with 
EU sectoral policy priorities. Finally, from an operational perspective, stakeholders also 
stressed  that  the  use  of  JASPERS  services  does  not  require  public  procurement 

57  Valid for ESIF and CEF mandate, for the IPA mandate it is only informing the Commission. 

31 

 
                                                 
procedures, and that this was a distinct advantage. However, willingness to pay for these 
services from their own technical-assistance budgets is rather low58.  

The synopsis of the stakeholder consultation is included in Annex 2 of the SWD.  

Another aspect that is relevant to JASPERS’ EU added value lies in the organisation of 
its review function. According to the regulatory framework in place, Member States were 
able to set up individual arrangements for the independent quality review of the  project 
documentation before they submitted their major project to the EC under Article 102(1) 
of  the  CPR.  The  creation  of  the  IQR  division  in  JASPERS,  which  provides  review 
services  available  to  all  project  promoters  across  Member  States,  reduced  the 
administrative burden for Member States in the area of independent quality reviews.  

6.  CONCLUSIONS 

The  purpose  of  this  evaluation  is  to  assess:  (i)  the  implementation  of  the  services 
provided by JASPERS; (ii) the contribution of JASPERS services to the development of 
high-quality,  mature  infrastructure  projects;  and  (iii)  the  contribution  of  JASPERS  to 
enhancing the technical  capacity of managing  authorities and beneficiaries in  preparing 
such projects. The evaluation covers all JASPERS activities during the 2014-2018 period 
across all  countries  that  made use of its advisory services  (i.e. Member States and pre-
accession countries). 

The JASPERS initiative is assessed against the five evaluation criteria decided on by the 
EC’s better regulation guidelines: effectiveness, efficiency, coherence, relevance and EU 
added value. 

As  explained  in  the  previous  sections,  the  effectiveness,  efficiency  and  relevance 
evaluation  criteria  were  assessed  via  proxy  analysis  because  of  limitations  in  the 
definition  of  objectives  and  the  design  of  the  data  monitoring  system.  The  broad 
definition  of  JASPERS’  general  objective  constrained  the  evaluation,  as  this  definition 
does not provide an indication of measurable outcomes that could inform the evaluation 
of JASPERS’ effectiveness and relevance. On the monitoring system, the available data 
on the exact amount of time worked on assignments started to be recorded only in 2017, 
while the evaluation covered the period up to the end of 2018. The limited timeframe of 
available data allows only for having initial reflections on the efficiency criteria.  

Overall,  the  evaluation  finds  that  JASPERS’  activities  expanded  both  in  scope  and 
volume during 2014-2018 compared to the previous period. On scope, although the ESIF 
mandate represented the largest share (close to 90%) of the overall portfolio in numbers 
of  assignments,  JASPERS’  services  were  supported  over  the  period  by  two  more  EC 
services, as DG MOVE and DG NEAR joined DG REGIO in financing the initiative for 
CEF and IPA projects. JASPERS’ services were available for all Member States and IPA 
countries.  JASPERS’  role  was  significantly  strengthened  by  the  EC  through  the 
creation of a new function for the quality review of major new ESIF projects. On volume, 
in  the  first  5  years  of  the  2014-2020  programming  period,  JASPERS  received  more 
requests for services than in the entire period 2007-2013.  

58  The  consultation  took  place  at  a  moment  when  the  2021-2027  cohesion  policy  package  was  still  under 
negotiation.  It  therefore  cannot  be  ruled  out  that  there  was  a  certain  attempt  by  Member  States  to  influence  its 
outcome. 

32 

 
                                                 
Effectiveness 

Much  of  JASPERS’  activities  consisted  of  helping  Member  States  to  prepare  major 
projects  to  obtain  EU  funding  in  a  timely  way.  Of  the  226  assignments  for  technical 
assistance completed up to end-2018, 158 (70%) resulted in major projects being adopted 
by the EC by October 2019. These accounted for a total financing from the ERDF and 
Cohesion Fund of EUR 23.9 billion, 65% of all funding going to major projects over the 
period. This is the same percentage as in the first 5 years of the 2007-2011 period (when 
more major projects but of a smaller size were financed).  

The  timing  of  the  involvement  of  JASPERS’  advisory  in  project  preparation  for  major 
projects  had  a  significant  effect  on  the  effectiveness  of  this  involvement.  Firstly,  the 
evidence indicates that, when compared to non-assisted projects, the projects assisted by 
JASPERS advisory had better chances of being adopted without interruptions for quality 
issues.  On  the  other  hand,  there  is  no  compelling  evidence  that  JASPERS’  support 
reduced either the number of issues that the EC raised or the duration of the interruptions. 
Nevertheless, early involvement of JASPERS advisory services in the project cycle was 
associated with  a shorter duration of the core project  preparation  relative  to  the overall 
project lifecycle. 

The evaluation found that JASPERS advisory collected limited data on what happened to 
non-major projects it assisted once the advisory work was completed. This restricted the 
possibility of assessing the effectiveness of JASPERS’ support. 

The evidence suggests that JASPERS was relatively effective in flagging quality issues 
in  the  PSA  assessments,  which  were  then  included  by  the  EC  in  its  letters  to  Member 
States setting out the reasons for requesting improvements (i.e. interrupting projects). 

While a large majority of activities for capacity building were completed, and achieved 
their  objective  of  delivering  training  to  the  Member  States,  there  is  little  evidence  that 
they  triggered  a  visible  ripple  effect  in  reducing  Member  State  reliance  on  JASPERS’ 
support,  which  should,  in  principle,  be  the  ultimate  objective.  The  demand  for  these 
services  increased  rather  than  declined  between  2007-2013  and  2014-2020.  Issues 
relating to cost benefit analysis, climate change and the environment, on which training 
and  workshops  focused,  continued  to  feature  prominently  in  EC  interruption  letters. 
Other issues that also featured extensively in EC interruption letters included feasibility 
studies and risk and sensitivity analysis, neither of which were much covered in capacity-
building activities. However, the evaluation found examples of a pragmatic approach to 
capacity building, in the form of JASPERS’: (i) ‘training the trainers’ activities at local 
level as a source of continuing knowledge dissemination; or (ii) coordinating training on 
specific issues across countries. These activities have the potential to be promoted more 
widely (as is now recognised by JASPERS management) as part of the strategic planning 
of capacity-building activities, with the long-term aim of reducing reliance on JASPERS’ 
assistance for preparing and implementing projects. 

Efficiency 

When  assessing  efficiency,  it  is  difficult  to  set  a  suitable  benchmark  against  which 
indicators  of  the  time  taken  to  complete  assignments  (or  the  staff  effort/work  effort 
involved) can be measured. In other words, although it might be possible to calculate the 
indicators  concerned,  it  is  still  difficult  to  interpret  these  indicators  and  determine 

33 

 
whether or not they are higher than they should be. This is a problem that is common to 
assessing the efficiency of cohesion policy interventions in general.  

The  duration  of  assignments  is  a  potential  indicator  of  efficiency,  but  it  does  not 
necessarily  reflect  the  amount  of  time  taken,  data  on  which  have  only  been  recorded 
since January 2017. The duration of an assignment is also linked to: (i) the initial quality 
of  documentation;  and  (ii)  the  scale  of  revisions  needed  to  meet  minimum  quality 
standards. The analysis of JASPERS’ efficiency was thus limited by the short timeframe 
of the available data on work per assignment. Before 2017, the monitoring framework at 
the level of individual assignments provided mostly descriptive information (i.e. without 
any quantitative data) of the assignment itself.  

Work intensity, defined as the days worked relative to the days available over the period 
of the assignment, amounted to 30% on average per assignment for technical assistance 
on  major  projects.  In  practice,  there  was  little  correlation  identified  between  work 
intensity  and  the  duration  of  assignments.  Preliminary  evidence  on  developments  over 
time  indicates  a  need  for  further  in-depth  reflection  on  possibilities  for  optimising  the 
allocation  of  work  effort  and  rationalising  costs.  However,  it  should  be  stressed  that 
the main added value of JASPERS lies in the technical expertise and accumulated 
experience of its experts, and that ownership of projects and timely  availability of 
project documentation remain the responsibility of project promoters.  

Although  JASPERS  services  were  generally  found  to  be  efficient  in  completing  the 
assignments  initiated  during  the  2014-2018  period,  the  evaluation  concludes  that  the 
planning and management of administratively closed assignments could be improved. 

Coherence 

During  the  2014-2018  period,  JASPERS’  services  were  coherent  with:  (i)  EU 
requirements  set  by  cohesion  policy legislation;  (ii) the EU-level  objectives for  climate 
and environment; and (iii) regulatory requirements for competition.  

The  evidence  compiled  by  the  evaluation  also  suggests  a  high  degree  of  internal 
coherence between the different independent JASPERS divisions. For over 90% of major 
projects  examined,  the  technical  advisory  support  provided  by  JASPERS  to  Member 
States  in  preparing  projects  was  consistent  with  the  conclusions  of  JASPERS  IQR 
services involved in reviewing the major projects submitted to the EC. 

However, for the period examined, limited evidence was found of JASPERS cooperating 
with  other  technical-assistance  initiatives  of  the  EC,  although  the  possibility  of  sharing 
experience was mentioned in its Quality Manual. 

Relevance 

JASPERS’ advisory services remained relevant in the 2014-2020 period, as reflected by 
the over 400 assignments requested by Member States up to December 2018. By October 
2019,  70%  of  these  assignments  had  resulted  in  major  projects  adopted  by  the  EC  – 
projects  which  mobilised  almost  EUR 24  billion  worth  of  investments  from  the  ERDF 
and Cohesion Fund.  

In addition,  for the 2014-2020 programming period, JASPERS provided a new kind  of 
support:  reviewing  the  quality  of  new  major  projects  submitted  for  EU  funding,  as 

34 

 
required by the regulations, with the aim of ensuring consistency in the assessment and 
reducing the time taken for it. 

JASPERS’ capacity-building activities and provision of horizontal and strategic support 
continued  in  the  2014-2020  period.  Most  of  the  requests  for  support  were  horizontal 
rather than strategic in nature and occurred just before or just after the beginning of the 
2014-2020 period as Member States sought to prepare for this new period. 

The  evaluation  finds  that  the  needs  analysis  developed  for  2014-2020  did  not  take  a 
sufficiently critical view on the possible bias of the stakeholders involved in the analysis, 
who were also potential direct beneficiaries of JASPERS’ services. 

JASPERS’  activities  continued  to  be  demand-driven,  assignments  being  initiated  at  the 
request of Member States or pre-accession countries. This left little room for the strategic 
planning  of  these  activities,  and  for  deploying  resources  where  they  could  produce  the 
most added-value in terms of the management and implementation of EU funding. 

EU added value 

The consultation carried out indicates that JASPERS services have been appreciated by 
stakeholders,  because  of  their  technical  expertise  and  EU-wide  experience.  However, 
another reason stakeholders appreciated JASPERS was because accessing its support did 
not entail a public procurement process and it was provided free of charge. 

JASPERS’ added value was seen as being especially high in: (i) its support for helping to 
ensure  that  projects  were  aligned  with  EU  priorities;  (ii)  the  potential  for  knowledge 
transfer among Member States; and (iii) its hands-on experience in countries.  

Moreover,  stakeholders  considered  that  JASPERS  services  will  also  be  needed  in  the 
future, not only because of the technical expertise brought by the experts and their EU-
wide knowledge, but equally because of their ability to help align national strategies with 
EU priorities.  

To conclude, JASPERS’ EU added value, as indicated by the stakeholders engaged with 
JASPERS support, primarily lies in the technical expertise brought by its experts. These 
experts have EU-wide experience acquired while working in many countries and with the 
EC, and would be difficult to replace. 

Overall,  in  relation  to  the  period  2014-2018,  the  evaluation  reaches  the  following  four 
conclusions. 

- 

- 

- 

JASPERS  technical  advisory  services  contributed  to  a  faster  and  better 
preparation of projects leading to a faster mobilisation of EU funds. 
JASPERS’  general  objective  has  been  broadly  achieved  together  with  all  of  the 
specific objectives, with the exception of the transfer of technical  knowledge to 
Member States in view of reducing their reliance on JASPERS advisory services. 
the evaluation aimed to address all the evaluation criteria in accordance with the 
Better  Regulation  guidelines,  it  relied  on  several  proxy  analysis  due  to  various 
limitations; in particular the limitation concerning the availability of data needed 
for  assessing  the  effectiveness  and  efficiency  of  JASPERS  services  was 
significant  (more  structured  performance  monitoring  at  the  level  of  assignments 
only started in 2017, and was further strengthened only after 2018).  

35 

 
-  There is scope for a more structured approach to the planning and monitoring of 
assignments, with: (i) well-defined and targeted objectives; (ii) clear and realistic 
timelines;  and  (iii)  milestones  that  enable  progress  in  implementation  to  be 
monitored over time. 

7.  DEVELOPMENTS  MADE  DURING  THE  LAST  2  YEARS  OF  THE  2014-2020 

PROGRAMMING PERIOD AND THE OUTLOOK FOR 2021-2027 

Because the evaluation coincides with the preparation of the new mandate for JASPERS 
that  will  cover  technical  assistance  during  the  2021-2027  programming  period,  a  short 
presentation  of  the  developments  that  took  place  during  2019-2020  is  worthwhile.  The 
following bullet points contain this short presentation. 

-  To improve effectiveness and internal coherence, JASPERS and the EC continued 
their  efforts  to  strengthen  the  coordination  and  early  warning  system  on  issues 
flagged  by  the  advisory  services.  To  this  end,  regular  meetings  were  organised 
between the EC and JASPERS on the one hand, and between JASPERS IQR and 
JASPERS  advisory,  on  the  other.  In  addition,  the  IQR  process  became  more 
formal in 2019-2020, following the example of the submission procedure. 

-  The demand for country-specific capacity-building activities has increased since 

2018, partly because of the need to prepare the 2021-2027 programmes. 

-  The review-services function will no longer exist for the 2021-2027 programming 
period, as the new legislation proposed a new assessment system for projects of 
strategic  importance,  and  this  assessment  is  to  be  carried  out  by  the  Member 
States. 

-  The EC is currently developing new working arrangements, which are planned to 
take  the  form  of  trilateral  work  programmes,  negotiated  for  Member  States  that 
plan  to  use JASPERS  regularly.  These  programmes:  (i)  will set  clear  objectives 
for JASPERS’ support at  country level; (ii) will decide on priority areas  for the 
support; and (iii) are expected to lead to better prioritisation of assignments. 

-  Following  the  ECA  performance  audit  of  JASPERS,  the  EC  drew  up  an  action 
plan  with  measures  to  be  implemented  starting  in  2018.  By  January  2019,  the 
following measures had been implemented fully or were in the process of being 
finalised: (i) measures relating to clarification of the roles and responsibilities of 
the main stakeholders; and (ii) measures relating to the working arrangements of 
the tripartite  cooperation between the EC,  JASPERS and the Member States.  In 
line with the audit recommendations, the EC has requested improvements to the 
monitoring  system.  A  new  set  of  performance  indicators  and  JASPERS  has 
reported regularly on these indicators since late 2018.  

-  On  external  coherence,  in  2019  it  was  decided  that,  for  the  next  mandate, 
JASPERS should be included in the future InvestEU Advisory Hub. This hub will 
be  a  single  point  of  entry  for  EU-funded  technical-assistance  initiatives. 
Currently,  the  EC  is  working  to  implement  this  decision  by  adjusting  the 
JASPERS  governance  system  to  the  context  of  the  Advisory  Hub.  There  was  a 
reorganisation of JASPERS inside the EIB to accommodate these changes. 

36 

 
 
-  Following the CEF pilot  experience with  JASPERS’ assistance, DG MOVE has 
decided to support a renewed mandate for JASPERS’ services for CEF projects in 
the  programming  period  2021-2027.  This  renewed  mandate  has  a  significantly 
larger budget (EUR 30 million for the period 2021-2024), and there is therefore 
increased potential for more extensive assistance to projects in all Member States. 
DG MOVE will delegate its budget to DG REGIO to promote cross-fertilisation, 
coordination and synergies between the cohesion policy programmes and the CEF 
for advisory services of the very same nature on transport projects (including on 
the trans-European transport network (TEN-T)) in all Member States. 

The  developments  in  2019-2020  have  already  addressed  most  of  the  points  for 
improvement  highlighted  by  this  evaluation.  The  DG  REGIO  negotiating  team  for 
JASPERS’ services for the 2021-2027 programming period has been regularly informed 
of  the  progress  of  the  evaluation.  The  evaluation’s  findings  and  conclusions  have  been 
discussed and taken into account  by  DG REGIO as  appropriate. DG REGIO has  given 
particular  attention  to  the  more  strategic  approach  of  the  assistance  to  be  provided  by 
JASPERS (the country-specific trilateral work programmes between the EC, the Member 
States and JASPERS as explained above).  

Two of the main aspects still to be discussed for the JASPERS services in the 2021-2027 
period concern: (i) the monitoring framework for which the conclusions of the evaluation 
will be taken into account; and (ii) knowledge transfer (capacity building) to the Member 
States. 

The proactive approach implemented in the last 2 years of the 2014-2020 programming 
period  provides  reassurance  that  the  development  of  the  JASPERS  initiative  and  its 
development will be closely scrutinised and monitored by the parties involved.   

37 

 
 
 
ANNEX 1: PROCEDURAL INFORMATION 

1.  LEAD DG, DeCIDE PLANNING/CWP REFERENCES 

Directorate General Regional and Urban Policy, Decide Planning: PLAN/2017/2128.  

2.  ORGANISATION AND TIMING 

Tender procedure launched 
Contract signed 
First Steering Group meeting 
Last Steering Group meeting 
Number of Steering Group meetings 
Participating DGs (in addition to DG 
REGIO) 

December 2017 
July 2018 
November 2018 
March 2021 
4 
CLIMA: Climate Action 
ENV: Environment 
MOVE: Mobility and Transport 
SG: Secretary General 
NEAR: Neighbourhood and Enlargement 
ECFIN: Economic and Financial Affairs 

3.  EXCEPTIONS TO THE BETTER REGULATION GUIDELINES 

Due to the specialised topic of the evaluation, the open public consultation was replaced 
by  a  targeted  consultation  of  relevant  stakeholders.  The  targeted  consultation  was 
organised online, during July – September 2019. 

4.  CONSULTATION OF THE RSB (IF APPLICABLE) 

The  draft  staff  working  document  (SWD)  was  presented  at  the  Regulatory  Scrutiny 
Board  (RSB)  meeting  on  19  May  2021.  A  preliminary  upstream  meeting  between  DG 
REGIO and the Regulatory Scrutiny Board took place in July 2018.  

5.  EVIDENCE, SOURCES AND QUALITY 

See section 4 of the Staff Working Document. 

6.  CONSULTATION OF THE REGULATORY SCRUTINY BOARD 

The  draft  SWD  was  presented  at  the  Regulatory  Scrutiny  Board  (RSB)  meeting  on  19 
May 2021 and the RSB issued a negative opinion on 21 May 2021. The RSB comments 
were addressed in this final document as indicated in the following table. 

RSB comment 
1.  The 
report  does  not 
articulate  what  should  have 
been  achieved  by  the  time 
of the evaluation. It does not 
sufficiently  establish  what 
be 
developments 
attributed  to  JASPERS  as 

can 

Reply 
The general aim of the JASPERS initiative, as defined 
by the 2006 Memorandum of Understanding, has been 
to  support  the  implementation  of  Cohesion  Policy  by 
increasing 
for  project 
preparation  and  by  improving  the  quality  of  technical 
advice available to project promoters, thus contributing 
to the preparation of high quality projects.  

resources  available 

the 

38 

 
 
 
RSB comment 
compared to other factors. 

not 

examine 

2.  The 
report  does  not 
adequately  assess  the  EU 
added  value  and  relevance 
of  JASPERS  services.  It 
does 
the 
JASPERS 
of 
potential 
conflicting  objectives  and 
the  risk  of  crowding  out 
capacity 
in 
Member  States,  and  does 
clear  policy 
not 
lessons 
JASPERS’ 
failure  to  meet  its  national 
capacity building objective. 

building 

draw 

from 

the  quality  of 

improvement  of 

Reply 
In the 2014-2020 programming period, whose first five 
years  are  addressed  by  this  evaluation,  the  general 
objective  of  the  JASPERS  initiative  continued  to  be 
the 
investments 
supported by EU Funds. 
The  evaluation  finds  that  the  JASPERS  general 
objective has been broadly achieved, as regards all of 
its specific objectives with the exception of the transfer 
of  technical  knowledge  to  MS  in  view  of  reducing 
their reliance on JASPERS advisory service. This  has 
been  included  in  the  conclusions  of  the  evaluation. 
With  respect  to  the  causal  links,  the  evaluation 
includes  several  econometric  analysis  developed  by 
REGIO  and  detailed  in  section  4.3  of  Annex  4 of  the 
SWD.  For  analyses  requiring 
the  separation  of 
JASPERS’ contribution from other factors for a given 
result,  the  regression  analysis  was  used.  This  is  the 
case,  for  example,  in  the  analysis  of  the  correlation 
between  JASPERS’  assistance  and  the  duration  of 
preparation  of  projects.  Nevertheless,  the  contribution 
of  JASPERS  cannot  be  dissociated  in  all  cases  from 
the other factors that might influence the analysis such 
as  the  legal  framework  in  place  or  the  administrative 
set-up of the major projects approval process. 
JASPERS’  EU  added  value,  as  indicated  by  the 
stakeholders 
support, 
primarily lies in the technical expertise brought by its 
experts who have EU-wide experience acquired while 
working  in  many  countries  and  with  the  European 
the 
Commission.  This  has  been  highlighted 
conclusions of the SWD.  
The  evaluation  analyses  the  relevance  perspective  of 
JASPERS  during  the  2014-2018  period.  The  needs 
analysis  made  by  JASPERS  before  the  2014-2020 
period  is  briefly  described  in  page  4  of  the  SWD. 
Because  of  limitations  described  in  section  4  of  the 
SWD,  the  relevance  of  the  intervention  is  assessed 
mainly  in  terms  of  its  scope  as  compared  with  the 
previous  programming  period,  while  the  connection 
with identified needs is examined in a limited way. 
The  matter  of  technical  capacity  building  in  MS  has 
been thoroughly analysed in the SWD. The conclusion 
was  that  during  the  2014-2018  evaluation  period, 
while  a  large  majority  of  activities  for  capacity 
building  were  completed  and  achieved  their  objective 
of  delivering  training  to  the  Member  States,  they  did 
not prove sufficient to trigger a visible ripple effect on 
the  technical  capacity  of  Member  States  that  would 
allow  to  address  issues  such  as  cost-benefit  analysis, 

engaged  with 

JASPERS 

in 

39 

 
RSB comment 

clear 

enough 

3. The efficiency analysis is 
not 
on 
observed  costs  and  benefits 
of  JASPERS.  The  potential 
for  efficiency  improvement 
is not clearly set out. 

Reply 
climate  change  and  environmental  requirements  in 
project preparation. A detailed analysis of the findings 
that  led  to  this  conclusion  is  presented  in  section  5  – 
Effectiveness of the SWD and in section 5.2 of Annex 
4  to  the  SWD.  The  evaluation,  however,  found 
examples  of  a  pragmatic  approach 
to  capacity 
building, in  the form  of ‘training the trainers’ at  local 
level  as  a 
source  of  continuing  knowledge 
dissemination  or  coordinating  training  on  specific 
issues  across  countries.  This  is  highlighted  in  the 
conclusions of the SWD. 
Furthermore  JASPERS’  assistance  is  seen  as  relevant 
in  the  current  and  future  context  of  the  green  and 
digital  transition  and  increased  funding  opportunities 
for MS, for which the preparation of mature and high 
quality  projects  is  fundamental.  At  the  EC  level,  the 
need  for  technical  assistance  services  linked  to  the 
future project pipeline has been recognized through the 
creation  of  the  InvestEU  Advisory  Hub  in  which 
JASPERS will be embedded. 
JASPERS  primary  objective  was 
the 
submission of better quality projects, in view of faster 
approval  thus  reducing  the  time  resources  of  the  EC 
and MS related to their assessment and leading as well 
to an earlier start of the implementation of investment 
on  the  ground.  The  optimization  of  costs  at  project 
level  was  an  additional  outcome  of  JASPERS’ 
involvement.  As  mentioned 
the  section  on 
limitation, before 2017, data on the actual person-days 
spent  for  each  assignment  by  the  JASPERS  experts 
was not recorded. A more complete dataset for annual 
actual  work  per  assignment  over  the  whole  period 
would  have  been  needed  for  a  full  estimation  of  the 
scope for a more optimal allocation of resources. This 
is why the analysis of the efficiency criterion is made 
based  on  proxy  analysis,  considering  the  available 
data, and it focuses on: 

to  ensure 

in 

-  The administrative closure of assignments;  
-  The  work 
intensity 

distributed 

across 

4.  The  report  is  not  clear 
how  and  why  it  uses  the 
from 
available 
different sources. 

evidence 

assignments (starting from 2017); 

-  The  correlation  between  human  and  financial 
resources and the number of assignments. 
We have further highlighted in  the conclusion section 
the limitations that have influenced the analysis of the 
efficiency criterion. 
The  SWD  has  been  revised  in  order  to  include 
references, where relevant to the data sources. The use 
of  the  various  sources  has  been  further  explained  in 
section 4. Methods of the SWD. 

40 

 
RSB comment 
5.  The  conclusions  do  not 
take  a  critical  view  of  the 
limitations 
and 
shortcomings  identified,  to 
inform policymaking. 

Reply 
The  conclusions  section  has  been  revised  in  order  to 
clarify the implications of the identified limitations and 
shortcomings of the evaluation.  

The recommendations made by the RSB in  terms  of what  to improve in  the text of the 
SWD have been taken into account as follows:  

RSB – what to improve 
1.  The  report  should  clarify  what 
JASPERS  should  have  achieved 
by  2018.  The  report  should  more 
clearly  analyse  the  new  elements 
added  to  JASPERS  in  the  2014-
2020 period. For instance, to what 
extent  has  the  broader  scope  of 
JASPERS  brought  EU  added 
value? 

of 

relevance 

2.  The  report  should  substantiate 
JASPERS 
the 
beyond  the  increased  demand  for 
its free services. Given the limited 
development  of  Member  States’ 
capacity  building  over  the  years 

the 

programming 

the  quality  of 

the  quality  of 

regulations,  with 

initiative  continued 

Reply 
The SWD further clarified the general aim of the 
JASPERS  initiative,  as  defined  by  the  2006 
Memorandum  of  Understanding    -    to  support 
cohesion policy implementation by increasing the 
resources available for project preparation and by 
technical  advice 
improving 
available  to  project  promoters,  thus  contributing 
to the preparation of high quality projects.  
It  specified  that  in  the  2014-2020  programming 
period, of which its first five  years are addressed 
by  this  evaluation,  the  general  objective  of  the 
to  be 
JASPERS 
the 
investments 
improvement  of 
supported by EU Funds. 
For 
period, 
2014-2020 
JASPERS  provided  a  new  kind  of  support  in  the 
form  of  reviewing  the  quality  of  new  major 
projects submitted for EU funding, as required by 
the 
the  aim  of  ensuring 
consistency  in  the  assessment  and  reducing  the 
time  taken.  This  is  explained  in  pages  7-9  of  the 
SWD. The scope of JASPERS has been extended 
in the 2014-2020 period to cover most of the MS, 
as shown in Figure 1 included in the SWD.  
The  SWD  further  clarifies  that  JASPERS’  EU 
added  value,  as  indicated  by  the  stakeholders 
engaged with JASPERS support, primarily lies in 
the technical expertise brought by its experts who 
have EU-wide experience acquired while working 
in  many  countries  and  with 
the  European 
Commission.  This  was  done  in  the  SWD  in  the 
analysis and conclusions related to the EU added 
value criteria.  
The evaluation analyses the relevance perspective 
of  JASPERS  during  the  2014-2018  period.  The 
needs  analysis  made  by  JASPERS  before  the 
2014-2020  period  is  described  in  page  4  of  the 
SWD.  JASPERS  assistance  remains  relevant  in 
the current and future context of green and digital 

41 

 
 
in 

change 

analysis, 

the  Member  States, 

climate 
requirements 

Reply 
transition and increased funding opportunities for 
MS, for which the preparation of mature and high 
quality  projects  is  fundamental.  At  EC  level,  the 
need for technical assistance services linked to the 
future  project  pipeline  has  been  recognized 
through  the  creation  of  the  InvestEU  Advisory 
Hub in which JASPERS will be embedded. 
The  matter  of  technical  capacity  building  in  MS 
has been thoroughly analysed in page 23-24 of the 
SWD.  The  conclusion  was  that  during  the  2014-
2018 evaluation period, while a large majority of 
activities  for  capacity  building  were  completed 
and achieved their objective of delivering training 
to 
they  did  not  prove 
sufficient  to  trigger  a  visible  ripple  effect  on  the 
technical  capacity  of  Member  States  that  would 
allow  to  address  effectively  issues  such  as  cost-
and 
benefit 
environmental 
project 
preparation.  
The  possible  explanations  are  included  in  the 
SWD (“This may be due to high turnover of staff 
in  national  authorities  administrations,  as  noted 
also  by  the  ECA  report,  but  also  due  to  the  fact 
that  often  JASPERS  interacted  with  consultants 
contracted  by  MS  for  the  preparation  of  project 
documentation  and  therefore  the  learning  effect 
for  the  staff  of  national  authorities  is  limited.”) 
and are not seen as being linked to the free nature 
is  deemed 
of 
appropriate 
the 
consultations  with  stakeholders  in  the  context  of 
the  evaluation  took  place  at  a  moment  when  the 
2021-2027  cohesion  policy  package  was  still 
under  negotiation,  therefore  a  certain  attempt  by 
the  MS  for  influencing  its  outcome  may  not  be 
excluded. 
 The SWD further indicates pragmatic approaches 
(such  as  “train  the  trainers”)  that  have  been 
implemented  during  the  period  of  evaluation  and 
that could be potentially replicated in the future in 
relation to technical capacity-building activities.   
Please  see  the  reply  provided  for  point  3  of  the 
table above referring to the RSB comments.  

the  JASPERS’  services. 

to  mention 

though 

that 

It 

critically 

assess 
of 

RSB – what to improve 
of  JASPERS  support,  the  report 
the 
should 
continued 
this 
relevance 
objective  and  whether  JASPERS 
is the most appropriate support to 
achieve 
it.  The  report  should 
assess  if  JASPERS’  free  services 
act  as  a  disincentive  to  develop 
the  Member 
in 
own  capacity 
States. 

3.  The  efficiency  analysis  should 
provide  a  clearer,  comparative 
review  of  costs  and  resources  in 
relation  to  the  benefits  achieved. 
The report should discuss to what 
in  human 
increase 
extent 
resources  in  JASPERS  has  been 

the 

42 

 
to 

and 

there 

optimise 

RSB – what to improve 
proportionate  to  the  benefits  of 
the  programme.  It  should  also 
is 
identify  areas  where 
potential 
the 
allocation of resources. The report 
should explain how JASPERS has 
managed  to  adapt  its  personnel 
capacity  to  changing  workloads 
and  whether  there  is  a  greater 
need for flexibility. 
4.  The  evaluation  should  be  clear 
about  where  causal  links  can  be 
established  between  the  observed 
developments 
JASPERS 
services.  It  should  be  transparent 
where  this  is  not  possible,  and 
discuss  what  other  factors  may 
have played a role. 
5.  The  report  should  present  and 
reference  more 
clearly  what 
evidence  stems  from  the  external 
study  and  the  contribution  of  the 
inforegio report (annex 4) to fill in 
the  gaps.  It  should  clarify  the 
status  of  the  inforegio  report  and 
why 
it  was  needed.  Some 
information  in  annex  could  be 
brought  forward  to  the  report,  in 
order  to  have  a  clearer  picture  of 
base.  Where 
the 
relevant, 
should 
report 
disaggregate the data to provide a 
more nuanced analysis. 
6. The conclusions should be fully 
aligned  with  the  evidence  and  be 
clear about uncertainties. 
7. Given the delay to complete the 
report,  there  is  scope  to  briefly 
explain  its  usefulness  in  policy 
making and informing JASPERS’ 
development  in  the  period  2021-
2027.  The  report  could  outline 
any  changes  introduced  after  the 
period  covered  by  the  evaluation 
and  explain  whether  its  results 
and findings can still form a basis 
for decision making. 

evidence 
the 

Reply 

Please  see  the  reply  provided  for  point  1  of  the 
table above referring to the RSB comments. 

The  SWD  has  been  revised  in  order  to  include 
references,  where  relevant  to  the  data  sources. 
The  use  of  the  various  sources  has  been  further 
explained in section 4. Methods of the SWD. 
The  content  of  the  analysis  has  been  further 
complemented  with  additional  details  from  the 
annexes. 

Please  see  the  reply  provided  for  point  5  of  the 
table above referring to the RSB comments. 

The  changes  introduced  after  the  period  covered 
by the evaluation and the usefulness of the report 
for  policy  making  and  informing  JASPERS’ 
development  in  the  period  2021-2027  have  been 
detailed in section 7 of the SWD. 

43 

 
  
ANNEX 2: STAKEHOLDER CONSULTATION  

The consultation strategy envisaged two main consultation activities as follows: targeted 
online consultation and a seminar with stakeholders. 

Taking  into  account  the  very  technical  character  of  JASPERS  as  a  technical  assistance 
initiative available mainly to public bodies, an open public consultation was not required. 
Yet,  a  targeted  consultation  has  been  organised  so  as  to  consult  broad  range  of 
stakeholders  (Managing  Authorities,  beneficiaries,  institutions  (EIB),  experts  and 
academics,…)  that  have  been  involved  in  JASPERS  process  and/or  have  some 
knowledge of the initiative. 

Online targeted consultation  

The  online  targeted  consultation  was  launched  on  3rd  June  2019  and  closed  after  17 
weeks,  on  30th  September  2019.  The  closure  was  postponed  with  respect  to  the  initial 
agreement (closure after 12 weeks) in order to widen, as much as possible, the group of 
respondents, also by taking advantage of the seminar held on 19th September 2019 during 
which the online consultation was further disseminated. 

The  target  group  comprised  stakeholders  directly  or  indirectly  involved  in  JASPERS 
activities  within  the  different  mandates  and  types  of  assignments  were  included.  They 
could have been either direct beneficiaries of JASPERS support (i.e. project promoters in 
the case of advisory support for major project preparation or REGIO major project team 
in  case  of  a  post-submission  appraisal  report)  or  stakeholders  indirectly  involved  in 
JASPERS  activities  (i.e.  private  consult-ants  preparing  feasibility  studies  for  project 
promoters  and  receiving  comments  by  JASPERS  or  independent  experts  hired  by 
JASPERS  to  develop  some  technical  content  for  the  delivery  of  guidelines  or  training 
sessions).  In  principle,  the  online  consultation  was  designed  for  the  following 
stakeholders: 

• 

• 

• 
• 
• 

EU  staff  (e.g.  REGIO  major  project  team  and  geographical  desks,  NEAR, 
INEA); 
National  coordination  units  (both  Member  States  and  IPA  countries)  for  the 
use of EU funds; 
Managing Authorities; 
Project beneficiaries (e.g. local authorities, service providers); 
External  consultants  and  stakeholders  involved  in  project  preparation  and 
implementation. 

The questionnaire was made available in all official languages of the EU and uploaded 
on  an  online  platform  managed  by  the  external  consultant  engaged  in  the  mid-term 
evaluation  of  the  JASPERS  initiative.  The  replies  to  the  survey  were  collected  in  an 
Excel database.  

Amongst  the  total  of  554  accesses  registered  to  the  link  at  which  the  online  targeted 
consultation  was  available,  the  online  targeted  consultation  collected  a  total  of  210 
“completed”    questionnaires,  whereas  the  other  344  questionnaires  were  considered 
invalid either be-cause they were empty (283) or completed only partially (61). 

44 

 
 
 
Figure 1. Total number of respondents by country 

Source: Mid-term evaluation of the JASPERS initiative 2014-2018, COWI (2021) 

Note:  The  total  number  of  respondents  considered  is  200  (9  European  Commission 
Officials and one multi-country respondent were excluded) 

All  intended  categories  of  stakeholders  have  been  reached.  National  public  authorities 
and service providers/private sector represent the main institutions covered by the online 
consultation,  followed  by  regional  and  local  public  authorities  and  other  types  of 
respondents.  Most  of  respondents  identified  themselves  as  project  promoters  or 
beneficiaries  (45,71%)  or  as  Managing  Authorities  (29%).  A  lower  percentage,  around 
11,43% of the total number of respondents, was represented by Consultancies providing 
technical  expertise.  Another  13%  of  respondents,  instead,  does  not  fall  under  these 
categories  and  identified  themselves  as  European  Commission  Officials,  Cooperating 
institutions, Intermediate Bodies.  

In  general,  respondents  declared  to  be  familiar  with  one  or  two  types  of  JASPERS 
services,  while  only  a  small  percentage  of  them  stated  to  be  familiar  with  all  types  of 
JASPERS services (11%). 

Regarding the frequency of interaction/involvement with JASPERS/JASPERS activities 
in the period 2014-2020, most of respondents taking part in the survey declared to have 
interacted with JASPERS on a frequent (63%) or occasional (25%) basis. Only 12% of 
them, instead, benefitted from JASPERS only once. 

In terms of topics  covered by JASPERS activities, most of respondents  had experience 
on the subjects of cost-benefit analysis and environmental issues and, to a lesser extent, 
also  with  risk  and  sensitivity  analysis  and  the  review  of  application  for  co-financing. 
Conversely, State aid and public procurement result to be the areas of which respondents 
have lower knowledge. 

In  terms  of  effectiveness,  stakeholders  appreciated  positively  the  role  of  JASPERS  in 
strengthening  their  administrative  capacity  (67%)  and  in  improving  the  sectoral  and 
strategic planning (61%).  

With  respect  to  the  relevance  of  JASPERS  services,  the  majority  of  respondents 
expressed  the  need  for  JASPERS  services  for  increasing  the  quality  of  the  projects 

45 

 
 
(97%),  for  ensuring  the  compliance  with  EU  environmental  directives  (90%),  for 
developing  the  skills  related  to  cost-benefit  analysis  (88%).  Most  of  the  consulted 
stakeholders  (86%)  opinionated  that  JASPERS  is  also  needed  for  speeding  up  the 
approval process of the projects.  

In the stakeholders’ view, JASPERS services are suitable to respond to their needs – the 
main subjects expressed as needs were also selected by stakeholders when they positively 
assessed JASPERS ability to respond to these needs. As well the majority of stakeholders 
were  satisfied  with  the  services  provided  by  JASPERS  especially  when  it  comes  to 
improving the general project documentation (85%) and the technical details in particular 
(83%).  

In  relation  to  external  coherence,  as  alternative  advisory  services,  most  of  the 
stakeholders  are  familiar  and  made  use  of  private  consultancy  services  (73%)  while  a 
third of them used also other advisory services of the EIB.  

In  terms  of  areas  where  JASPERS  services  provide  added  value  in  comparison  with 
other advisory services, more than 80% of stakeholders indicated the technical expertise, 
the  hands-on  experience,  the  possibility  of  knowledge  transfer  between  Member  States 
and for ensuring the alignment with EU requirements.  

86% of the Managing Authorities and 92% of the beneficiary/project promoters indicated 
that  despite  improvements  in  internal  capacity,  there  are  still  areas  where  JASPERS 
support is needed. One of the most important areas where JASPERS might contribute in 
the future is ensuring that national strategies are aligned to EU sectoral policies (for 70% 
of Managing Authorities and 89% of beneficiaries/project promoters). It should be noted 
though  that  less  than  a  third  of  the  MAs  and  beneficiaries  are  prepared  to  pay  for 
JASPRS support using own resources for technical assistance.  

Seminar with stakeholders 

As part of the contract signed for the “Mid-term evaluation of the JASPERS initiative in 
2014-2020”,  a  seminar  took  place  at  the  DG  REGIO  premises  in  Brussels  on  19th 
September 2019 with the goal to pre-sent the evaluation’s interim findings and in order to 
collect  feedback  and  insights  from  stakeholders.  The  seminar  was  organised  in  two 
sessions: a first part focused on “Interim findings from the Mid-term  evaluation” and a 
second one on “Future perspectives”. 

Nearly  80  participants  took  part  in  the  seminar  to  a  least  one  of  the  sessions.  Most  of 
them were Managing Authorities (34) or Project beneficiaries (9) representing a total of 
16  Member  States.  A  number  of  JASPERS  experts  (13)  and  European  Commission 
Officials  (around  20)  also  contributed  to  the  discussion  during  the  morning  and  the 
afternoon  session.  The  latter  were  mainly  from  DG  REGIO,  but  also  DG  ECFIN,  DG 
ENV, DG CLIMA and DG MOVE were represented. 

During the first session, the discussion of the findings was structured per type of service 
provided by JASPERS: 

-  Technical  advisory:  the  open  debate  highlighted  several  positive  elements  of 
JASPERS advisory support such as their extensive hands-on experience, smooth 
cooperation, facilitating the relation of the Member State or beneficiary with the 
Commission,  capacity  building  role,  independence,  and  focus  on  quality 
(regarded  as  instrumental  for  countering  politically-driven  projects).  The 

46 

 
establishment  of  a  strong  relationship  between  JASPERS  and  the  authorities  of 
the  Member  States  was  identified  as  a  key  driver,  since  continuity  in  support 
generates efficiency, as well as synergies in combining strategy preparation with 
a  strong  project  pipeline.  Large  consensus  was  found  over  the  importance  of 
involving JASPERS at an early phase in the project cycle and, more generally, on 
the need for a continuation of JASPERS advisory function. 

-  Review  function:  the  stakeholders  opinionated  that,  in  terms  of  internal 
coherence,  a  trade-off  should  be  managed  between  the  necessary  alignment 
among Technical Advisory support and the JASPERS IQR Division on one side 
and  the  independence  of  JASPERS  IQR  division  on  the  other  side.  In  fact,  the 
two JASPERS functions should not dialogue with each other in principle, but at 
the same time convey the same messages to national authorities and beneficiaries. 
Overall, involving JASPERS IQR division is perceived by some Member States 
as a must for ensuring a smooth approval by the Commission. Its effectiveness in 
documentation  improvement  was  particularly  acknowledged  as  a  crucial  benefit 
deriving from appraisal assignments. 

satisfaction  with 

-  Capacity building and horizontal/strategy support: the Member States expressed a 
general 
JASPERS’  capacity  building  activities  and 
horizontal/strategy  support.  JASPERS’  ability  to  prepare  useful  tools  that 
facilitate  the  tasks  of  national  counterparts  and  become  part  of  their  toolkit 
emerged  as  a  significant  advantage.  Train-the-trainers  courses,  according  to 
Member  States’  experience,  were  effective  in  achieving  knowledge  transfer 
amongst  authorities  and  experts’  motivation  was  boosted  by  this  approach.  The 
suggestion  emerged  for  JASPERS  to  be  more  proactive  and  to  organise 
conferences  where stakeholders  share examples of both  good  and bad practices, 
problems and challenges from different projects and countries. 

During  the  second  session,  each  MS  attending  the  seminar  expressed  their  needs 
regarding  the  future  services  to  be  provided  by  JASPERS.  As  an  overall  input,  it  was 
indicated that traditional sectors still require support from JASPERS. At the same time, 
attendees voiced the wish for a strengthening of  JASPERS capacities in  non-traditional 
sectors  (e.g.  broadband  and  smart  development  in  general).  In  addition,  Member  State 
representatives emphasised the need to ensure a smooth transition between programming 
periods. 

The findings that emerged from the public consultation were integrated in the evaluation 
and  triangulated  with  the  finds  from  the  quantitative  analysis  and  the  qualitative  data 
collected via individual interviews with stakeholders. 

47 

 
 
 
ANNEX 3: METHODS AND ANALYTICAL MODELS 

Data 

The data collected and analysed for this evaluation encompassed a variety of sources as 
follows: 

1)  JASPERS’ JADE dataset of assignments for 2014-2020 

Provided by JASPERS for the evaluation, this dataset includes 1124 assignments, 
of  which  1096  are  for  the  current  programming  period  and  the  remaining 
assignments are for preparation for the next programming period 2021-2027. This 
data includes details on the main characteristics of an assignment, such as type of 
service,  dates  of  creation  and  completion/  closure  (if  applicable),  status  by  end 
December  2018,  sector,  and  country.  For  some  of  the  ESIF  major  projects 
submitted to the Commission, the dataset includes also the project reference code 
in  the  Commission’s  SFC2014  system,  although  not  on  a  systematic  basis.  For 
this reason, we matched JASPERS’ assignments with major projects submitted to 
the Commission based on checks performed using data and documents from the 
Commission’s SFC2014 system.  

2)  European Commission SFC2014 database for ESIF major projects59 

SFC2014 is  the system  used by the Commission  and the Member States for the 
management  of  the  ESIF  funds  in  the  programming  period  2014-2020.  For  the 
evaluation we used the SFC2014 data for all major projects notified or submitted 
to  the  Commission  until  October  201960.    This  dataset  includes  320  major 
projects,  and  it  was  used  as  a  basis  for  matching  the  major  projects  with 
JASPERS’ assignments in JADE. The SFC2014 data informs us on the financing 
proposed or approved for the major projects submitted, the timetables of projects, 
as well as a series of other project details included in the project application for 
EU financing. Importantly for the evaluation, the SFC2014 system stores all the 
documentation submitted by the Member States for these projects, including also 
the  PSA  reports  and  the  action  completion  notes  (ACNs)  issued  by  JASPERS’ 
advisory for assisted projects submitted for Art. 102.2, and the IQR reports issued 
by JASPERS IQR for projects assessed for the procedure based on Art. 102.1.  

3)  Assessments and approval of ESIF major projects  

For  the  evaluation  of  JASPERS’  review  function,  from  the  Commission’s 
SFC2014  system  we  collected  structured  data  from  all  the  available  (66)  PSA 
reports issued by JASPERS IQR for the Commission and all (66) Commission’s 
observation letters for all projects submitted for Art. 102.2 and interrupted during 
the approval process. We also screened all (55) IQR reports issued by JASPERS 
for major projects notified for Art. 102.1. The PSA and IQR reports are also the 
main basis on which we matched the projects submitted to the Commission and 
JASPERS’ corresponding advisory and review assignments in JADE.  

59  Throughout the evaluation a distinction is made between JASPERS’ assignments for project preparation and the 
projects themselves. This is due to the fact that it is possible that one assignment covers more than one projects, or 
that a project can be covered by more than one assignments. We explain in text whenever this is the case.  

60  This  was  the  latest  possible  date  when  we  could  update  the  data  based  on  the  SFC2014  system  before 

consolidating the analysis for the evaluation.  

48 

 
 
 
                                                 
4)  Implementation of ESIF major projects  

From the Commission’s SFC2014 system we collected data on the financial and 
physical  implementation  from  the  2019/2018  annual  implementation  reports  of 
166  major  projects  approved  over  the  reference  period.  This  dataset  includes 
projects assisted and not assisted by JASPERS advisory.  

5)  JASPERS’ support for ESIF non-major projects 

For  the  ESIF  non-major  projects  assisted  by  JASPERS  advisory,  we  collected 
structured data from all (49) available action completion notes, project fiches, and 
value added fiches provided by JASPERS. 

6)  CEF projects approved in the current period 

For the analysis of JASPERS’ CEF mandate, we used a dataset with all approved 
CEF  projects  until  August  2019  in  order  to  match  JASPERS’  assignments  with 
these projects, and compare them with non-assisted projects. Overall, this dataset 
includes  265  CEF  projects.  In  addition,  DG  MOVE  provided  for  the  evaluation 
also the final report of the implementation of JASPERS’ CEF1 mandate.  

7)  IPA projects approved in the current period 

For  the  analysis  of  JASPERS’  IPA  mandate  we  matched  the  IPA  assignments 
with the IPA projects approved over the reference period. The dataset of all IPA 
projects  approved  was  provided  by  DG  NEAR  and  it  includes  253  projects 
assisted and not assisted by JASPERS.  

8)  Planning of assignments  

For  the  analysis  of  the  planning  of  assignments  for  JASPERS’  portfolio  we 
collected data on the evolution of these assignments over time from 33 quarterly 
country  action  plans  for  three  countries  sampled  for  this  evaluation  (Croatia, 
Poland, Romania). The country plans were provided by JASPERS. 

9)  Actual work hours per assignment 

For the analysis of efficiency of JASPERS’ services, we analysed a dataset of 190 
assignments  created  and  completed  during  the  period  2017-2019  and  for  which 
JASPERS  had  started  recording  the  actual  working  time  per  assignment.  The 
dataset was provided by JASPERS. 

10) JASPERS’ human and financial resources  

For the analysis of human and financial resources in the section on efficiency, we 
collected structured data on JASPERS’ annual budgets and staff over the period 
2007-2018 from JASPERS’ annual reports published online.  

11) JASPERS’ completed assignments for 2007-2013 

Dataon  assignments  completed  by  JASPERS  for  the  period  2007-2013  was 
collected  from  JASPERS’  online  annual  reports  2010-2018.  On  this  basis,  we 
could identify the assignments by type of service, sector, and country.  

12) Performance of JASPERS activities 2007-2013 

For the comparison of evaluation findings for the current period with equivalent 
findings  from  the  period  2007-2013,  we  collected  comparable  data  from  the 
previous evaluation of JASPERS carried out by DG REGIO in 2012.  

49 

 
 
 
 
13) Major projects approved by the Commission for 2007-2013 

SFC2007 is  the system  used by the Commission  and the Member States  for the 
management  of  ESIF  funds  during  the  programming  period  2007-2013.  From 
SFC2007  we  used  the  data  on  major  projects  approved  by  the  Commission  for 
ERDF  and  Cohesion  Fund  for  the  period  2007-2013.  This  dataset  includes  767 
major  projects,  with  data  on  the  EU  and  total  funding  for  the  projects,  sector, 
country,  fund,  JASPERS  assistance,  and  use  of  framework  contracts  for  project 
appraisals.  

14) JASPERS’ Network Platform online 

From  JASPERS’  Networking  Platform  online  we  collected  data  on  all  training 
and  workshop  activities  included  in  the  annual  programmes  starting  with  2013 
and  until  2018.  In  addition,  we  also  collected  the  data  on  all  publications 
published by JASPERS’ in the online library of the platform. 

15) Stakeholder’s perceptions of JASPERS’ activities 

For  this  evaluation,  we  conducted  an  online  targeted  consultation  with 
stakeholders with direct experience with JASPERS’ services, including Managing 
Authorities,  projects  promoters/  beneficiaries,  professionals  from  consultancy 
companies,  staff  of  the  European  Commission,  etc.  Overall,  210  stakeholders 
responded to this consultation61.  

Furthermore,  through  an  external  contract,  we  also  explored  more  in-depth  the 
implementation  of  JASPERS’  services  based  on  87  assignments  sampled  from 
JASPERS’ portfolio62, and on stakeholders’ interviews from 6 countries sampled for the 
evaluation  (Croatia,  Italy, North  Macedonia, Poland, Romania, and Slovakia). We used 
these additional elements as background information for the evaluation.  

Finally, for the evaluation, we organised also a seminar with stakeholders of JASPERS’ 
services, where we discussed the interim findings of the evaluation  and perspectives on 
the future of JASPERS with representatives from 15 Member States - all with experience 
with JASPERS’ services. The seminar was held in September 2019, in Brussels63. 

Methods 

As  regards  the  methods  of  analysis,  we  used  extensively  statistical  analysis  of  the  data 
collected  for  the  evaluation,  with  an  emphasis  on  median  values  and  distributional 
patterns. Whenever necessary for the comparison of projects assisted by JASPERS with 
projects not assisted, we tested the statistical significance of differences in median values 
for assisted and non-assisted projects by using the Wilcoxon rank-sum test (also known 
as  the  Mann–Whitney  two-sample  statistic).  We  used  this  method  to  understand,  for 
instance,  whether  the  median  number  of  observations  in  Commission’s  interruption 
letters  for  projects  assisted  by  JASPERS  is  lower  than  the  number  of  observations  in 
interruption letters for non-assisted projects.  

61  See Annex 2 of this Staff Working Document.  
62  Among  the  sampled  assignments,  77%  are  for  the  ESIF  mandate,  and  11%  the  CEF  and  IPA  mandate  each, 
covering  all  JASPERS’  services.  JASPERS  provided  all  available  documentation  for  the  sampled  assignments 
including,  for  instance,  project  fiches,  ACNs,  guidance  notes,  value  added  fiches,  and  feedback  data  from 
beneficiaries. 

63  See Annex 2 of this Staff Working Document.  

50 

 
 
                                                 
In  addition,  for  analyses  which  required  separating  JASPERS’  contribution  from  other 
co-founding factors for a given result, the evaluation used regression analysis. This is the 
case,  for  example,  in  the  analysis  of  the  correlation  between  JASPERS’ assistance  and 
the duration of preparation of projects. As regards the regression models used, depending 
on the type of the dependent variable, three types of models were used, as follows: 

 

linear regression analysis with robust standard errors when the dependent variable 
is continuous 
logistic regression models when the dependent variable is binary (1/0) 

 
  ordered  logistic  regression  models  when  the  dependent  variable  is  discrete  with 

ordered values (ordinal).  

51 

 
ANNEX  4:  INFOREGIO  REPORT  “ANALYSIS  AND  ASSESSMENT  OF  SERVICES  AND 
PERFORMANCE OF JASPERS INITIATIVE 2014-2018” 

Analysis and assessment of services and 
performance of JASPERS initiative 
2014-2018 

May 2021 

52 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Disclaimer: The information and views set out in this document are those of the author and 
do not necessarily reflect the official opinion of the Commission. The Commission does not 
guarantee  the  accuracy  of  the  data  included  in  this  report.  Neither  the  Commission  nor 
any person acting on the Commission’s behalf may be held responsible for the use which 
may be made of the information contained therein. 

EUROPEAN COMMISSION 

Directorate-General for Regional and Urban Policy 

Directorate B — Policy Directorate 

Unit B2— Evaluation and European Semester 

E-mail:  REGIO-EVAL@ec.europa.eu   

European Commission 
B-1049 Brussels 

53 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Table of Contents 

1. 

INTRODUCTION ................................................................................................... 56 
Purpose and scope ......................................................................................... 56 

1.1 

2. 

2.1 

JASPERS INITIATIVE AND ITS INTERVENTION LOGIC .............................. 57 
Description of JASPERS and its objectives .................................................. 57 
ESIF Mandate .......................................................................................................... 58 
CEF Mandate .......................................................................................................... 61 
IPA Mandate ............................................................................................................ 62 
JASPERS’ intervention logic ........................................................................ 63 
Baseline and points of comparison ................................................................ 66 
Baseline: programming period 2007-2013 ............................................................. 66 
Points of comparison ............................................................................................... 69 
Previous evaluations and audits ..................................................................... 69 

2.2 
2.3 

2.4 

3. 

JASPERS’ PORTFOLIO OF ACTIVITIES DURING THE PERIOD 2014-2018 73 

4.  EVALUATION CRITERIA, DATA AND METHODS ........................................ 78 
Evaluation criteria and issues evaluated ........................................................ 78 
Data ................................................................................................................ 79 
Methods ......................................................................................................... 82 
Robustness and limitations of findings .......................................................... 82 

4.1 
4.2 
4.3 
4.4 

5.2 

5.1 

5.  EVALUATION FINDINGS AND SUPPORTING EVIDENCE ........................... 84 
RELEVANCE ............................................................................................... 84 
Overall portfolio ...................................................................................................... 88 
ESIF Mandate .......................................................................................................... 89 
CEF Mandate ........................................................................................................ 109 
IPA Mandate .......................................................................................................... 111 
Stakeholder consultation ....................................................................................... 114 
EFFECTIVENESS ...................................................................................... 116 
ESIF Mandate ........................................................................................................ 120 
CEF Mandate ........................................................................................................ 149 
IPA Mandate .......................................................................................................... 151 
Stakeholders consultation ...................................................................................... 155 
EFFICIENCY .............................................................................................. 155 
Planning and management of assignments ........................................................... 157 
Human and financial resources............................................................................. 166 
COHERENCE ............................................................................................. 169 
Coherence with EU priorities ................................................................................ 171 
Coherence with other advisory services at EU level ............................................. 173 
Internal coherence between the advisory and review functions ............................ 174 
EU ADDED VALUE .................................................................................. 176 

5.4 

5.3 

5.5 

54 

 
 
 
 
6. CONCLUSIONS ....................................................................................................... 180 
JASPERS’ portfolio of services ................................................................................. 180 
Technical advisory for ESIF major projects .............................................................. 180 
Technical advisory for ESIF non-major projects ....................................................... 182 
Technical advisory for CEF projects ......................................................................... 183 
Technical advisory for IPA projects .......................................................................... 185 
ESIF review function ................................................................................................. 186 
Capacity building and horizontal and strategic support............................................. 186 
Stakeholders’ views on the relevance and effectiveness of JASPERS’ services ...... 188 
Efficiency of planning and work allocation ............................................................... 189 
External and internal coherence ................................................................................. 191 
EU added value .......................................................................................................... 192 

REFERENCES .............................................................................................................. 193 

ANNEX 1: REPLIES TO THE ONLINE TARGETED CONSULTATION  .............. 196 

ANNEX 2: DATA AND REGRESSION MODELS .................................................... 206 

ANNEX 3: JASPERS SUPPORT IN 2007-2013 .......................................................... 215 

ANNEX 4: JASPERS SUPPORT 2014-2020 ............................................................... 221 

55 

 
 
 
 
 
1.  INTRODUCTION 

1.1 

Purpose and scope 

JASPERS (Joint Assistance to Support Projects in European Regions) is a joint initiative 
for technical advisory of the European Commission and the European Investment Bank 
(EIB)64. JASPERS was effectively launched in 2006 with the main objective to help the 
new  Member  States  use  the  available  EU  funding  better  and  faster.  Over  a  period  of 
fourteen  years,  JASPERS  has  grown  to  be  one  of  the  largest  initiatives  for  technical 
assistance supported by the European Commission, with services available to all Member 
States and pre-accession countries, and in all sectors relevant for EU investments.    

The objective of this evaluation is to assess the implementation of JASPERS’ activities 
and its performance during the first five years of the current programming period 2014-
2020.  The  evaluation  is  designed  according  to  the  European  Commission’s  Better 
Regulation  guidelines,  and  it  is  structured  around  the  five  main  evaluation  criteria: 
relevance, effectiveness, efficiency, coherence, and EU added value.  

As  regards  the  scope  of  the  evaluation,  it  covers  all  JASPERS’  activities  implemented 
until end 2018 across all Member States and sectors. These activities are assessed also in 
the  broader  context  of  the  decision  making  procedures  for  the  approval  of  projects 
submitted for EU funding for the Commission services relevant for JASPERS activities.  

The structure of this evaluation report is as follows. Next, in Chapter 2, we explain the 
main coordinates of JASPERS activities and the logic of intervention of its services, as 
relevant for the current programming period. The chapter also includes a presentation of 
the  main  baseline  and  points  of  comparisons  for  the  evaluation  and  a  summary  of  the 
main  findings  of  other  assessments  of  JASPERS  published  before  this  evaluation.  In 
Chapter 3, we present the JASPERS’ portfolio of activities for the current programming 
period,  as  accumulated  over  the  reference  period  for  the  evaluation  2014-2018,  and  its 
evolution  over  these  years.  Chapter  4  explains,  by  evaluation  criterion,  the  main  issues 
analysed in the evaluation, the data collected as evidence, and the methods used for the 
analysis. This chapter concludes with our assessment of the robustness and limitations of 
the  evaluation  findings.  The  evaluation  findings  and  the  supporting  evidence  are 
presented  in  Chapter  5,  with  sections  dedicated  to  each  of  the  five  evaluation  criteria. 
Additional evidence supporting the evaluation findings is included in the annexes to the 
document. Finally, Chapter 6 includes a discussion of the main results of the evaluation 
and the conclusions drawn on this basis.  

64  Until 2013, two more partners were involved in setting up the initiative: the European Bank for Reconstruction 

and Development (EBRD) and Kreditansalt fur Wiederaufbau (KfW).  

56 

 
 
 
 
 
 
 
                                                 
2.  JASPERS INITIATIVE AND ITS INTERVENTION LOGIC 

2.1 

Description of JASPERS and its objectives 

Since  the  launch  of  the  initiative  and  until  2013,  JASPERS  worked  on  providing  an 
independent  and  free-of-charge  advisory  service  to  national  authorities  in  EU  Member 
States and pre-accession countries for the preparation of quality projects co-financed by 
the European Regional Development Fund (ERDF) and the Cohesion Fund. This activity 
was  carried  out  under  the  ESIF  mandate  of  the  Directorate  General  of  Regional  and 
Urban  Policy  (DG  REGIO)  in  the  European  Commission.  The  mandate  included  also 
activities for capacity building and horizontal and strategic support.  

In  the  current  programming  period  2014-2020,  relative  to  2007-2013,  JASPERS’ 
activities expanded both in scope and volume. In terms of scope, JASPERS carries out its 
core  task  of  technical  advisory  for  project  preparation  for  additional  two  mandates 
supported  by  the  Directorate  General  for  Mobility  and  Transport  (DG  MOVE)  for 
Connecting  Europe  Facility  (CEF)65  projects,  and  Directorate  General  for  European 
Neighbourhood Policy and Enlargement Negotiations (DG NEAR) for the Instrument for 
Pre-Accession  Assistance  (IPA  II)66  assignments.  Activities  for  capacity  building  and 
horizontal  and  strategic  support  are  also  included  in  the  scope  of  these  two  new 
mandates.  In  addition,  a  novelty  for  the  current  period  is  JASPERS’  review  function 
through which it provides services for the assessment of major projects to Member States 
and to the Commission in the context of the ESIF mandate. In terms of volume, during 
the first five years of 2014-2020, JASPERS started more assignments than for the entire 
period 2007-2013.   

JASPERS  activities  in  the  current  period  are  based  on  the  Framework  Partnership 
Agreement67 established in 2014 between the European Commission and the EIB for the 
ESIF mandate, and subsequently  amended in  2015 to  include the CEF mandate, and in 
2016 for the IPA II mandate. 

In  organisational  terms,  JASPERS  is  organised  in  7  divisions,  of  which  5  are  thematic 
(for  Roads,  Rail,  air,  and  maritime,  Water  and  wastewater,  Smart  development,  and 
Energy and solid waste). The two other divisions are for the Independent Quality Review 

65  CEF  is  the  EU  funding  instrument  supporting  interconnected  trans-European  networks  in  the  field  of  transport, 
energy, and digital services. JASPERS’ CEF mandate in the period 2014-2020 covers CEF projects in the field of 
transport.   

66  The  IPA  II  instrument  sets  the  framework  for  pre-accession  assistance  for  2014-2020.  Beneficiary  countries 

include Albania, Bosnia and Herzegovina, Kosovo, Montenegro, North Macedonia, Serbia, and Turkey.  

67  Framework Partnership Agreement (FPA) for the management of the JASPERS technical assistance facility 2014-
2020. The FPA is based on the broader Financial and Administrative Framework Agreement (FAFA) signed by 
the European Commission and the EIB in 2014. 

57 

 
 
 
 
 
 
  
                                                 
(IQR  division)  and  the  Networking  and  Competence  Centre  (NCC  division)68.  In 
addition, JASPERS includes a Quality Management Unit. JASPERS’ head office is in the 
EIB  in  Luxembourg,  and  JASPERS  IQR  division  is  located  in  Brussels.  Its  thematic 
divisions are located across regional offices in Bucharest, Sofia, Vienna and Warsaw, and 
the  teams  managing  the  different  assignments  can  combine  people  from  different 
regional offices depending on their expertise and availability. 

In the following paragraphs we present the main coordinates for the three mandates and 
services provided by JASPERS in the current period.  

ESIF Mandate 

From the operational perspective, JASPERS’ ESIF mandate is based on annual Specific 
Grant  Agreements  (SGAs)  signed  between  the  Commission  and  the  EIB  for  the 
implementation of JASPERS’ activities. These agreements define the scope and priorities 
of  JASPERS’  activities,  the  conditions  for  implementation  and  the  annual  resources 
allocated.  

JASPERS’ ESIF mandate includes four main types of services : 1) technical advisory for 
preparation  of  major  projects ;  2)  technical  advisory  for  preparation  of  non-major 
projects ; 3) assessment of major projects through the review function; and  4) activities 
for capacity building and horizontal and strategic support. In addition, in the context of 1) 
and 2), JASPERS can also provide assistance for implementation of projects. Services of 
type 1), 2) and 4) are continued from the previous period, while  the review function is 
new.  

As  regards  technical  advisory  for  ESIF  major  projects69,  the  objective  of  JASPERS’ 
assistance is to help Member States in preparing such projects better and faster in order to 
access EU funding. JASPERS provides this service at the request of the Member States, 
and it carries out the activity with the agreement of the European Commission. Advisory 
for major projects is available for all Member States, with priority to less developed and 
transition regions.  

In the current programming period, applying for EU funding for a major project entails 
detailed preparation of project technical specifications, a feasibility study, a cost benefit 
analysis  and  an  environmental  impact  assessment.  The  project  application  summarizes 
not  only  the  technical,  economic  and  financial  feasibility  of  the  project  documented  in 
the  above  studies,  but  also  aspects  related  to  the  alignment  of  the  project  with  the 

68 

JASPERS IQR unit was created in 2014 and was staffed to appraise all categories of major projects developed by 
the advisory units (based on JASPERS’ Quality Manual 2018). The reader should note that all legislative texts and 
documents cited in text are presented will full details in the last section of References. 

69  Major  projects  are  defined  as  projects  for  which  the  total  eligible  costs  exceed  EUR  50  million,  except  for  the 
transport  sector  for  which  the  threshold  is  EUR  75  million.  For  more  details  see  the  Common  Provisions 
Regulation (EU) No 1303/2013, Art. 100. 

58 

 
 
   
 
 
 
 
                                                 
objectives of EU Cohesion Policy, and with the EU requirements and standards regarding 
environmental impacts, adaptation and mitigation measures for climate change, and EU 
competition  rules70.  An  application  for  EU  financing  for  a  major  project  is  subject  to 
Commission’s approval.  

In  this  context,  the  role  of  JASPERS  advisory  is  to  provide  assistance  on  any  of  these 
aspects of project preparation, if and when required by the Member State. While working 
on an assignment for advisory of major projects, JASPERS issues a series of documents, 
some procedural and some for guidance addressed to the project beneficiary. Among the 
procedural documents, the most important type of document that we used extensively for 
this  evaluation  is  the  Action  Completion  Note  (ACN).  In  an  ACN  JASPERS  describes 
the project assisted, its contribution to project preparation during the advisory work, and 
it  identifies  also  significant  or  other  type  of  issues  still  outstanding  with  the  project 
quality  at  the  time  when  the  assignment  is  completed.  As  regards  the  guidance 
documents, these are notes issued by JASPERS during the advisory work which serve to 
guide the project beneficiary in the preparation of the project.   

JASPERS’  technical  advisory  for  non-major  projects  is  organised  in  a  similar  manner 
and  with  similar  objectives71.  According  to  the  SGAs,  the  advisory  for  non-major 
projects  should  prioritise  EU  countries  with  fewer  major  projects  such  as  Cyprus, 
Estonia,  Latvia,  Lithuania,  Slovakia,  Slovenia,  and  Malta.  Further,  the  SGAs  mention 
that  non-major  projects  can  be  assisted  also  in  other  Member  States  if  they  are  pilot 
projects  of relevance to  EU policy objectives and/or projects  which could  be replicated 
on a much wider scale by national authorities. From the operational perspective, upon the 
request  from  a  Member  State  and  with  the  agreement  of  the  European  Commission, 
JASPERS  opens  an  assignment  for  the  project  and  issues  similar  documents  (such  as 
guidance  notes  and  ACNs)  as  for  the  major  projects.  In  contrast  to  the  major  projects, 
however,  the  applications  for  EU  financing  for  non-major  projects  are  assessed  and 
approved by the Managing Authorities in the Member States.  

With  regard  to  the  review  function,  it  is  designed  to  assist  the  Member  States  and  the 
Commission  with  assessments  of  whether  the  major  projects  prepared  fulfil  the  quality 
criteria and the regulatory compliance required for EU funding as follows.  

For  the  programming  period  2014-2020,  at  the  Commission  proposal,  the  legislators 
provided for a new system for the appraisal and approval of new major projects funded 
from the EU Cohesion Policy. This system entails an independent quality review of the 

70  Regulatory  requirements  for  the  application  for  EU  funding  for  a  major  project  are  laid  down  in  the  Common 
Provisions Regulation (CPR) (EU) 1303/2013, Art. 101. Regulation (EU) 2015/207 includes the template for the 
application in Annex II, and the minimum requirements for a cost benefit analysis in Annex III. Furthermore, for 
the  cost  benefit  analysis,  the  European  Commission  also  published  a  guide  to  CBA  of  investment  projects  for 
Cohesion Policy in 2014 (see DG REGIO(2014) in References).    

71  Non-major  projects  are  projects  which  do  not  fulfill  the  regulatory  requirements  for  a  major  project  explained 

earlier.  

59 

 
 
 
 
 
 
                                                 
major projects applying for EU funds, and it is organised in two main procedures. First, 
the procedure based on Art 102.1 in the Common Provisions Regulation (CPR)72 requires 
the  Member  State  to  notify  to  the  Commission  the  application  for  a  major  project 
accompanied by an Independent Quality Review performed by independent experts73. A 
second  possibility  for  access  to  EU  funding  for  major  projects  is  provided  for  by  Art 
102.2  in  the  CPR.  For  this  option,  the  Member  State  needs  to  submit  the  full  project 
documentation  and  the  project  application  for  the  Commission’s  assessment  and 
approval74,75.  

In  line  with  these  regulatory  procedures  for  the  adoption  or  approval  of  new  major 
projects,  JASPERS’  review  function  includes  two  main  strands:  a)  the  Independent 
Quality  Review  (IQR),  organized  as  a  service  provided  to  the  Member  States  for  the 
assessment  of projects  subsequently  notified for  Commission’s adoption based on CPR 
Art.  102.1,  and  b)  the  Post  Submission  Assessment  (PSA)  service  provided  to  the 
Commission  for  support  with  the  assessment  of  projects  submitted  for  Commission’s 
approval based on CPR Art 102.2.  

The  main  differences  between  the  two  types  of  assessments  include  the  following.  For 
the IQR, the Member State submits all project documentation directly to JASPERS IQR 
division for assessment. The IQR division in JASPERS uses the same methodology for 
project  appraisal  as  the  advisory  division.  Upon  assessing  the  project,  however, 
JASPERS  IQR  issues  its  own  opinion  whether  the  project  meets  the  required  quality 
criteria  and  confirms  its feasibility  and  economic  viability.  The  IQR  report  includes  an 
assessment  covering  all  aspects  of  the  project  documentation  and  application  and  it  is 
sent  to  the  Member  State  upon  completion.  If  the  IQR  conclusions  are  positive,  the 
Member  State  notifies  the  project  application  together  with  the  IQR  report  to  the 
Commission for approval based on Art. 102.1.  

For  the  procedure  based  on  Art  102.2,  the  Member  State  submits  the  full  project 
documentation together with the application to the Commission since, in this case, it is 
the  Commission  who  assesses  the  quality  of  the  project  submitted.  Upon  receiving  the 
project  documentation  and  the  application  for  EU  financing,  the  Commission  asks 
JASPERS IQR  division  to  assess all aspects  of the project,  except  for  alignment of the 
project  with  the  operational  programme,  state  aid,  environmental  issues,  and  the 
completeness of project application. JASPERS IQR division issues an opinion on project 

72  The Common Provisions Regulations (CPR) stands for Regulation (EU) No 1303/2013.  
73  The requirements for the independent experts carrying out the quality review and the steps for the quality review 
are regulated in Commission Delegated Regulation 480/2014. The template for the notification of a major project 
based on CPR Art. 102.1 is included in the Commission Implementing Regulation (EU) No 1011/2014. 

74  CPR Art. 101 lists the documentation necessary for the approval of a major project submitted for the procedure 

based on Art. 102.2. 

75  A  third  procedure  based  on  CPR  Art.  103  refers  to  phased  projects.  These  are  projects  approved  during  2007-
2013, and which are continued with a distinct phase in 2014-2020. The evaluation focuses less on phased projects 
since, from the perspective of project preparation in 2014-2020, they are already prepared in the previous period, 
and therefore not comparable with the new major projects.  

60 

 
 
 
 
 
                                                 
quality in the form of a PSA report sent to the Commission. When compared to an IQR 
report,  a  PSA  report  has  a  different  structure  in  that  it  is  less  comprehensive,  and  it 
distinguishes  between  critical  and  non-critical  issues  identified  for  project  quality  and 
documentation for the topics covered76,77. 

Finally,  as  regards  the  capacity  building  and  horizontal  and  strategic  support,  these  are 
activities  which  aim  at  strengthening  the  capacity  for  project  identification,  preparation 
and  implementation  in  the  Member  States,  as  well  as  at  providing  support  for  the 
the 
preparation  of  national  and  regional  strategies  and  plans  necessary  for 
implementation of EU Cohesion Policy. From the operational perspective, the process of 
assignment creation and implementation is similar as for the advisory function. Also for 
this type of service, JASPERS issues ACNs when finalising the assignments.  

CEF Mandate 

The  Connecting  Europe  Facility  (CEF)  conducted  two  pilots  with  small  budgets  for 
experimenting  with  JASPERS’  assistance  to  the  Member  States,  and  for  which  DG 
MOVE signed with JASPERS two Specific Grant Agreements (SGAs) in 2015 and 2018. 
The  scope  of  the  CEF  mandate  includes  advisory  work  for  preparation  and 
implementation  of  projects,  as  well  as  advice  on  horizontal  issues  relevant  for  one  or 
more projects, or for one or more countries.  

JASPERS started its work with CEF projects under a first pilot mandate (CEF1.1) carried 
out  between  July  1st,  2015  and  March  31st,  2018.    This  first  CEF  mandate  had  the 
objective  to  provide  JASPERS’  technical  assistance  for  preparation  of  eligible  projects 
with  the  intention  to  be  submitted  to  CEF  calls  for  the  Cohesion  envelope  of  EU 
financing78.  Countries  and  projects  to  be  assisted  were  identified  based  on  a  list  of 
priority projects annexed to the first Specific Grant Agreement (Annex I).  

For the period 2018-2020, DG MOVE decided to continue working with JASPERS for a 
second  pilot  mandate  (CEF1.2).  In  order  to  improve  the  promotion  of  JASPERS  in  all 
Member  States  and  to  facilitate  its  access,  in  this  case  the  entry  point  for  JASPERS’ 
advisory  for  CEF  projects  was  the  European  Investment  Advisory  Hub  (EIAH)79.  The 

76  A critical issue is defined as an instance of not fulfilling the minimum criteria for quality review of major projects 
listed in the Commission Delegated Regulation (EU) No 480/2014, or of non-compliance with relevant EU legal 
obligations. A non-critical issue applies when the quality standards for the project, or information requirements, 
are  not  fully  met  but  this  does  not  affect  the  overall  positive  assessment  of  the  project  (based  on  JASPERS’ 
Quality Manual (2018)). 

77  More details on the structure of these reports are included in the section for coherence in Chapter 5. 
78  The CEF cohesion envelope is the budget transferred from the Cohesion Fund to the CEF funding programme. 
This budget is reserved for projects in Member States eligible for the Cohesion Fund.  Cohesion Member States 
include:  Bulgaria,  Croatia,  Cyprus,  Czechia,  Estonia,  Greece,  Hungary,  Latvia,  Lithuania,  Malta,  Poland, 
Portugal, Romania, Slovakia, and Slovenia. 

79  The European Investment Advisory Hub (EIAH) is a partnership between the EIB and the Commission as part of 
the Investment Plan for Europe. It is designed to act as a single access point to various services for advisory and 

61 

 
 
 
 
 
 
                                                 
technical assistance under this mandate was available for CEF projects applying for CEF 
Blending calls80, with a particular focus, though not limited to, cohesion Member States. 
Therefore,  this  second  mandate  expanded  the  geographical  scope  of  assistance  to  all 
Member States and focused more on an innovative form of financing (blending).  

From  the  operational  perspective,  in  a  similar  manner  as  for  ESIF  projects,  JASPERS 
accepts  assignments  for  support  to  preparation  of  CEF  projects  requested  by  Member 
States and upon the agreement of the European Commission. When a CEF assignment is 
finalized,  JASPERS  advisory  issues  a  completion  note  called  Project  Development 
Report (PDR).  

IPA Mandate 

JASPERS’  IPA  mandate  for  the  current  period  functions  on  the  basis  of  the  Specific 
Grant  Agreement  (SGA)  signed  with  DG  NEAR  in  2016.  Its  advisory  work  for  IPA 
projects aims at providing technical support for all phases of preparatory work for these 
projects.  Technically,  JASPERS  has  two  mandates  for  the  IPA  countries :  the  IPA  II 
mandate  covering  all  pre-accession  countries  for  2016-202081,  and  the  Serbia  National 
Programme  agreement  for  2016-2019.  For  more  flexibility  in  narrative,  however,  we 
refer to both mandates as the “IPA mandate” for the current period.  

According to the SGA for the IPA mandate, JASPERS’ support is to concentrate on the 
following issues: a) projects identified to be part of the Single Pipeline of Projects (SPP) 
or  by  the  National  IPA  Coordinator  in  the  case  of  Turkey;  b)  transport  and  energy 
projects for Western Balkans 6 Connectivity Agenda; c) preparation of sector strategies 
and project pipelines for IPA II; and d) know-how transfer (such as cost benefit analysis, 
environmental impact assessment etc.). 

From the operational perspective, JASPERS accepts an assignment for the IPA mandate 
upon the request  from the beneficiary  country  and with  the agreement  of the European 
Commission.  When  finalizing  the  assignments,  JASPERS  advisory  issues  action 
completion notes (ACNs). 

Finally,  for  all  these  three  mandates,  JASPERS  formalizes  its  work  on  assignments  in 
rolling country action plans maintained throughout the period. According to JASPERS’ 
Quality Manual82, the country action plans include working arrangements signed with the 

technical assistance. It supports the identification, preparation and development of investment projects across the 
European Union.  

80  CEF Blending Operations include investments combining the use of grants and/ or financial instruments from the 
EU  budget  and  financing  from  Implementing  partners  (if  under  CEF  Blending  Facility)  or  from  financing 
institutions  (private  or  public  for  2017  Blending  calls)  (via  loan,  debt,  equity  or  any  other  repayable  form  of 
support). 

81  Albania, Bosnia and Herzegovina, Kosovo, the Republic of North Macedonia, Montenegro, Serbia and Turkey. 
82  See JASPERS’ Quality Manual (2018) in References. 

62 

 
 
 
 
 
 
 
                                                                                                                                                 
beneficiary  countries  and  the  list  of  projects  and  areas  where  JASPERS  is  likely  to  be 
asked for support. These plans are updated in line with the requests for support received 
from the beneficiary countries. 

In the following section we present JASPERS’ intervention logic defined by main type of 
service. 

2.2 

JASPERS’ intervention logic  

The main coordinates of the intervention logic of JASPERS’ activities include the type of 
activities, deliverables, and expected results and impacts. We start with the intervention 
logic for the services of technical advisory support presented in Figure 2.2.1 

Figure 2.2.1: Intervention logic for JASPERS’ advisory support 

For  advisory  services,  the  main  specific  objective  of  JASPERS’  initiative  is  to  enable 
better  preparation  of  the  projects  intended  for  EU  financing,  thus  contributing  to  their 
faster approval and to the more general goal of achieving a better implementation of EU 
funds.  Therefore,  the  immediate  results  of  the  assignments  delivered  are  expected  in 
terms  of  better  projects  prepared  with  the  assistance  of  JASPERS  and  submitted  for 
approval of EU financing, and of timely approval of these projects.  

In case of ESIF major projects, timely approval is reflected in a smooth decision process 
in the Commission, without interruptions due to quality issues. For CEF projects, on the 
other hand, results are reflected by the submission of the projects to CEF competitions.  
Similarly, for IPA projects, results are indicated by projects’ approval for IPA financing.  

Next,  the  intervention  logic  for  JASPERS’  review  function  for  the  ESIF  mandate  is 
illustrated  in  Figure  2.2.2  below,  distinguishing  between  the  two  main  types  of  quality 
assessments: IQR and PSA.  

Figure 2.2.2: Intervention logic for JASPERS’ review function 

63 

 
 
  
 
 
 
 
 
 
 
The specific objective of JASPERS’ review activities is to provide an expert independent 
assessment of the quality of a major project submitted for EU financing from ERDF or 
Cohesion  Fund.  As  explained  above,  in  the  case  of  IQR,  the  review  is  more 
comprehensive as JASPERS IQR needs to vouch for the project economic and financial 
viability and its compliance with EU requirements and standards. In the case of the PSA 
review, on the other hand, JASPERS IQR supports the Commission with its expert view 
on the more technical aspects of the project and its economic and financial viability.  

In both cases, however, the immediate expected result is a timely approval of projects of 
good quality, and the more general objective is better implementation of EU funds.  

For  capacity  building  and  horizontal  and  strategic  support,  the  intervention  logic  is 
presented in Figure 2.2.3. 

Figure  2.2.3 :  Intervention  logic  for  capacity  building  and  horizontal  and  strategic 
support 

Through these activities, JASPERS’ specific objective is to assist the beneficiary country 
in the identification, preparation and implementation of projects for EU funding through 
transfer  of  knowledge.  In  this  process,  JASPERS  leverage  is  based  on  its  experience 
accumulated  by  working  on  a  variety  of  projects  across  sectors  and  countries,  and  in 
close  cooperation  with  the  European  Commission.  In  doing  so,  JASPERS’  services  are 
expected  to  contribute  to  enhanced  technical  capacity  in  the  beneficiary  countries  – 

64 

 
 
 
 
 
 
 
 
 
capacity  relevant  for  a  better  implementation  of  EU  funds  due  to  more  effective 
identification and preparation of projects.  

In  the  remaining  sections  of  this  chapter,  we  present  briefly  JASPERS’  activities  until 
2013 (that is, prior to the reference period for this evaluation), main points of comparison 
in  the  current  evaluation,  and  the  main  findings  on  JASPERS’  past  activities  and 
performance from previous evaluations and audits.    

65 

 
 
 
 
 
2.3 

Baseline and points of comparison  

Baseline: programming period 2007-2013 

Over  the  period  2007-2013,  the  European  Commission  invested  132  million  euro, 
representing  71%  of  the  total  costs  with  JASPERS’  initiative  over  the  period.  The 
remaining  29%  was  covered  primarily  by  the  EIB  with  in-kind  support  for  additional 
technical  expertise,  and  also  by  EBRD  and  KfW.  For  this  period,  JASPERS’  work 
contributed to mobilising more than 32 billion euro ERDF and Cohesion Fund resources 
through  383  major  projects  assisted  by  JASPERS  for  preparation  and  subsequently 
approved by the Commission83. 

As  explained  earlier,  during  the  period  2007-2013,  JASPERS’  work  was  based  on  the 
ESIF  mandate  for  DG  REGIO.  For  this  period,  JASPERS  completed  around  790 
assignments for projects for EU Cohesion Policy, and 6 assignments for IPA projects. A 
proportion  of  82%  of  these  activities  were  for  the  advisory  function  for  preparation  of 
major  and  non-major  projects  and,  to  a  lesser  extent,  for  project  implementation.  The 
remaining  activities  were  dedicated  primarily  to  horizontal  and  strategic  support.  In 
Figure  2.3.1  we  present  the  distribution  of  these  activities  by  sector  and  main  type  of 
assignment.  

83  The total number of major projects (assisted and non-assisted for preparation) adopted for the period 2007-2013 is 
767, mobilising close to 65 billion euro from ERDF and Cohesion Fund. The EU resources mobilized by projects 
assisted  by  JASPERS  mentioned  in  text,  however,  underestimate  the  total  amount  of  EU  funding  for  assisted 
projects as they do not include the support for non-major projects, for which we do not have an estimate. 

66 

 
 
 
 
 
 
 
                                                 
Figure  2.3.1  Distribution  of  JASPERS’  activities  by  sector  and  type  assignment,  2007-
2013 (number assignments) 

Note: CB = Capacity Building, HSS = Horizontal and Strategic Advice ; data covers 796 
assignments. 
Source : JASPERS’ annual reports 2010-2013 

On  this  basis,  we  learn  that  the  sector  of  Water  and  wastewater  treatment  was 
predominant  in  JASPERS’  portfolio,  with  a  third  of  all  assignments  for  the  period, 
followed by the sectors of Knowledge economy, energy, and waste combined. As regards 
the  type  of  service,  we  note  that  the  activities  for  capacity  building  and  horizontal  and 
strategic support focused to a larger extent on the sectors of Knowledge economy, energy 
and waste, Water and wastewater, and on multi-sector assignments. 

As  regards  the  distribution  of  assignment  by  country,  Figure  2.3.2  presents  the  details. 
The  distribution  by  country  indicates  that  76%  of  JASPERS’  activities  in  2007-2013 
were  concentrated  in  five  beneficiary  countries:  Poland,  Romania,  Bulgaria,  Czech 
Republic and Hungary. The remaining assignments were distributed across additional 9 
Member States and 3 IPA countries. 

Finally, for a perspective of implementation over time, we present also the dynamics of 
assignments  in  JASPERS’  portfolio  reported  as  completed  on  an  annual  basis  over  the 
period 2007-2013. Figure 2.3.3 presents the details. 

67 

 
 
 
 
 
 
 
 
Figure 2.3.2: Distribution of JASPERS’ activities by country and type assignment, 2007-
2013 (number assignments) 

Note: CB = Capacity Building, HSS = Horizontal and Strategic Advice ; data covers 796 
assignments 
Source : JASPERS’ annual reports 2010-2013 

Figure 2.3.3 Assignments completed by JASPERS annually, period 2007-2013 (number 
assignments) 

Note: CB = Capacity Building, HSS = Horizontal and Strategic Advice ; data covers 796 
assignments 
Source : JASPERS’ annual reports 2010-2013 

68 

 
 
 
 
 
 
From this data, we learn that the peak year for completion of assignments was the 2010 
(the fourth year of the programming period), with the annual number (160) representing 
20%  of  all  assignments  completed  for  the  period).  Moreover,  by  the  fifth  year  of  the 
period  (i.e.  the  year  comparable  with  the  reference  end  date  for  the  evaluation  of  the 
current  period),  JASPERS  had  carried  out  68%  of  all  assignments  completed  in  the 
period 2007-2013. 

Points of comparison 

As regards the previous period, we compare JASPERS’ activities and performance in the 
current period relative to the first five years of the programming period 2007-2013 since 
the  current  evaluation  covers  the  first  five  years  of  2014-2020.  This  information  is 
collected  from  JASPERS’  annual  reports  for  2007-2013,  and  from  the  previous 
evaluation  of  JASPERS  activities  carried  out  by  DG  REGIO  in  2012.  The  next  sub-
section includes further details on this evaluation. 

Furthermore,  throughout  the  current  evaluation,  in  order  to  estimate  JASPERS’ 
contribution  to  project  preparation  and  the  subsequent  performance  of  the  assisted 
projects  in  the  process  of  approval  for  EU  funding  and  later  in  implementation,  we 
compare systematically the developments for the projects assisted by JASPERS advisory 
with the non-assisted projects. 

As regards the review function, the analysis focuses primarily on the project assessments 
done by JASPERS IQR for the major projects submitted for Art 102.2, and it compares 
the PSA assessments of JASPERS IQR division with the Commission’s assessments for 
the same projects, on the one hand, and also with the assessment provided by JASPERS 
advisory, on the other hand.   

2.4 

Previous evaluations and audits 

There  are  three  previous  assessments  carried  out  either  as  evaluations  or  as  audits  of 
JASPERS’  activities  over  time.  These  assessments  are  as  follows:  the  EIB  evaluation 
published in 2010, the REGIO evaluation published in 2012, and the performance audit 
of the European Court of Auditors published in 2018.  

The  first  JASPERS’  evaluation  carried  out  by  the  EIB  in  2010  covered  the  first  three 
years  (2006-2009)  of  JASPERS’  existence84.  In  this  evaluation,  the  EIB  evaluators 
reached  the  conclusion  that  JASPERS  initiative  was  fully  justified  in  terms  of  the 
demand for its services, and that it generated economic benefits. They also found that, as 
the  initiative  had  been  rapidly  established,  it  also  proved  an  ability  to  adapt  to 

84  See EIB (2010) in References.  

69 

 
 
 
 
 
  
 
 
 
                                                 
circumstances.  Over  time,  JASPERS  acquired  a  good  reputation  with  its  counterparts, 
with the initiative evolving into a partnership in which the Commission, JASPERS, and 
the Member States cooperate on issues of common interest for the implementation of EU 
funds.  

Given this background of high relevance for the initiative, however, the EIB experts also 
identified scope for improvements in JASPERS’ activities and performance. As regards 
the effectiveness of technical advisory, the evaluation concluded that, while the quality of 
some  projects  had  been  improved  by  JASPERS,  for  a  majority  of  the  assignments 
analysed this had not been the case. Similarly, in terms of JASPERS’ support for grant 
applications, the evaluation concluded on limited effectiveness, primarily due to reduced 
coherence between JASPERS’ feedback and the Commission’s assessment.  

For the effectiveness of capacity building activities, the EIB evaluators found that, as the 
initiative was  mainly  focused on projects,  the issue of long  term capacity building  was 
seen more as a beneficial spin off of project level activities rather than a priority in itself.  

Moreover, the EIB evaluation also looked at the efficiency of JASPERS’ operations and 
found that, during the initial  years, JASPERS experienced unclear operational priorities 
and  limited  optimisation  of  the  use  of  resources.  As  a  result,  the  initiative  became 
involved  in  many  low  priority  activities.  In  this  regard,  the  experts  recommended  that 
JASPERS  should  strive  for  a  maximum  possible  leverage  for  its  advice,  targeting  the 
functioning  of  the  system  rather  than  its  products  –  i.e.  “educating  the  trainers,  rather 
than the trainees“. 

Next,  in  2012,  DG  REGIO  carried  out  its  own  evaluation  of  JASPERS’  activities  and 
performance  -  an  evaluation  which  covered  the  previous  period  until  201185.  This 
assessment  focused  on  the  analysis  of  the  use  of  JASPERS’  services  by  sector  and 
Member State, the duration, scale and scope of JASPERS’ assistance, its impact on the 
timeline  of  major  projects,  and  its  impact  on  administrative  capacity  in  the  beneficiary 
countries.  Overall,  as  regards  the  advisory  function,  the  evaluators  concluded  that 
JASPERS proved to be of substantial value to the Member States for the preparation of 
projects  and  applications  for  EU  funding.  They  also  concluded  that  JASPERS  would 
have a greater impact on project quality if it were involved in project preparation at  an 
early stage.  

As  regards  capacity  building  and  horizontal  support,  the  REGIO  evaluation  noted  the 
creation  of  JASPERS’  Networking  Platform  in  2011  as  a  useful  forum  for  the 
dissemination  of  guidance  and  other  outputs  from  the  horizontal  assignments. 
Nevertheless,  while  recognising  that  JASPERS’  work  on  projects  had  the  potential  to 
enable a greater understanding EU requirements and standards in the Member States, the 

85  See DG REGIO (2012) in References. 

70 

 
 
 
 
 
 
 
                                                 
evaluators also found that there was limited potential for advisory assignments at project 
level  to  help  strengthen  the  administrative  capacity  at  country  level,  especially  in 
Member  States  with  few  projects  under  development.  The  evaluation  concluded  that 
horizontal  and  strategic  support  had  higher  potential  for  developing  administrative 
capacity  provided  that  it  was  based  on  a  strategic  approach  identifying  activities  that 
would have a larger impact on project planning and preparation. 

Finally, a third and more recent assessment of JASPERS’ activities and performance was 
provided  by  the  European  Court  of  Auditors  (ECA)  in  201886.  This  performance  audit 
covered  the  first  10  years  (2006-2016)  of  JASPERS’  activities,  and  it  audited  four 
Member States: Croatia, Malta, Poland, and Romania87. As regards the effectiveness of 
JASPERS’ activities, the ECA auditors concluded that JASPERS contributed to a faster 
project  approval  and  had  a  positive  effect  on  the  quality  of  projects  and  their 
documentation.  For  the  implementation  of  projects  assisted  by  JASPERS,  the  ECA 
auditors found that, while JASPERS’ comprehensive assistance could generally not have 
an  impact  on  the  financial  implementation  of  EU  funds,  the  assisted  projects  had  been 
less  frequently  affected  by  legality  and  regularity  errors  during  the  compliance  audit 
exercises carried by ECA during 2014-2015.  

Concerning  JASPERS’  activities  for  capacity  building,  ECA  noted  that  JASPERS  had 
increased its focus on capacity building activities in the more recent  years, but it found 
no evidence that these activities have delivered in terms of strengthened capacity in the 
beneficiary countries.   

Furthermore,  for  the  administrative  set-up  and  approach  of  JASPERS’  activities,  the 
ECA audit also identified scope for improvement for a number of issues. First, the ECA 
auditors  found  that  the  role  and  responsibilities  of  the  main  stakeholders  should  be 
defined  better  by  calibrating  more  precisely  the  working  arrangements  between 
JASPERS  and  the  Member  States  for  all  types  of  services,  including  also  the  capacity 
building  activities  and  the  horizontal  and  strategic  support.  Second,  from  the 
organizational  perspective,  ECA  emphasized  the  need  to  strengthen  the  functional 
independence  of  JASPERS  IQR  relative  to  the  advisory  division  by  establishing  clear 
procedures for the IQR processes that would help enhance the impartiality of the review 
service. Third, as regards JASPERS’ operational planning, ECA auditors concluded that 
JASPERS did not target its assistance sufficiently as it did not define clear priority areas 
for  its  assistance  to  the  Member  States  and  relied  instead  for  the  activities  to  emerge 
based  on  the  demand  from  beneficiary  countries.  Finally,  the  ECA  auditors  also 
concluded  that  there  was  a  need  for  a  better  system  to  monitor  the  efficiency  of 

86  See ECA (2018) in References.  
87  The  assessments  for  Croatia  and  Poland  were  carried  out  in  cooperation  with  the  national  supreme  audit 

institutions.  

71 

 
 
 
 
 
                                                 
JASPERS’  operations.  Overall,  they  identified  scope  for  improvement  as  regards  the 
planning, monitoring and evaluation of JASPERS activities. 

Following  the  ECA  performance  audit  of  JASPERS,  the  Commission  established  an 
action  plan  with  measures  to  be  implemented  starting  with  2018.  These  measures  are 
designed to address the ECA recommendations accepted by the Commission. By January 
2019,  the  measures  related  to  the  clarification  of  role  and  responsibilities  of  the  main 
stakeholders, and the working arrangement within the tripartite cooperation between the 
Commission, JASPERS and the Member States had been implemented fully or were in 
the  course  of  being  finalized.  As  regards  measures  implemented  for  the  planning  and 
performance of JASPERS’ activities, their effects are partially reflected in the evidence 
used for the current evaluation since the overlap between the timing of the evaluation and 
the period of implementation for these measures is very limited. Nevertheless, whenever 
relevant,  throughout  the  document  we  provide  details  also  regarding  the  latest 
developments in this regard.  

In the next chapter we present the main developments with JASPERS’ activities over the 
reference period for the evaluation.  

72 

 
 
 
 
 
 
3.  JASPERS’ PORTFOLIO OF ACTIVITIES DURING THE PERIOD 2014-2018 

For the first five years of the programming period 2014-2020, the European Commission 
invested  close  to  150  million  euro  in  JASPERS,  representing  around  81%  of  the  total 
costs of the initiative. The remaining 19% was covered by the EIB with in-kind support 
for additional technical expertise.  

In  terms  of  projects  approved  for  EU  financing,  JASPERS’  work  contributed  to 
mobilising almost 23 billion euro ERDF and Cohesion Fund resources through 142 major 
projects assisted for preparation and adopted or approved by the Commission88. Further 
2.2  billion  euro  from  the  CEF  cohesion  envelope  were  mobilised  with  23  projects 
assisted by JASPERS for the CEF1.1 mandate, and 82 million euro with 10 IPA projects 
assisted by JASPERS and approved for IPA funding. 

Over  this  reference  period,  JASPERS’  portfolio  included  1096  assignments  for  all 
mandates  and  services.  Figure  3.1  illustrates  the  distribution  of  these  assignments  by 
mandate and main type of assignment. 

Figure 3.1 JASPERS’ portfolio by mandate and assignment type, 2014-2018 

Note: CB = Capacity Building, HSS = Horizontal and Strategic Advice ; data covers 
1096 assignments 
Source : JASPERS’ JADE database, December 2018 

The  largest  mandate  in  JASPERS’  portfolio  is  the  ESIF  mandate,  with  close  to  960 
assignments, followed by the IPA mandate with around 100 assignments. As regards the 
type of services, JASPERS’ traditional function of technical advisory represents  almost 
two  thirds  of  the  portfolio.  Technical  advisory  covers  support  primarily  for  project 
preparation for all the three mandates.  

The  activities  for  capacity  building  and  horizontal  and  strategic  support  in  JASPERS’ 
portfolio  are  mostly  not  project  specific  as  they  focus  on  horizontal  topics  and 

88  The EU resources mobilized by ESIF projects assisted by JASPERS advisory are underestimated as they do not 
include the EU funding for the non-major projects assisted by JASPERS and approved by Managing Authorities. 
We do not have an estimate for the total funding approved for such projects.  

73 

 
 
 
 
 
 
  
       
 
 
 
                                                 
preparation  of  national  strategies  and  plans.  They  are  distributed  primarily  across  the 
ESIF and the IPA mandates. 

 Finally, the review function is specific to the ESIF mandate, and it includes services for 
project  assessment  provided  to  Member  States  (for  IQR  assessments)  and  to  the 
Commission (for PSA assessments).  

In  Figure  3.2  we  include  the  distribution  of  these  assignments  by  sector,  covering  all 
mandates and all services.  

Figure  3.2 :  Distribution  of  JASPERS’  assignments  by  sector,  2014-2018  (number 
assignments) 

Note: CB = Capacity Building, HSS = Horizontal and Strategic Advice ; data covers 
1096 assignments. 
Source : JASPERS’ JADE database, by December 2018 

The distribution by sector indicates that assignments for the transport sectors of Rail, air 
and  maritime  are  the  most  numerous  in  JASPERS’  portfolio,  representing  33%  of  all 
assignments.  The  majority  of  these  assignments  (70%)  are  for  technical  advisory  for 
project  preparation.  The  second  largest  sector  in  terms  of  its  weight  in  the  overall 
portfolio is the sector for Water and wastewater treatment, which also has a substantial 
number  of  assignments  for  technical  advisory,  when  compared  with  the  remaining 
sectors. The multi sector assignments are primarily capacity building activities, including 
workshops and training activities on topics  relevant across sectors (such as cost benefit 
analysis, climate adaptation and mitigation etc.).  

74 

 
 
 
 
 
 
 
Furher,  in  Figure  3.3,  we  illustrate  also  the  distribution  of  JASPERS’  portfolio  across 
countries.  

Figure 3.3: Distribution of JASPERS’ assignments by country, 2014-2018 (number 
assignments) 

Note: CB = Capacity Building, HSS = Horizontal and Strategic Advice; data covers 
1096 assignments 
Source : JASPERS’ JADE database, by December 2018 

Across  countries,  until  end  2018, JASPERS  initiated  assignments  in  21 Member  States 
and 5 IPA countries. The countries with the largest numbers of assignments are Poland, 
Romania,  and  Croatia,  representing  collectively  52%  of  all  assignments  in  JASPERS’ 
portfolio.  Poland  stands out  also  for  the  relatively  large  number  of  assignments  for  the 
review  function  –  assignments  which  correspond  to  major  projects  submitted  to  the 
Commission for approval of EU financing.  

As  regards  the    dynamics  over  time,  76%  of  assignments  were  started  in  2014  and 
thereafter,  while  the  remaining  24%  were  started  prior  to  2014  -  in  preparation  for  the 
programming  period  2014-2020.  By  December  2018,  51%  of  assignments  had  been 
completed, and 9% administratively closed89, while the rest were still ongoing.  

89  Administratively  closed  assignments  are  assignments  which  were  closed  before  the objective  of  the  assignment 
could be achieved. The timeline and reasons for administrative closure of assignments are analysed in details in 
Chapter 5, in the section for efficiency. 

75 

 
 
 
 
 
 
 
 
                                                 
In Figure 3.4 below we present the evolution of the ongoing assignments in JASPERS’ 
portfolio by year between 2014 and 2018.  

Figure 3.4: Ongoing assignments by main type by year, 2014-2018 (number 
assignments) 

Note: CB = Capacity Building, HSS = Horizontal and Strategic Advice ;  data covers 
1096 assignments 
Source : JASPERS’ JADE database, by December 2018 

The  yearly  data  for  ongoing  assignments  indicates  that,  already  by  the  first  year  of  the 
than  300 
programming  period  2014-2020,  JASPERS’  portfolio 
assignments for technical advisory and for capacity building and horizontal and strategic 
support90. 

included  more 

The  assignments  for  the  review  function  started  in  2015,  and  increased  over  time  as 
project preparation was completed and projects were submitted for EU-financing.  

Moreover, while over time some assignments were completed or closed, we see that the 
number of assignments ongoing per year increased steadily until 2017. The year 2018 is 
the first year in the period where the total number of ongoing assignments is lower than 
in  the  previous  year.  By  end  December  2018,  the  number  of  assignments  still  ongoing 
was 435. Figure 3.5 illustrates the number of completed assignments on an annual basis 
over the period 2014-2018.  

90  As explained in text, many of these assignments were started prior to 2014, in preparation for the period.  

76 

 
 
 
 
 
 
 
 
                                                 
On  this  basis,  we  note  that  the  pace  of  assignment  completion  has  accelerated 
significantly  starting  with  year  2017.  For  the  first  two  years,  JASPERS  completed 
assignments for technical advisory and for capacity building and horizontal and strategic 
support,  while  the  completion  of  review  assignments  started  with  2017.  A  detailed 
analysis  of  the  timeline  of  all  assignments  is  included  in  Chapter  5,  in  the  section  for 
efficiency.  

Figure 3.5 Completed assignments by main type by year, 2014-2018 (number 
assignments) 

Note: CB = Capacity Building, HSS = Horizontal and Strategic Advice ; data covers 535 
assignments 
Source : JASPERS’ JADE database, by December 2018 

In  the  following  chapter  we  present  the  evaluation  criteria  on  which  this  evaluation  is 
based, together with the data and methods used. The chapter includes also our assessment 
of the robustness and limitations of the evaluation findings.  

77 

 
 
 
 
 
 
 
 
 
4.  EVALUATION CRITERIA, DATA AND METHODS 

4.1 

Evaluation criteria and issues evaluated 

For this evaluation we assessed JASPERS’ activities and performance during the period 
2014-2018  based  on  the  five  evaluation  criteria  from  the  Commission’s  Better 
Regulation: relevance, effectiveness, efficiency, coherence, and EU added value.   

For the criterion of relevance, we took into account the fact that JASPERS’ activities are 
initiated  at  the  request  of  Member  States,  and  also  the  fact  that  JASPERS  accepted  all 
assignments requested by the Member States for the current period. The aspects analysed 
for  this  criterion  include  the  structure  of  JASPERS  portfolio  emerging  during  the 
reference  period  across  mandates,  and  by  type  of  service,  project  size,  sector,  and 
country, as well as the coverage with JASPERS assistance of projects submitted for EU 
financing. For capacity building and horizontal activities, we analysed the frequency of 
topics  and  issues  for  which  this  type  of  support  was  requested  by  the  beneficiary 
countries. Moreover, for all types of services, we analysed stakeholders’ perceptions as 
regards  the  relevance  of  JASPERS  activities  and  their  experiences  with  the  services 
provided.  

As regards the effectiveness of JASPERS’ services, we analysed the extent to which and 
how the objectives of JASPERS’ services for a faster and better preparation of projects 
were fulfilled. We analysed also the contribution of JASPERS to a more consistent and 
faster process of assessment and approval of ESIF major projects, and its impact on the 
financial implementation.  Subject  to  data availability, similar issues were analysed also 
for all other remaining services of JASPERS. Finally, we analysed also the stakeholders’ 
perceptions as regards the effect of JASPERS’ on a variety of issues relevant for project 
preparation and administrative capacity in the beneficiary countries.  

For  efficiency,  the  evaluation  looked  primarily  at  the  efficiency  of  the  planning  of 
JASPERS’  assignments  and  the  rate  of  administrative  closures  over  the  full  reference 
period. We analysed also the work intensity by type of assignment over a period of two 
years for which data was available. The section on efficiency also includes an analysis of 
the evolution over time of the financial and human resources dedicated to the initiative in 
the current period, relative to 2007-2013.  

The  evaluation  criterion  of  coherence  presents  the  analysis  of  the  extent  to  which 
JASPERS’ activities are aligned with and promote the EU requirements and standards for 
the  objectives  of  EU  Cohesion  Policy,  and  with  higher  level  EU  priorities  for 
environment, climate change adaptation and mitigation, and EU competition policy. For 
this  criterion  we  also  analysed  the  internal  coherence  of  JASPERS’  services  by 
comparing the consistency between the project assessments provided by JASPERS IQR 

78 

 
 
 
 
 
 
 
in  the  context  of  the  review  function,  with  the  assessments  delivered  by  JASPERS 
advisory in the context of technical support for project preparation.  

Finally,  the  criterion  of  EU  added  value  was  assessed  primarily  based  on  the 
stakeholders’ perceptions on the added value of JASPERS’ services relative to other type 
of    schemes  for  technical  advisory  and  capacity  building.  In  this  context,  stakeholders’ 
were also asked for their opinions on the need for JASPERS’ services in the future, and 
on their willingness to pay for these services from own technical assistance.  

4.2 

Data 

For this evaluation we collected and analysed data from a variety of sources as follows: 

16) JASPERS’ JADE dataset of assignments for 2014-2020 

Provided by JASPERS for the evaluation, this dataset includes 1124 assignments, 
of  which  1096  are  for  the  current  programming  period  and  the  remaining 
assignments are for preparation for the next programming period 2021-2027. This 
data includes details on the main characteristics of an assignment, such as type of 
service,  dates  of  creation  and  completion/  closure  (if  applicable),  status  by  end 
December  2018,  sector,  and  country.  For  some  of  the  ESIF  major  projects 
submitted to the Commission, the dataset includes also the project reference code 
in  the  Commission’s  SFC2014  system,  although  not  on  a  systematic  basis.  For 
this reason, we matched JASPERS’ assignments with major projects submitted to 
the Commission based on checks performed using data and documents from the 
Commission’s SFC2014 system.  

17) European Commission SFC2014 database for ESIF major projects91 

SFC2014 is  the system  used by the Commission  and the Member States  for the 
management  of  the  ESIF  funds  in  the  programming  period  2014-2020.  For  the 
evaluation we used the SFC2014 data for all major projects notified or submitted 
to  the  Commission  until  October  201992.    This  dataset  includes  320  major 
projects,  and  it  was  used  as  a  basis  for  matching  the  major  projects  with 
JASPERS’ assignments in JADE. The SFC2014 data informs us on the financing 
proposed or approved for the major projects submitted, the timetables of projects, 
as well as a series of other project details included in the project application for 
EU financing. Importantly for the evaluation, the SFC2014 system stores all the 
documentation submitted by the Member States for these projects, including also 
the  PSA  reports  and  the  action  completion  notes  (ACNs)  issued  by  JASPERS’ 

91  Throughout the evaluation we make a distinction between JASPERS’ assignments for project preparation and the 
projects themselves. This is due to the fact that it is possible that one assignment covers more than one projects, or 
that a project can be covered by more than one assignments. We explain in text whenever this is the case.  

92  This  was  the  latest  possible  date  when  we  could  update  the  data  based  on  the  SFC2014  system  before 

consolidating the analysis for the evaluation.  

79 

 
 
 
 
 
 
                                                 
advisory for assisted projects submitted for Art. 102.2, and the IQR reports issued 
by JASPERS IQR for projects assessed for the procedure based on Art. 102.1.  

18) Assessments and approval of ESIF major projects  

For  the  evaluation  of  JASPERS’  review  function,  from  the  Commission’s 
SFC2014  system  we  collected  structured  data  from  all  the  available  (69)  PSA 
reports issued by JASPERS IQR for the Commission and all (69) Commission’s 
observation letters for all projects submitted for Art. 102.2 and interrupted during 
the approval process. We also screened all (55) IQR reports issued by JASPERS 
for major projects notified for Art. 102.1. The PSA and IQR reports are also the 
main basis on which we matched the projects submitted to the Commission and 
JASPERS’ corresponding advisory and review assignments in JADE.  

19) Implementation of ESIF major projects  

From the Commission’s SFC2014 system we collected data on the financial and 
physical  implementation  from  the  2019/2018  annual  implementation  reports  of 
166  major  projects  approved  over  the  reference  period.  This  dataset  includes 
projects assisted and not assisted by JASPERS advisory.  

20) JASPERS’ support for ESIF non-major projects 

For  the  ESIF  non-major  projects  assisted  by  JASPERS  advisory,  we  collected 
structured data from all (49) available action completion notes, project fiches, and 
value added fiches provided by JASPERS for the sampled countries. 

21) CEF projects approved in the current period 

For the analysis of JASPERS’ CEF mandate, we used a dataset with all approved 
CEF  projects  until  August  2019  in  order  to  match  JASPERS’  assignments  with 
these projects, and compare them with non-assisted projects. Overall, this dataset 
includes  265  CEF  projects.  In  addition,  DG  MOVE  provided  for  the  evaluation 
also the final report of the implementation of JASPERS’ CEF1.1 pilot mandate.  

22) IPA projects approved in the current period 

For  the  analysis  of  JASPERS’  IPA  mandate  we  matched  the  IPA  assignments 
with the IPA projects approved over the reference period. The dataset of all IPA 
projects  approved  was  provided  by  DG  NEAR  and  it  includes  253  projects 
assisted and not assisted by JASPERS.  

23) Planning of assignments  

For  the  analysis  of  the  planning  of  assignments  for  JASPERS’  portfolio  we 
collected data on the evolution of these assignments over time from 33 quarterly 
country  action  plans  for  three  countries  sampled  for  this  evaluation  (Croatia, 
Poland, Romania). The country plans were provided by JASPERS. 

80 

 
 
 
 
 
 
 
 
24) Actual work hours per assignment 

For the analysis of efficiency of JASPERS’ services, we analysed a dataset of 190 
assignments  created  and  completed  during  the  period  2017-2019  and  for  which 
JASPERS  had  started  recording  the  actual  working  time  per  assignment.  The 
dataset was provided by JASPERS. 

25) JASPERS’ human and financial resources  

For the analysis of human and financial resources in the section on efficiency, we 
collected structured data on JASPERS’ annual budgets and staff over the period 
2007-2018 from JASPERS’ annual reports published online.  

26) JASPERS’ completed assignments for 2007-2013 

Data  on  assignments  completed  by  JASPERS  for  the  period  2007-2013  was 
collected  from  JASPERS’  online  annual  reports  2010-2018.  On  this  basis,  we 
could identify the assignments by type of service, sector, and country.  

27) Performance of JASPERS activities 2007-2013 

For the comparison of evaluation findings for the current period with equivalent 
findings  from  the  period  2007-2013,  we  collected  comparable  data  from  the 
previous evaluation of JASPERS carried out by DG REGIO in 2012.  

28) Major projects approved by the Commission for 2007-2013 (SFC2007) 

SFC2007 is  the system  used by the Commission  and the Member States  for the 
management  of  ESIF  funds  during  the  programming  period  2007-2013.  From 
SFC2007  we  used  the  data  on  major  projects  approved  by  the  Commission  for 
ERDF  and  Cohesion  Fund  for  the  period  2007-2013.  This  dataset  includes  767 
major  projects,  with  data  on  the  EU  and  total  funding  for  the  projects,  sector, 
country,  fund,  JASPERS  assistance,  and  use  of  framework  contracts  for  project 
appraisals.  

29) JASPERS’ Network Platform website 

From JASPERS’ Networking Platform  website  we collected data on all  training 
and  workshop  activities  included  in  the  annual  programmes  starting  with  2013 
and  until  2018.  In  addition,  we  also  collected  the  data  on  all  publications 
published by JASPERS’ in the online library of the platform. 

30) Stakeholder’s perceptions of JASPERS’activities 

For  this  evaluation,  we  conducted  an  online  targeted  consultation  with 
stakeholders with direct experience with JASPERS’ services, including Managing 
Authorities,  projects  promoters/  beneficiaries,  professionals  from  consultancy 

81 

 
 
 
 
 
 
 
 
companies,  staff  of  the  European  Commission,  etc.  Overall,  210  stakeholders 
responded to this consultation93.  

Furthermore, based on the documents from an external contract, we also explored more 
in-depth  the  implementation  of  JASPERS’  services  based  on  87  assignments  sampled 
from  JASPERS’  portfolio94,  and  on  stakeholders’  interviews  from  6  countries  sampled 
for the evaluation (Croatia, Italy, North Macedonia, Poland, Romania, and Slovakia). We 
used these additional elements as background information for this evaluation report.  

4.3  Methods 

As  regards  the  methods  of  analysis,  we  used  extensively  statistical  analysis  of  the  data 
collected  for  the  evaluation,  with  an  emphasis  on  median  values  and  distributional 
patterns. Whenever necessary for the comparison of projects assisted by JASPERS with 
projects not assisted, we tested the statistical significance of differences in median values 
for assisted and non-assisted projects by using the Wilcoxon rank-sum test (also known 
as  the  Mann–Whitney  two-sample  statistic).  We  used  this  method  to  understand,  for 
instance,  whether  the  median  number  of  observations  in  Commission’s  interruption 
letters  for  projects  assisted  by  JASPERS  is  lower  than  the  median  number  of 
observations in interruption letters for non-assisted projects.  

In  addition,  for  analyses  which  required  separating  JASPERS’  contribution  from  other 
co-founding factors for a given result, we used regression analysis. This is the case, for 
example,  in  the  analysis  of  the  correlation  between  JASPERS’  assistance  and  the 
duration of preparation of projects. As regards the regression models used, depending on 
the type of the dependent variable, we used three types of models as follows95: 

 

linear regression analysis with robust standard errors when the dependent variable 
is continuous 
logistic regression models when the dependent variable is binary (1/0) 

 
  ordered  logistic  regression  models  when  the  dependent  variable  is  discrete  with 

ordered values (ordinal).  

4.4 

Robustness and limitations of findings 

In our assessment, a very large majority of the findings in this evaluation are robust as 
they are based on evidence collected mostly at the level of the entire relevant population. 

93  Summaries of the replies to the consultation are included in Annex 1.  The report with full details of the results 

from the targeted consultation is published on the website of this evaluation.  

94  Among  the  sampled  assignments,  77%  are  for  the  ESIF  mandate,  and  11%  the  CEF  and  IPA  mandate  each, 
covering  all  JASPERS’  services.  JASPERS  provided  all  available  documentation  for  the  sampled  assignments 
including,  for  instance,  project  fiches,  ACNs,  guidance  notes,  value  added  fiches,  and  feedback  data  from 
beneficiaries. 

95  For further details on these models, see Long and Freese (2013) in References. 

82 

 
 
 
 
 
 
 
                                                 
In Annex 2, Tables 2.1.1-2.1.3 we summarize the data coverage by evaluation criterion 
and  topic  for  evaluation,  thus  identifying  the  extent  to  which  each  topic  analysed  is 
covered by representative data.   

There  are,  however,  exceptions  for  topics  for  which  the  possibility  to  collect  the 
necessary  data  was  much  more  limited.  First,  this  is  the  case  of  the  analysis  of 
effectiveness of JASPERS’ advisory for non-major projects which remains partial due to 
lack of follow up data on what happens with these projects once JASPERS’ advisory is 
completed. JASPERS does not collect this type of data, and to our data requests made to 
the six Member States sampled for this evaluation, only two of them responded. 

Another  important  topic  which  is  only  partially  covered  by  the  evaluation  refers  to  the 
efficiency  of  the  time  allocation  for  JASPERS’  activities.  For  this  topic,  we  analysed 
only  the  assignments  completed  during  a  period  of  two  years  (2017-2019).  In  our 
estimate, however, a more complete dataset for annual actual work per assignment over 
the entire period would have been needed for a full estimation of the remaining scope for 
a more optimal allocation of resources. More complete data, however, was not available 
for the full reference period of the current evaluation as JASPERS started collecting this 
data in 2017, and only at assignment level.  

Finally,  a  third  topic  less  covered  by  this  evaluation  refers  to  the  IQR  review  service. 
While  for  the  PSA  review  service  we  could  conduct  an  extensive  analysis  for 
effectiveness  and internal  coherence as we collected the necessary structured data from 
the Commission SFC2014 system and from JASPERS, for the IQR process the scope for 
analysis  was  much  more  limited.  We  learnt  that,  during  the  reference  period  of  the 
evaluation,  the  IQR  process  was  less  formal  than  the  PSA  assessment  for  the 
Commission,  with  much  of  the  information  related  to  the  assessment  of  projects 
conveyed  by  email  in  a  non-structured  form96.  As  a  result,  we  could  not  carry  out  a 
systematic  analysis  of  the  effectiveness  and  coherence  of  this  service  relative  to  the 
advisory services comparable with the one for the PSA.  

The remaining of this document includes the presentation of the evaluation findings and 
the supporting evidence, followed by the chapter of conclusions.  

Finally, the author would like to thank JASPERS’ staff for their cooperation in sharing 
data  for  this  evaluation  and  for  their  availability  for  clarifications  whenever  needed. 
Furthermore,  we  are  also  grateful  to  our  Commission  colleagues  participating  in  the 
Steering Group of the evaluation for their continuing cooperation and support.  

96  At  a  later  stage  in  the  evaluation  process,  we  learnt  that  JASPERS  formalized  also  the  IQR  process  starting 

primarily with year 2019, which is outside the main reference period for this evaluation. 

83 

 
 
 
 
 
 
 
 
 
                                                 
5.  EVALUATION FINDINGS AND SUPPORTING EVIDENCE 

In this chapter we present the main results of the evaluation of JASPERS initiative by the 
five evaluation criteria of the Commission’s Better Regulation : relevance, effectiveness, 
efficiency,  coherence,  and  EU  added  value.  For  presentation  purposes,  we  further 
organised the finding by JASPERS’ mandate and type of function (advisory, review, and 
horizontal  functions)  to  the  extent  it  proved  necessary  to  distinguish  along  these 
dimensions.  This  applies  in  particular  for  the  criteria  of  relevance,  effectiveness  and 
coherence, but less so for efficiency and EU added value where the issues analysed are of 
a more horizontal nature and therefore apply to all mandates.  

As  regards  the  presentation  of  the  supporting  evidence,  the  main  text  includes  the 
evidence  that  could  be  presented  in  a  more  concise  manner  for  an  integrated  narrative, 
while  the  annexes  include  further  details  and  disaggregation  of  the  data  supporting  the 
conclusions.  

5.1 

RELEVANCE 

For  the  evaluation  criterion  of  relevance  we  analysed  the  structure  of  JASPERS’ 
portfolio  over  the  reference  period  2014-201897,  often  in  comparison  with  its  activities 
during  the  previous  programming  period  2007-2013.  Furthermore,  the  relevance  of 
activities  during  the  current  period  is  also  analysed  in  the  broader  context  of  the 
Commission’s  decision  making  process  for  the  approval  of  EU  financing  for  projects 
prepared with the assistance of JASPERS advisory or reviewed by JASPERS IQR.  

Given that all JASPERS’ activities are initiated at the request of the Member States, the 
main  evaluation  approach  applied  was  to  assess  the  structure  of  JASPERS’  portfolio 
which  emerged  so  far.  The  main  issues  analysed  here  include  the  distribution  of 
assignments requested by  Member States by sector and by type of assignment for each 
mandate,  the  concentration  of  support  across  Member  States,  the  type  and  timing  of 
support provided by JASPERS in the context of project preparation, the types of project 
supported,  as  well  as  the  Member  States’  choices  between  the  two  procedures  for  the 
approval of major projects for the ESIF mandate. The section includes also an analysis of 
the distribution of the activities for capacity building and horizontal and strategic support. 

This  section  is  first  organised  by  JASPERS’  mandate  and  it  concludes  with  the 
stakeholder  online  consultation  on  relevance  of  JASPERS’  activities  for  all  mandates. 
The main findings are summarized in the Text box 5.1.1 below.  

97  As mentioned in the first chapter, the cutoff date for the reference period for the evaluation of JASPERS’ portfolio 

of assignments is 31 December 2018.   

84 

 
 
 
 
 
 
 
 
 
                                                 
Text Box 5.1.1 Main findings for the relevance of JASPERS’ activities 2014-2018 

OVERALL 

1)  In  the  current  period  JASPERS  remained  relevant  for  the  preparation  of  projects, 
with  its  activities  mandated  by  three  Commission  services  and  providing  more 
services  for  a  larger  number  of  countries  compared  to  the  previous  period.  In 
addition, JASPERS has become relevant also for the new function of quality review of 
ESIF major projects submitted for EU funding. 

ESIF MANDATE 

2)  Overall  mandate:  Relative  to  the  previous  period,  JASPERS’  ESIF  portfolio  has 
diversified  and  expanded.  With  around  960  assignments  for  the  first  5  years  of  the 
current  period, the ESIF portfolio now includes close to 60% assignments for project 
preparation  (compared  with  82%  previously),  23%  for  capacity  building  and 
horizontal  advice  (compared  with  18%  previously),  and  the  remaining  for  the  new 
review function. 

3)  Overall  mandate:  Across  sectors  and  relative  to  the  previous  period,  the  main 
changes  in  the  portfolio  structure  include  an  increased  share  for  assignments  in 
transport (higher by 16%), and a reduced share for water and wastewater treatment 
assignments (lower by 11%). The share of assignments for transport is now highest in 
the ESIF portfolio (50%). 

4)  Overall mandate: A majority of JASPERS’s activities (57%) remain concentrated in 
three  main  beneficiary  countries  (Romania,  Croatia  and  Poland),  compared  with 
59%  of  assignments  concentrated  in  three  beneficiary  countries  (Romania,  Poland 
and Bulgaria) in the previous period.  

5)  Major projects: 58% of projects submitted to the Commission for adoption/ approval 
were assisted by JASPERS advisory for project preparation, compared with 50% in 
2007-2013.  Most  countries  maintained  high  to  very  high  rates  of  JASPERS’ 
assistance for their projects submitted, while most of the non-assisted projects were 
submitted by Poland (76%).  

6)  Major  projects:  Compared  to  non-assisted  projects,  the  ones  assisted  by  JASPERS 
advisory  tend  to  be  larger  in  size.  This  pattern  is  maintained  from  the  previous 
period.  

7)  Major  projects:  Most  frequent  topics  for  JASPERS’  technical  advice  for  the 
preparation  of  new  projects  include  the  feasibility  study  and  cost  and  benefit 
analysis.  

8)  Major  projects:  For  42%  of  new  major  projects  assisted  and  submitted,  JASPERS 
technical  assistance  was  requested  from  the  start  of  the  project  preparation.  For 
additional 50% of the projects, JASPERS assistance started during the stage of core 
preparation.  

85 

 
 
 
 
 
 
 
 
 
 
 
 
9)  Non-major projects: The share of  assignments for non-major projects  in JASPERS’ 
portfolio  for  technical  advisory  has  increased  from  21%  to  25%  between  the  two 
periods.  

10)  Non-major projects: Across sectors, main changes in the portfolio structure include 
an increased share for assignments in transport (higher by 7%), and a reduced share 
for  water  and  wastewater  treatment  assignments  (lower  by  4%).  The  share  of 
assignments  for  water  and  wastewater  treatment,  however,  remains  highest  in  the 
portfolio (34%).  

11)  Non-major projects: A majority (74%) of these assignments is concentrated in three 
main  beneficiary  countries  (Croatia,  Romania,  Bulgaria).  In  the  previous  period, 
63%  of  such  assignments  were  concentrated  in  three  main  beneficiary  countries 
(Romania, Poland, Bulgaria).  

12)  Non-major  projects:  In  almost  70%  of  assignments,  JASPERS  analyzed  the  non-
major projects in a similar manner as for major projects since beneficiaries prepared 
feasibility  studies  and  cost  benefit  analyses  for  these  projects  even  in  absence  of  a 
regulatory obligation.  

13)  Non-major  projects:  Most  frequent  topics  for  JASPERS  technical  advice  for  the 
preparation of projects include the feasibility study and cost and benefit analysis.  

14)  Review function: JASPERS’ new review function is highly relevant as it assesses all 
new  major  projects  submitted  under  the  legislative  framework  for  the  period  2014-
2020. Of the two procedures (Art 102.1 and Art 102.2), more than 70% of new major 
projects  were  assessed  with  the  support  of  the  PSA  service  provided  by  JASPERS 
IQR to the Commission for the procedure based on Art. 102.2.  

15)  Review function: While there is significant variability across projects and countries, 
when  compared  with  the  procedure  based  on  Art.  102.1,  the  PSA  based  procedure 
(Art  102.2)  is  more  likely  to  be  chosen  by  Member  States  for  larger  projects,  for 
projects submitted before the full  year with the Omnibus Regulation adopted (2019), 
and for projects not assisted by JASPERS advisory. 

16)  Capacity  building  and  HSS:  Across  all  support  activities  for  technical  capacity, 
while the share of multi-country trainings and workshops on topics of high relevance 
for  all  countries  has  increased,  the  horizontal  and  strategic  support  tailored  to 
country specific needs remains prevalent. 

17)  Capacity  building  and  HSS:  A  majority  (57%)  of  activities  for  capacity  building 
included in JASPERS’ portfolio in the current period are multi-country assignments. 
The  most  frequent  topics  for  these  training  activities  include  climate  change, 
environment, state aid, and cost benefit analysis.   

18)  Capacity  building  and  HSS:  The  demand  for  country  specific  capacity  building 

activities was not high during the reference period. 

86 

 
 
 
 
 
 
 
 
 
 
 
 
19)  Capacity  building  and  HSS:  Two  thirds  of  country  specific  assignments  for 
horizontal  and  specific  support  are  of  horizontal  nature.  One  third  supported 
national strategies and plans.  

20)  Capacity  building  and  HSS:  Close  to  60%  of  country  specific  horizontal  and 
strategic  support  is  concentrated  in  3  Member  States  (Romania,  Poland  and 
Croatia). 

21)  Capacity building and HSS: More than 60% of activities for capacity building and 
horizontal  and  strategic  advice  were  started  at  the  beginning  of  (or  before)  the 
programming period 2014-2020.  

CEF1.1 MANDATE 

22) CEF1.1: For the first pilot, JASPERS advisory assisted 29 projects, of which 23 were 

approved for CEF financing for 6 cohesion countries. 

23)  CEF1.1: Compared with all projects approved for CEF financing, the CEF projects 
assisted  by  JASPERS  advisory  for  the  first  pilot  are  more  concentrated  on  works 
(rather than studies), and they tend to be much larger in terms of eligible costs. 

24)  CEF1.1: In a sample of 9 assignments, frequent topics for advisory include project 
technical specification, cost benefit analysis, environmental issues, procurement and 
implementation, and grant application. 

IPA MANDATE 

25)  IPA:  Relative  to  the  previous  period,  JASPERS’  IPA  mandate  has  expanded  and 

diversified in 2014-2020. 

26)  IPA:  Almost  three  quarters  of  assignments  in  JASPERS’  portfolio  are  for  advisory 
for  project  preparation,  with  the  largest  share  (42%)  for  the  sector  of  Water  and 
wastewater treatment. 

27)  IPA: More than half of the advisory assignments are concentrated in two beneficiary 

IPA countries (North Macedonia and Serbia). 

28)  IPA:  Overall,  projects  assisted  by  JASPERS  advisory  and  approved  for  financing 
represent 7% of all IPA projects approved for financing until 2019. In most of these 
assignments,  JASPERS  helps  with  the  guidance  and  supervision  of  contracts  for 
technical assistance.  

ONLINE CONSULTATION 

29)  Online consultation: Almost all of the 210 respondents consider JASPERS relevant 
for  support  to  increase  project  quality,  and  for  assisting  the  alignment  of  projects 
with EU environmental directives. Further, a large majority perceive JASPERS to be 

87 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
relevant for support with the development of skills for cost benefit analysis. 

30)  Online  consultation:  The  most  frequent  reasons  for  requesting  JASPERS’  services 

are perceptions of increased chances for faster project approval.  

31)  Online  consultation:  A  large  majority  of  stakeholders  think  that  JASPERS’  has  a 
high  ability  to  respond  to  needs  for  improving  project  quality,  for  ensuring 
compliance  with  EU  environmental  regulations,  and  for  assisting  with  the 
development of skills for cost benefit analysis.  

32)  Online  consultation:  Overall,  a  very  large  majority  of  stakeholders  rated  highly 
JASPERS’  services  in  terms  of  coherence  with  EU  standards,  quality  of  advice, 
effectiveness and timeliness. 

For each of these evaluation findings the supporting evidence is as follows.  

Overall portfolio 

In  the  current  period  JASPERS  remained  relevant  for  project  preparation,  with  its 
activities  mandated  by  three  Commission  services  and  providing  more  services  for  a 
larger  number  of  countries  compared  to  the  previous  period.  In  addition,  JASPERS 
has become relevant also for the new function of quality review of ESIF major projects 
submitted for EU funding. 

JASPERS’s  technical  support  continued  to  be  highly  relevant  also  in  the  current 
programming period 2014-2020. As explained in Chapter 2, JASPERS assistance is now 
available  for  all  Member  States  and  IPA  countries,  and  it  is  mandated  by  three 
Commission  services  (DG  REGIO,  DG  MOVE,  and  DG  NEAR).  Its  services  include 
technical  advisory  for  the  preparation  of  major  and  non-major  projects,  activities  for 
capacity  building  and  horizontal  and  strategic  support,  and  a  new  review  function  for 
projects  supported  by  Cohesion  Policy.  For  the  first  5  years  of  the  current  period, 
JASPERS registered around 1100 assignments in 26 countries.  

By  comparison,  in  the  previous  period  2007-2013,  JASPERS  provided  technical 
assistance  for  14  Member  States  and  3  IPA  countries  through  almost  800  completed 
assignments.  Its  services  included  technical  advisory  for  the  preparation  of  major  and 
non-major projects, activities for capacity building, and horizontal and strategic support 
for projects supported by Cohesion Policy (see Table A3.2 in Annex 3 for details).  

In what follows, we look into more detail at the distribution of JASPERS’ activities by 
mandate. 

88 

 
 
 
 
 
 
 
 
 
 
 
 
 
ESIF Mandate 

Relative  to  the  previous  period,  JASPERS’  portfolio  for  the  ESIF  mandate  has 
diversified and expanded. As regards sectors, the portfolio now also includes a higher 
share  of  assignments  for  transport,  and  a  lower  share  for  water  and  wastewater 
treatment.  

JASPERS’  portfolio  for  the  ESIF  mandate  in  current  period  until  December  2018 
includes close to  960 assignments, of which 59% are for direct  technical  assistance for 
project  preparation  (and  to  a  much  lesser  extent  implementation),  23%  for  horizontal 
activities and capacity building, and the remaining 18% for its new function of IQR and 
post-submission assessments of major projects. Table 5.1.2 presents the details by sector 
and assignment type.  

Table 5.1.2 JASPERS portfolio for the ESIF mandate 2014-2018, by sector and type 

Note: TA = Technical Assistance. 
Source: JASPERS’ JADE database, December 2018 

Half of these assignments were created for projects and issues relevant for the transport 
sector (Rail, air, maritime, and roads), and 20% for the sector of Water and wastewater 
treatment.  

When compared with the portfolio of assignments completed for th period 2007-2013 by 
sector, we note primarily an overall increase in the share of assignments for transport by 
16%, primarily due to a significant increase in the share of assignments for Rail, air and 
maritime  (from  19%  to  32%)  between  the  two  periods.  As  regards  the  share  of 
assignments  for  Water  and  wastewater,  it  has  diminished  by  11%  (from  31%  o  20%) 
between the two periods. The distribution of JASPERS’ activities by sector and type of 
assignment for the previous period is presented in Table A3.1 in Annex 3.  

A majority of JASPERS’ activities for the ESIF mandate remain concentrated in three 
main beneficiary countries. 

When  looking  at  the  distribution  of  the  ESIF  assignments  at  country  level  we  see  that 
57% of these assignments were carried out in three main beneficiary countries (Romania, 

89 

Sector / Type assignmentTA major projectsTA non-major projectsImplementation projectsCapacity buildingHorizontal and strategicReviewIQRAssessment  PSATotal% in totalEnergy and solid waste4018433131111011%Rail, air, maritime159323945253330632%Roads75130115125316918%Smart development and ICT4530243151112813%Water and wastewater9547131111119620%Multi sector3711485%Other220.2%Total414140105416656119959100%% in total43%15%1%6%17%6%12%100% 
 
 
 
 
 
 
 
 
 
Croatia,  and  Poland),  5%  are  multi  country  assignments,  and  the  rest  are  distributed 
across 18 more Member States (see Table A4.1 in Annex 4 for details).  

By comparison, during the previous period, 59% of JASPERS’ activities for this mandate 
were also concentrated in three main beneficiary countries (Poland, Romania, Bulgaria), 
1% were multi-country assignments, and the rest of activities were distributed across 11 
more Member States (see Table A3.2 in Annex 3 for details).  

Technical advisory for major projects 

A majority (58%) of major projects submitted to the Commission have been supported 
by JASPERS advisory for project preparation.  Most countries maintained high to very 
high rates of JASPERS’ assistance for their projects submitted, while 76% of the non-
assisted projects were submitted by Poland.  

As regards the assignments for the preparation of major projects, in order to understand 
what  happened  with  the  projects  assisted  by  JASPERS  advisory  once  the  technical 
assistance  is  completed,  we  looked  at  the  data  in  Commission’s  SFC2014  system  for 
major projects  submitted and approved until  October 2019 (that  is,  10 months after the 
reference end date for JASPERS’ portfolio).  

In  SFC2014  there  are  three  types  of  projects:  1)  new  projects  submitted  for  approval 
based on CPR Art 102.1 (subject to IQR), 2) new projects submitted for Commission’s 
assessment  and  decision  based  on  CPR  Art  102.2  (subject  to  PSA),  and  3)  projects 
phased  from  the  previous  programming  period  and  submitted  based  on  CPR  Art  103. 
Overall, until October 2019, 184 major projects submitted to the Commission had been 
assisted  by  JASPERS  advisory  for  project  preparation,  implying  a  rate  of  coverage  of 
submitted projects by JASPERS technical advisory of 58%. Table 5.1.3 summarizes the 
number  of  projects  submitted  for  the  3  types  of  procedures  by  sector  and  JASPERS 
advisory. 

Table  5.1.3  Major  projects  submitted  to  the  Commission  until  October  201998,99 

Note : TA = Technical Advisory. 
Source: European Commission SFC2014, October 2019 

98  As matching the phased projects to advisory assignments in JASPERS’ portfolio proved more difficult, the rate of 

JASPERS assistance for these projects may be slightly underestimated.  

99  This data includes only two projects assisted by JASPERS advisory for implementation.   

90 

All projectsJASPERS TA assistedAll projectsJASPERS TA assistedAll projectsJASPERS TA assistedEnergy and solid waste367%1385%40%2065%Rail, air, maritime2767%4085%4326%11057%Roads11100%6532%130%8936%Smart development and ICT786%1867%1258%3768%Water and wastewater1292%14100%3868%6480%Total6080%15061%11040%32058%Sector \ Procedure % JASPERS TA assisted in totalArt. 102_1Art. 102_2Art. 103Total 
 
 
 
 
 
 
 
                                                 
Data in Table 5.1.3 indicates, for example, that 110 of the major projects submitted were 
for  Rail,  air  and  maritime,  with  27  of  them  subject  to  IQR  and  notified  to  the 
Commission  for  approval.  Furthermore,  67%  of  the  27  projects  subject  to  IQR  in  this 
sector  had  been  assisted  by  JASPERS  advisory  for  project  preparation.  The  same 
interpretation applies to all sectors and types of projects presented in the table.  

These data indicate that a majority of major projects submitted to the Commission were 
assisted  by  JASPERS  advisory  in  all  sectors,  except  for  Roads  where  the  rate  of 
assistance is lower (36%). The highest rates of assistance apply to Water and wastewater 
(80%) and Smart development and ICT (68%).  

When compared with the data for projects approved in the previous programming period, 
while  the  overall  rate  of  Jaspers  assistance  for  major  projects  increased  (from  50%  for 
2007-2013 to  58% currently), the relative weight of sectors has also  changed.  In 2007-
2013, the higher rate of coverage by JASPERS assistance was observed for Energy and 
solid  waste  (72%),  followed  by  Water  and  wastewater  treatment  (63%).  For  Smart 
development and ICT, on the other hand, the proportion of assisted projects was 35% in 
2007-2013,  therefore  increasing  significantly  to  68%  in  the  current  period.  The 
distribution of assignments by sector for 2007-2013 is presented in Table A3.4 in Annex 
3. 

In what follows, among all major projects submitted to the Commission for approval for 
the current programming period, for this evaluation we focus primarily on the new major 
projects  submitted  for  Art.  102.1  and  102.2.  This  is  because  the  phased  projects 
submitted for Art.103 had been prepared during the previous programming period 2007-
2013100,  and  therefore  are  less  comparable  with  the  new  major  projects  as  regards 
JASPERS’ services for advisory and project appraisal.  

For  the  two  types  of  procedures  for  approval/  adoption  of  new  major  projects  in  the 
current  programming period we observe  a higher prevalence of assisted  projects  (80%) 
among  the  new  projects  submitted  to  IQR,  relative  to  the  direct  assessment  by  the 
Commission (61%). In absolute numbers, however, the number of projects submitted for 
the PSA based procedure, and which had been assisted by JASPERS advisory, is much 
higher. 

At country level, for the new projects, we find that 14 countries out of 18 used JASPERS 
technical  assistance  for  the  preparation  of  at  least  75%  of  their  new  major  projects 
submitted for funding. Table A4.3 in Annex 4 presents the distribution of these projects 

100  For  phased  projects,  the  project  documentation  had  already  been  prepared  and  approved  during  the  previous 
programming  period  and  the  application  for  funding  in  the  current  programming  needs  to  provide  sufficient 
assurance  that  the  phased  project  has  not  changed  significantly  and  it  still  complies  with  the  regulatory 
requirements.  

91 

 
 
 
 
 
 
 
 
                                                 
by  country.  By  comparison,  for  the  previous  period,  latest  data  available  in  SFC2007 
indicates  that  50%  of  almost  770  such  projects  were  assisted  by  JASPERS,  with  the 
coverage of assistance ranging from 5% and 100% at country level. This means that, In 
10 out of the 14 countries assisted, JASPERS advisory contributed to the preparation of 
all  or  almost  all  their  major  projects  approved  by  the  Commission  (see  Table  A3.3  in 
Annex 3 for the details for 2007-2013). 

Further  for  the  comparison  of  the  two  periods  in  relative  terms,  we  can  distinguish  the 
following four situations101. First, there are countries which increased the use JASPERS 
technical  assistance for their new major projects  submitted in  the current  period. These 
include  countries  with  previous  experience  with  JASPERS  services  such  as  Bulgaria, 
Italy, Romania and Slovenia. Second, some other Member States are new to JASPERS 
support in the current period. These include Austria, France, Spain and Portugal. Third, 
four  Member  States  (Hungary,  Latvia,  Lithuania  and  Slovakia)  maintained  full  support 
(100%) of JASPERS advisory for all the projects submitted to the Commission in both 
periods. And fourth, two Member States - Czechia and Poland - used JASPERS advisory 
to a lesser extent, with a reduction of more than 10% in the number of projects assisted in 
all projects submitted at least until October 2019.  

In  absolute  terms,  however,  we  find  that  a  large  majority  of  non-assisted  new  major 
projects submitted to the Commission belong to Poland. From a total of 210 new major 
projects  submitted  by  all  countries,  53  out  of  70  projects  not  assisted  by  JASPERS 
advisory  came  from  Poland  (with  43  in  Roads,  and  10  in  Rail,  air,  maritime).  In  the 
previous period the share of Poland in all non-assisted major projects was only 22%, and 
therefore  it  increased  significantly  during  the  first  5  years  of  the  current  period. 
Moreover,  as  Poland  has  reduced  its  overall  use  of  JASPERS  advisory  for  new  major 
projects  submitted to  the Commission  by 12% relative to  the previous  period, this data 
also indicates that the country has increased reliance on own resources for the preparation 
of projects at least in some sectors. This applies especially for Roads102, where 84% of 
new projects submitted did not require JASPERS advisory in the current period and, to a 
lesser  extent,  it  also  applies  to  Rail,  air  and  maritime,  with  30%  of  new  projects 
submitted not assisted by JASPERS advisory.   

Major projects assisted by JASPERS tend to be larger in terms of median total costs.  

101  Please note that the data for 2007-2013 covers the full period and therefore is not perfectly comparable with the 
first  5  years  of  2014-2020.  Nevertheless,  we  consider  that  this  comparison  can  still  be  relevant  for  emerging 
trends.  
In Poland, the majority of the non-assisted major projects for Roads are prepared according to the “Blue Book” 
developed with the support of JASPERS advisory during 2007-2013. This illustrates the potential for JASPERS’ 
technical advisory to have a long lasting impact on technical capacity at country level.  

102 

92 

 
 
 
 
 
 
                                                 
As  regards  the  size  of  projects,  we  looked  at  all  the  178  new  major  projects  adopted/ 
approved  by  the  Commission  during  the  reference  period  and  calculated  the  median 
values for total cost and EU funding by sector. Table 5.1.3 presents the details.  

93 

 
 
 
 
 
Table  5.1.3  Median  values  of  major  project  financing  by  sector,  source  and  JASPERS 
advisory  

Note: Data refers to new major projects (Art 102.1 and 102.2) approved/ adopted by EC. 
It covers 178 projects, of which 120 assisted by JASPERS advisory. 
Source: European Commission SFC2014, October 2019 

When  comparing  assisted  and  non-assisted  projects,  we  find  that  the  major  projects 
assisted by JASPERS advisory tend to be larger in terms of total costs in all sectors, with 
differences  significantly  higher  for  the  sectors  of  Roads  (329  million  euro  for  assisted 
projects vs 255 million euro for non-assisted projects) and Energy and solid waste (209 
million  euro  vs  129  million  euro)103,  EU  funding  maintains  the  same  pattern,  with  the 
exception of ERDF and EIB loans for Rail, air and maritime for which the median value 
of EU grants/ loans is higher for non-assisted projects. 

Relative to period 2007-2013 we find that, while the total funding for all major projects 
tends to be higher in the current period, the same pattern of larger major projects assisted 
by JASPERS advisory can be observed in both periods. The median total costs and EU 
contribution by sector and JASPERS’ support for 2007-2013 are presented in Table A3.5 
in Annex 3. 

Most  frequent  topics  for  JASPERS’  technical  advice  for  new  projects  include 
feasibility analysis and cost and benefit analysis.  

In  order  to  understand  the  type  of  advice  that  JASPERS  advisory  provides  to  assisted 
projects,  we  analysed  the  contribution  of  JASPERS’s  support  as  reflected  by  action 
completion notes (ACNs) for completed assignments. We carried out this analysis for 81 
new  major  projects  assisted  by  JASPERS  and  submitted  to  the  Commission  for 
assessment  and  approval  based  on  Art  102.2.  This  sample  represents  88%  of  all  such 
projects  in  the  Commission’s  database104.  The  data  collected  from  the  ACNs  of  these 
projects is summarised in Table 5.1.4.  

103  For  Water  and  wastewater  the  comparison  cannot  be  made  since  all  projects  in  this  sector  were  assisted  by 

JASPERS advisory for preparation.  

104  There  are  92  major  projects  submitted  for  Art.  102.2  and  which had  been  supported  by  JASPERS  advisory  for 
preparation.  Of  these  projects,  Member  States  provided  the  ACNs  for  81  projects  together  with  project 
application. We note that there is no regulatory obligation for the Member States to provide the ACNs of a project 
assisted  by  JASPERS  together  with  the  project  application,  although  we  consider  it  as  a  good  practice.  By 
comparison, for projects submitted for adoption based on Art 102.1, the corresponding ACNs (if applicable) are 
not available in the Commission’s SFC2014 system. Starting with 2019, however, JASPERS sends all the ACNs 

94 

With Jaspers TANot assistedWith Jaspers TANot assistedWith Jaspers TANot assistedWith Jaspers TANot assistedEnergy and solid waste2091298746914591Rail, air, maritime19112298103123404452Roads3292552268120412915677Smart development and ICT123119614956Water and wastewater1558724Sector\ Median financingEIB loan (mill euro)Total cost (mill euro)ERDF grant (mill euro)CF grant (mill euro) 
 
 
 
 
 
 
                                                 
Table 5.1.4 Projects supported by JASPERS for preparation by topic  

Projects advised 
for the topic 

% in all project 
analysed 

Topic 

(1) 

(2) 
77 
77 
49 

(6) 
95% 
95% 
60% 

Feasibility study 
Cost benefit analysis 
Climate issues 
Risk and sensitivity 
analysis 
Environmental issues 
Project description 
Project costs 
Financial plan 
Capacity beneficiary 
State aid 
Implementation schedule 
Indicators 
Note: Data covers 81 major projects submitted for approval based on Art 102.2. 
Source: ACNs for projects assisted by JJASPERS advisory.  

59% 
54% 
42% 
37% 
32% 
28% 
25% 
25% 
17% 

48 
44 
34 
30 
26 
23 
20 
20 
14 

The  classification  of  topics  presented  in  column  (1)  of  Table  5.1.4  is  derived  from  the 
structure  of  the  analysis  provided  by  JASPERS  in  ACNs.  For  each  of  these  topics, 
JASPERS  provides  a  detailed  overview  of  the  project,  followed  by  a  text  box  with 
JASPERS’ assessment, comments for improvement, and significant outstanding issues (if 
applicable).  Further,  the  ACNs  also  mention  frequently  the  topics  on  which  JASPERS 
provided technical advice, as well as topics on which the experts provided a review. On 
this basis, we classified the projects analysed by topic in two main types: 1) projects for 
which  JASPERS’  experts  stated  that  they  contributed  to  the  revision  on  the  topic  in 
previous  versions  of  project  documentation,  or  provided  comments  or  signaled 
outstanding issues at the time when the ACN was finalized; and 2) projects which did not 
meet the criteria for 1), but for which JASPERS’ experts provided a positive assessment 
or a description105.  

In  this  way  we  learn  that  the  most  frequent  topics  on  which  JASPERS  contributed  to 
project preparation (type 1) above) include cost-benefit analysis and feasibility studies ( 
in 95% of the project analysed). For the remaining topics, the frequency of contribution 

for  the  completed  assignments  to  the  Commission,  and  these  are  subsequently  distributed  to  the  relevant  desk 
officers in DG REGIO.     

105  See the section on effectiveness for a more detailed classification. 

95 

 
 
 
 
 
                                                                                                                                                 
varies significantly across projects, possibly also due to the scope of the initial request for 
the assignment from the Member State. 

For a comparison with the period 2007-2013 we collected the findings on this issue from 
the previous Commission’s evaluation of JASPERS carried out in 2012. Tables A3.6 in 
Annex 3 presents the details. On this basis, we note that cost-benefit analysis had been a 
major  topic  for  assistance  for  a  large  majority  of  assisted  projects  also  in  the  previous 
period,  while  for  the  remaining  topics  the  extent  of  JASPERS’s  contribution  appears 
more  extensive  in  the  current  period,  as  measured  by  frequency  of  topics  in  assisted 
projects. 

For 42% of new major projects assisted and submitted, JASPERS technical assistance 
was  requested  from  the  start  of  the  project  preparation.  For  additional  50%  of  the 
projects, JASPERS assistance started during then stage of core preparation.  

Further  on  JASPERS’  contribution,  we  also  analysed  the  stage  of  JASPERS’ 
involvement  in  project  preparation.  This  analysis  covers  almost  all  (99%)  of  the  140 
projects assisted by JASPERS advisory and submitted to the Commission for adoption/ 
approval.  For  these  projects  we  compared  the  calendar  duration  of  the  advisory 
assignments reported in JADE with the timeline for the preparation of projects reported 
by Member States in their applications for major projects in SFC2014.  

On this basis, we classified the advisory assignments  as  follows:  Group 1:  assignments 
created  from  the  start  of  the  core  preparatory  stage  of  the  project  or  earlier;  Group  2: 
assignments  created  after  the  start  but  during  the  core  preparatory  stage;  and  Group  3: 
assignments  created  after  the  core  preparatory  stage.  For  the  definition  of  the  core 
preparatory  stage,  we  considered  the  following  steps  in  process  preparation:  feasibility 
study, cost benefit analysis, environmental impact assessment and design studies106.  

Table 5.1.5 presents the number of projects assisted by JASPERS advisory by sector and 
stage of involvement (i.e. group). 

106  We present the timeline of project preparation and implications for JASPERS’ work in more details in the analysis 

of effectiveness in the next section.  

96 

 
 
 
  
 
 
 
 
 
 
                                                 
Table 5.1.5: Stage involvement of JASPERS advisory in project preparation 

Sector 

Number 
projects 

Group 1  
(from start) 

Energy and solid waste 
Rail, air, maritime 
Roads 
Smart development and 
ICT 
Water and wastewater 
Total 

13 
52 
32 

18 
24 
139 

3 
23 
5 

11 

Group 2  
(during 
prep) 
8 
26 
23 

Group 3  
(after 
prep) 
2 
3 
4 

% Group 
1 by 
sector 
23% 
44% 
16% 

6 

1 

61% 
71% 
42% 

% in total 

17 
59 
42% 
 Note: Data covers 99% of new major projects assisted by JASPERS and submitted to the 
Commission. 
Source:  JASPERS’  JADE  database,  December  2018;  European  Commission  SFC2014, 
October 2019 

7 
70 
50% 

0 
10 
7% 

On  this  basis  we  learn  that  JASPERS’  assistance  was  required  from  the  start  of  the 
project  preparation  in  42%  of  cases.  Across  sectors,  this  was  the  case  especially  for 
projects in Water and wastewater treatment (71%) and for Rail, air and maritime (44%). 
The sector with  the lowest  relative share of projects  assisted from  the beginning of the 
preparation stage is Roads (16%).  

In terms of size of projects, we find that projects assisted from the start of the preparation 
stage have a median cost  of 212 million  euro, while the ones for which  advisory  starts 
during  the  preparation  stage  tend  to  be  smaller  (198  million  euro).  The  few  projects 
assisted after the end of the core preparation stage, however, are the largest (315 million 
euro median total cost).  

Further  details  on  the  nature  of  JASPERS’  contribution  to  supported  projects  can  be 
inferred  by  linking  the  data  on  stage  of  involvement  with  the  data  on  JASPERS’ 
contribution from the ACNs. Table 5.1.6 below presents the details.  

The  data  in  Table  5.1.6  covers  87%  of  all  new  major  projects  assisted  by  JASPERS 
advisory  and  submitted  to  the  Commission  for  Art.  102.2  (for  which  we  could  collect 
data from ACNs).  

On this basis, we learn that for 40% of projects in which JASPERS provided advice on 
feasibility  study,  the  advisory  assignments  started  from  the  beginning  of  project 
preparation.  

97 

 
 
  
 
 
 
 
 
 
 
 
Table  5.1.6  JASPERS’  contribution  by  topic  and  stage  involvement  in  project 
preparation 

Note: Data covers 80 projects assisted by JASPERS and submitted for approval for Art. 
102.2. 
Source: ACNs for JASPERS advisory support (for topic) and JADE and SFC2014 
(classification by group). 

For the remaining topics the interpretation is similar. For the financial plan, climate, and 
environmental  issues,  we  learn  that  the  advisory  support  was  most  frequent  for  the 
assignments which started during the preparatory stage.  

More generally, however, we also note that advisory support for most topics appears also 
in  assignments  started  after  the  core  preparation  (group  3)  –  a  situation  which  can  be 
explained  if  JAPERS  was  asked  to  review  the  project  documentation  and  application. 
This  also  indicates that the extent and depth  of JASPERS’ advice on topics is  likely to 
differ  by  stage,  possibly  with  implications  on  the  effectiveness  of  the  technical 
support107. This aspect is analysed further in the section of effectiveness.  

Technical advisory for non-major projects 

Representing  a  quarter  of  advisory  support  for  project  preparation,  assignments  for 
non-major  projects  for  the  sector  of  Water  and  wastewater  treatment  maintain  the 
highest share in the portfolio. 

During the reference period, JASPERS advisory provided support also for 140 non-major 
projects,  representing  25%  of  all  assignments  for  advisory  for  the  ESIF  mandate.  The 
non-major projects include all projects which cannot qualify as major projects according 
to the regulatory definition for ESIF investments, and they are subject to the assessment 
and approval of Managing Authorities in the Member States. As regards the completion 

107 

It may also be an indication that the manner in which the Member States approach JASPERS’ advisory assistance 
was not fully coordinated.  

98 

Group 1 (from start)Group 2 (during prep)Group 3 (after prep)(1)(2)(3)(4)(5)Feasibility study7740%52%8%Cost benefit analysis7740%52%8%Climate issues4931%57%12%Risk and sensitivity analysis4844%50%6%Environmental issues4436%57%7%Project description3450%41%9%Project costs3040%47%13%Financial plan2635%58%8%Capacity beneficiary2348%48%4%State aid2040%50%10%Implementation schedule2055%45%0%Public procurement1753%41%6%Indicators1443%50%7%Stage involvementProjects advised for the topicTopic 
 
 
 
 
 
 
                                                 
of advisory assignments for these projects during the reference period, more than half of 
them were already completed or administratively closed by end December 2018.  

In terms of distribution by sector, more than one third of the non-major project assisted 
by  JASPERS  are  in  Water  and  wastewater  (34%),  followed  by  Rail,  air  and  maritime 
(with 23%). Table 5.1.7 presents the details.  

Table 5.1.7 Non-major projects assisted by JASPERS TA, by sector and status 

Sector \ Status 

On-going 

Total  % in total 

Energy and solid waste 
Rail, air, maritime 
Roads 
Smart development and ICT 
Water and wastewater 
Total 
% in total 
Sources : JASPERS’s JADE database by end December 2018 

9 
21 
10 
6 
18 
64 
46% 

18 
32 
13 
30 
47 
140 

Completed 
and closed 
9 
11 
3 
24 
29 
76 
54% 

13% 
23% 
9% 
21% 
34% 
100% 

On this basis, we see that the sector of Water and  wastewater treatment maintained the 
highest share of assignments for non-major projects in both periods. 

Relative to the previous period, we see that the structure of the portfolio for non-major 
projects  has  not  changed  much.  During  2007-2013,  JASPERS  assisted  most  frequently 
non-major projects in Water and wastewater (38% compared to 34% currently), followed 
by  Knowledge  Economy,  energy  and  solid  waste  combined  (33%  compared  to  34%). 
Table A3.7 in Annex 3 presents the distribution of assignments for non-major projects by 
sector for the period 2007-2013.  

A  majority  of  assignments  for  non-major  projects  is  concentrated  in  three  main 
beneficiary countries.  

As regards distribution by country, for the current period, JASPERS’ portfolio indicates 
that  a  large  majority  (74%)  of  these  assignments  were  carried  out  in  three  main 
beneficiary countries:  Croatia, Romania and Bulgaria, while the remaining assignments 
are  distributed  across  other  11  Member  States.  Table  A4.4  in  Annex  4  presents  the 
details. 

Therefore,  we  note  that  the  distribution  of  JASPERS’s  assignments  for  non-major 
projects  in  the  current  period  is  highly  skewed  towards  these  three  main  beneficiary 
countries which are also among the largest  beneficiaries for advisory support for major 
projects.  On  the  one  hand,  this  indicates  higher  needs  for  technical  support  for  project 

99 

 
 
 
 
  
  
 
 
 
 
 
preparation  for  these  countries.  On  the  other  hand,  however,  this  finding  also  suggests 
that  the  realization  of  assignments  for  non-major  projects  is  not  fully  aligned  with  the 
initial  expectations  and  priorities  set  out  in  the  grant  agreements  between  the 
Commission  and JASPERS. While the grant  agreements for the ESIF mandate indicate 
non-major projects as a priority especially for countries with fewer major projects (CY, 
EE,  LV,  LT,  MT,  SK,  and  SI),  we  find  that  JASPERS  assisted  only  11  non-major 
projects  in  these  countries.  This  is  due  to  the  fact  that  JASPERS  advisory  services  are 
primarily driven by the demand coming from the Member States, and the demand from 
the countries with fewer major projects for advisory for non-major projects proved lower 
than expected initially.  

By comparison, in  the previous period, JASPERS advisory  completed 133 assignments 
for  ESIF  non-major  projects  across  12  Member  States,  representing  21%  of  all 
assignments  for  project  preparation.  Also  for  2007-2013,  a  majority  (63%)  of 
assignments  for  non-major  projects  were  concentrated  in  three  countries  (Romania, 
Poland and Bulgaria). Between the two periods, significant reductions in relative shares 
of such assignments in JASPERS’ portfolio are observed for Slovenia (12%) and Poland 
(10%).  Table  A3.8  in  Annex  3  presents  the  distribution  of  JASPERS’  assignments  for 
non-major projects by country for 2007-2013.  

For  70%  of  non-major  projects  assisted  by  JAPERS,  beneficiaries  in  the  sampled 
countries  prepared  feasibility  studies  and  cost  benefit  analyses  even  in  absence  of  a 
regulatory  obligation.  Most  frequent  topics  for  advisory  support  include  feasibility 
issues and the cost benefit analysis.   

Further  for  the  analysis  of  JASPERS’  advisory  for  non-major  projects  we  screened  49 
ACNs and closure notes for JASPERS’ assignments for non-major projects completed in 
4 Member States sampled for this evaluation: Croatia, Poland, Romania and Slovakia108. 
The  assignments  analysed  represent  74%  of  the  ESIF  portfolio  for  assignments 
completed for non-major projects109. 

In terms of size, the median total cost for these projects is 35 million euro, with values 
ranging  from  a  minimum  of  7  million  euro  to  a  maximum  of  79  million  euro110. 
Relatively  larger  non-major  projects  in  terms  of  median  total  cost  are  assisted  by 
JASPERS  in  sectors  such  as  Energy  and  solid  waste  (54  million  euro)  and  Water  and 
wastewater (49 million euro), while projects assisted in Smart development and ICT have 
a smaller median cost (19 million euro).  

108 

Italy, a Member State also sampled for this evaluation and relevant for the ESIF mandate, had no assignments for 
non-major projects until end 2018. 

109  The ESIF portfolio includes 66 assignments completed for non-major projects. 
110  The  difference  between  median  and  average  costs  in  this  care  is  not  high.  The  average  total  cost  is  EUR  36 
million  for  the  39  ESIF  projects  with  data  available.  This  does  not  take  into  account  one  outlier  of  EUR  484 
million which refers to an assignment for project initially conceived as a major project.  

100 

 
 
 
 
 
 
                                                 
By comparison, for the previous programming period, the REGIO evaluation from 2012 
informs us that the average total  cost for non-major projects assisted by  JASPERS was 
30 million euro, with the largest projects assisted in Water and wastewater (average total 
cost of 35 million euro).  

As regards topics of JASPERS’ contribution to the preparation of non-major projects, we 
learn that there are two types of situations. First, for almost 70% of the projects analysed, 
JASPERS  carried  out  an  analysis  similar  to  major  projects,  contributing  to  and/  or 
reviewing all relevant topics such as project description, feasibility analysis, cost benefit 
analysis etc. This indicates that for a majority of non-major projects beneficiaries prepare 
project  documentation  at  similar  standards  as  for  major  projects,  even  if  the  regulation 
does not require them to do so111. Further, when providing assistance for such projects, 
JASPERS issued an ACN with  the same structure as  for a major project  (labelled long 
ACN). Second, there are also non-major projects for which JASPERS’ analysis is much 
shorter,  covering  broadly  the  main  aspects  of  the  project,  and  so  is  the  corresponding 
ACN (labelled short ACN).  

As  for  major  projects,  we  classified  the  technical  assistance  according  to  evidence  for 
JASPERS’  contribution  recorded  in  the  advisory  notes.  We  considered  that  JASPERS 
provided advice and direct input into project analysis and preparation whenever that was 
clearly stated in the action completion note for the respective topic, or when JASPERS’ 
experts made comments or raised significant issues for the topic in question in the final 
version of the ACN. The results are summarized in Table 5.1.8. 

Table 5.1.8 Topics of technical advice for assisted non-major projects 

111  We also found that, in some cases, projects were initially prepared as major projects and reclassified as non-major 

at later stages.  

101 

Long ACNShort ACNProject description1061633%Option analysis2593469%Technical solution1101122%Demand analysis2563163%Financial and economic analysis2743163%Projects costs70714%Procurement and implementation1621837%Indicators2024%Risk and sensitivity analysis1401429%Financial plan1201224%State aid52714%Capacity beneficiary61714%Total projects analysed331649100%Topic JASPERS' contribution to project analysisProjects assisted by JASPERS on All projects with ACN% in total 
 
 
 
 
 
 
                                                 
Source: ACNs for 49 completed non-major projects with JASPERS advisory.  

Table 5.1.8 reads as follows. For project description, for examples, JASPERS assisted 10 
projects for which it issued a long ACNs, and 6 projects for which it issued short ACNs. 
Overall,  the  projects  for  which  Jaspers  provided  support  related  to  project  description 
represent 33% of all projects analysed.  

On this basis, we find that, for non-major projects, JASPERS contributes most frequently 
to  topics  such  as  option  analysis  and  cost-benefit  analysis  (in  60-70%  of  the  projects). 
Least  frequent  topics  for  JASPERS’  contribution  are  indicators,  state  aid,  project  costs 
and capacity of beneficiaries. As expected, there are also differences in the coverage of 
technical  contribution  between  projects  with  long  and  short  ACNs,  with  significantly 
more substantive and wider technical support provided for the first type of projects.  

In terms of differences in size between these two types of projects, when compared with 
the  projects  with  long  ACNs,  the  non-major  projects  with  short  ACNs  tend  to  have 
smaller  median  total  costs  (up  to  18  million  euro)  in  sectors  such  as  Rail,  air  and 
maritime and Roads, but larger median total costs (40 million euro) for projects in Smart 
development and ICT. 

Review function 

JASPERS’  new  review  function  is  highly  relevant  since  it  assesses  the  quality  of  all 
new major  projects  submitted to  the Commission under the legislative framework for 
the period 2014-2020. Of the two procedures, 71% of new major projects were assessed 
by the PSA service. 

For  both  procedures  (Art.  102.1  and  Art.  102.2)  described  in  Chapter  2,  JASPERS’ 
contribution is highly relevant for the assessment of major projects since essentially all 
new  major  projects  notified  or  submitted  for  approval  of  EU  financing  need  to  go 
through one of the two procedures.  

Based on JASPERS’ portfolio data, we find that JASPERS carried out 175 assignments 
for IQR and PSA assessments until December 2018. This data is, however, very dynamic 
as  it  changes  frequently  with  new  major  projects  submitted  for  assessment  by  Member 
States. When looking into the Commission’s SFC2014 system for projects submitted 10 
months later (in October 2019), for instance, the number of projects with IQR and PSA 
assessments were already 210. Furthermore, for the PSA service, it is also possible that 
there  are  more  than  one  PSA  reports  for  a  given  projects  if  the  Commission  asks 
JASPERS to issue additional PSA reports for projects revised following interruptions for 
quality issues during the approval procedure.  

102 

 
 
 
 
 
 
 
 
 
Therefore,  for  the  analysis  of  the  review  function,  we  used  the  data  on  new  major 
projects submitted or notified from the Commission’s SFC2014 database since it covers a 
longer  period  after  July  2018  when  the  revision  of  the  CPR  through  the  Omnibus 
Regulation was adopted (discussed below). This dataset spans until October 2019, and it 
includes 210 major projects relevant for JASPERS’ review function. 

In  Table  A4.3  in  Annex  4  already  mentioned  above  we  summarize  the  number  of  new 
major  projects  submitted  by  Member  States  to  the  Commission  for  the  two  procedures 
for  adoption/  approval.  This  data  indicates  that,  of  all  210  new  projects  in  the 
Commission database by October 2019, 71% had been submitted for the Commission’s 
assessment  and  adoption  based  on  Art  102.2,  thus  implying  PSA  assessments  from 
JASPERS. The remaining 29% of projects  approved by the Commission were  assessed 
based on the IQR process provided for by Art. 102.1 procedure. 

While  there  is  significant  variability  across  projects  and  countries,  the  PSA  based 
procedure is more likely to be chosen for larger projects, for projects submitted before 
the  full  year  with  the  Omnibus  Regulation  (2019),  and  for  projects  not  assisted  by 
JASPERS advisory.  

At country level we can identify two main practices. First, there are Member States who 
seem  to  have  a  preference  for  one  of  the  two  services.  Until  October  2019,  countries 
which  used  only  the  IQR  service  included  Croatia,  Germany,  Latvia,  Lithuania  and 
Malta.  For  Art  102.2  procedure,  countries  which  submitted  projects  only  directly  for 
Commission’s  assessment  included  Austria,  Greece,  Hungary,  Portugal,  Slovenia  and 
Spain. Second, there are also Member States which chose to use both possibilities for the 
assessment of their new major projects. These include: Bulgaria, Czechia, France, Italy, 
Poland, Romania and Slovakia. 

The choice of one of the two procedures for the assessment of the major projects is in the 
remit of the Member State. One possible reason for this choice refers to practical aspects 
of implementation. Prior to the revision of the CPR with the Omnibus Regulation in July 
2018112, Member States could certify expenditure for major projects submitted based on 
CPR  Art  102.2  as  soon  as  the  Commission  determined  the  completeness  of  project 
documentation and application in SFC2014. In contrast, if the project was first submitted 
to  JASPERS  IQR,  for  certification,  the  Managing  Authority  would  need  to  wait  until 
JASPERS’  assessment  was  complete  and  the  project  application  was  submitted  to  the 
Commission. Given the time difference between the two processes, our conjecture is that 
this might have provided an incentive to Member States to favor the procedure based on 
CPR Art 102.2 and trigger the use of the PSA service in the process113.  

112  Regulation (EU, Euratom) 2018/1046. 
113 

It is also possible that Member States were initially reluctant to use the procedure based on Art 102.1 since it was 
very new compared with the more established procedure of Commission’s assessment.  

103 

 
 
 
 
 
 
                                                 
With  the  Omnibus  regulation,  however,  this  incentive  was  balanced  between  the  two 
procedures, so that now, also for projects subject to IQR, there is the possibility to certify 
expenditure  as  soon  as  JASPERS  IQR  determines  the  completeness  of  project 
documentation  and  application.  In  Table  5.1.9  we  present  the  submission  of  major 
projects by type of procedure (102.1 or 102.2) and over time.  

Table 5.1.9 New major projects by year of submission and procedure 

Total  % of total 

Year 
submission 
14 
2015 
22 
2016 
64 
2017 
61 
2018 
49 
2019 
210 
Total 
% in total 
100% 
Source: European Commission SFC2014, October 2019 

Art. 102.2 
(PSA) 
14 
19 
51 
44 
22 
150 
71% 

Art. 102.1 
(IQR) 
0 
3 
13 
17 
27 
60 
29% 

7% 
10% 
30% 
29% 
23% 
100% 

This data indicates that the number of projects submitted for the procedure based on Art. 
102.2  was  systematically  higher  until  2019  –  a  first  year  which  shows  a  reversed 
emerging  trend.  Therefore,  this  preliminary  evidence  for  the  first  full  year  with  the 
Omnibus  Regulation  adopted  seems  to  point  towards  a  more  balanced  use  of  the  two 
procedures by the Member States.  

Nevertheless, there could be also other factors that influence the decision to use one of 
the  two  procedures  for  submission  of  major  projects,  such  as  the  characteristics  of  the 
projects and the experience of the Managing Authority or the beneficiary with JASPERS’ 
services.  For  this  reason,  we  ran  a  logistic  regression  model  in  order  to  analyse  the 
correlation between the use of the two procedures and factors such as: use of JASPERS 
advisory assistance for project preparation, the size of the projects in terms of total costs 
or EU amounts, year of submission relative to year 2019, and sector. This analysis covers 
96%  of  all  major  projects  submitted  to  the  Commission  for  the  two  procedures  with 
complete data for all variables in the model. The results from these models are included 
in Table A2.2.2 in Annex 2. 

From this analysis we learn that the likelihood of the use of the procedure based on Art 
102.2  (and  PSA)  increases  for  larger  projects  and  for  projects  submitted  before  2019. 
Furthermore, we also find that projects with JASPERS advisory in preparatory stages are 
more likely to be submitted for Art 102.1 (IQR), and so are also the projects prepared for 
sectors such as Rail, air and maritime and, to a lesser extent, Water and wastewater114.  

114  The estimates for sector have a weaker statistical significance.  

104 

 
 
 
 
  
 
 
 
                                                 
Capacity buildings and horizontal and strategic support 

Next,  for  the  ESIF  mandate,  we  look  at  JASPERS’  activities  for  capacity  building  and 
horizontal and strategic support. While all the activities aim at supporting the beneficiary 
countries  in  the  identification  and  preparation  of  projects  for  EU  investments,  in  the 
evaluation we distinguish between two main types of activities: trainings and workshops 
on relevant  topics (classified as capacity building  activities), and hands-on assignments 
supporting  beneficiaries  and  Managing  Authorities  in  carrying  out  more  horizontal  and 
strategic  tasks  such  as  preparation  of  national  plans  and  strategies  providing  the 
framework  for  the  EU  investments.  In  practice,  all  these  types  of  activities  aim  at 
supporting relevant technical capacities in beneficiary countries. 

Across all support for technical capacity, while the share of multi-country training and 
workshops on topics of high relevance for all countries has increased, the horizontal 
and strategic support tailored to country specific needs remains prevalent. 

The activities for capacity building and horizontal and strategic support were maintained 
in  JASPERS’s  mandate  also  in  the  current  period.  Based  on  JADE  data,  we  find  that 
JASPERS has carried out 220 such assignments until December 2018, with 75% of them 
for horizontal and strategic advice.  

By comparison, in 2007-2013 JASPERS completed close to 140 such assignments, with 
96%  for  horizontal  and  strategic  advice.  Therefore,  we  note  an  increase  in  the  relative 
share of assignments for capacity building to from 4% to 25% between the two periods.  

These assignments can be tailored to the needs of a specific country, or can be organised 
with  a  multi  country  perspective  for  topics  of  high  relevance  to  several  or  all  Member 
States. In Table 5.1.10 we present the classification by type of assignment and audience.  

Table 5.1.10 Assignments for capacity building and horizontal and strategic support  

Type assignments 

Country specific  Multi country 

23 

Capacity building 
Horizontal and strategic 
support 
Total 
% in total 
Source : JASPERS’s JADE database, 31 December 2018 

159 
182 
83% 

31 

7 
38 
17% 

Total 
54 

166 
220 
100% 

Starting with capacity building activities, in contrast to the previous period when almost 
all  assignments  were  country  specific,  we  see  that  there  has  been  an  increase  in  the 
frequency  of  multi  country  assignments  in  the  current  period.  JASPERS  explained  that 

105 

 
 
 
 
 
 
 
 
 
this  increase  in  multi-country  activities  was  enabled  by  the  launch  of  the  Networking 
Platform in 2013.   

A majority (57%) of activities for capacity building included in JASPERS’ portfolio in 
the current period  are  multi-country assignments.  The most  frequent  topics for these 
training  activities  include  climate  change,  environment,  state  aid,  and  cost  benefit 
analysis.   

Of all 54 assignments for capacity building in JASPERS’ portfolio for 2014-2020, 57% 
are  multi-country  assignments.  By  December  2018,  21  of  these  assignments  were 
completed  and  are  relevant  for  an  analysis  of  their  impacts  on  the  capacity  for  project 
preparation  in  the  Member  States.  In  addition,  we  found  3  multi  country  horizontal 
assignments  which  are  very  similar  in  nature  to  the  ones  for  capacity  building,  and 
therefore we include them in the analysis. Finally, from the online website of JASPERS’ 
Networking  Platform,  we  identified  other  14  multi-country  workshops  and  events 
organised by JASPERS during the period 2014-2018 - assignments which are in addition 
to  the  assignments  recorded  in  JASPERS’  database.  Overall  we  collected  a  set  of  38 
multi-country assignments completed over the reference period. Table 5.1.11 presents a 
classification of these assignments by topic.  

Table 5.1.11 Multi country assignments for capacity building 

Topic 

Number 
assignments 

Years when organised 

Climate change and 
environment 
State aid 
Cost benefit analysis 
Urban development  
Projects in energy sector 
Other topics 
Total 
Source : JASPERS’ JADE database, December 2018 and JASPERS’ Networking 
Platform online, September 2020 

2012, 2014-2018 
2014-2016 
2012, 2015-2018 
2016, 2017 
2014-2015 
2012-2018 

9 
5 
6 
5 
2 
11 
38 

The most frequent multi-country trainings/ workshops were organised for topics related 
to climate change and environment, and for state aid. Workshops for cost benefit analysis 
and urban development are the next most frequent types of workshops organised,  while 
for the remaining topics JASPERS organised about one event each topic over the period 
2012-2018. About 62% of these activities were carried out at the very beginning of the 
programming period (or before its start) until year 2015.  

106 

 
 
 
 
 
  
 
 
 
 
The demand for  country specific capacity building activities was  not  high during the 
reference period of the evaluation. 

As  regards  the  country  specific  assignments  for  capacity  building,  19  out  of  23 
assignments were completed or still on-going by December 2018 in 10 Member States.  
More  than  60%  of  these  assignments  are  dedicated  to  climate  change  adaptation  and 
environment,  with  the remaining assignments  created mostly for horizontal aspects  and 
planning and implementation.   

From the analysis at country level carried out for the five Member States sampled for this 
evaluation,  however,  we  learn  that  the  demand  for  this  type  of  tailored  support  for 
capacity building was rather limited during the reference period. These countries include 
Croatia,  Italy,  Poland,  Romania  and  Slovakia,  and  they  account  for  43%  of  all  such 
assignments.  

From Poland, for instance, we learnt that, while the country sends numerous participants 
to multi-country trainings and workshops for capacity building, for tailored support it has 
identified  only  climate  change  adaptation  as  a  relevant  topic  and  requested  one 
assignment.  Similarly,  Slovakia  (with  2  assignments)  did  not  engage  in  significant 
country  specific  capacity  building,  partly  due  to  challenges  they  experienced  with  staff 
turnover.  Like  Poland,  however,  also  Slovakia  participated  frequently  in  multi  country 
activities  for  capacity  building.  Further,  Croatia  had  only  one  request  for  this  type  of 
support,  while  Italy  did  not  use  it  at  all.    The  only  country  in  the  sample  with  a  larger 
number  of  assignments  for  country  specific  support  was  Romania  (with  5  completed 
assignments),  which  also  opted  for  tailored  support  primarily  for  environmental  issues 
and climate change adaptation.  

Two  thirds of country specific assignments  for horizontal  and specific support are of 
horizontal nature. One third supported national strategies and plans.  

As  regards  the  horizontal  and  strategic  support,  132  out  of  159  assignments  had  been 
completed or were still on-going by December 2018. Of these, 95% are country specific 
assignments.  The  classification  of  these  assignments  by  main  type  and  sectors, 
determined based on their titles, is presented in Table 5.1.12 below. 

From this data we learn that the assignments supporting national strategies, masterplans 
and  plans  represent  32%  of  all  country  specific  assignments.  The  remaining  68%  were 
created  for  a  variety  of  horizontal  tasks,  including  support  for  analysis  of  sector  level 
investments, guidelines for CBA and other sector specific topics, preparation of project 
pipeline, preparation of operational programmes and other horizontal tasks. Therefore, on 
this  basis,  we  conclude  that  this  type  of  country  specific  support  is  predominantly 
horizontal and focuses relatively less on strategic support.  

107 

 
 
 
 
 
 
 
 
 
Table 5.1.12 Country specific assignments for horizontal and strategic advice 
Assignments  

Objective support 

Sectors/ topics 

Support national/ sectoral 
plans 

Support national strategies 

Capacity building 

Support for sectoral analysis 

Guidelines 

Project pipeline preparation 

Operational programme 
preparation 
Other  

Total 

22 

18 

14 

13 

11 

10 

9 

28 
125 

8 Transport, 8 Urban mobility; 3 
Waste; rest for other 3 sectors 
7 Transport; 3 Smart specialisation; 
3 Wastewater; rest for other 4 
sectors 
3 Multiple funding schemes; 2 
Disaster risk management; rest for 
other 9 topics 
6 Energy, 2 Transport, 2 Urban 
development; rest in other 3 sectors 
5 CBA; 4 Water and wastewater; 
rest for other 2 topics 
3 Energy efficiency; 2 Waste; rest 
for other 5 sectors 

Note:  Data  covers  assignments  completed  or  on-going  by  December  2018.  “Other” 
refers  to  a  variety  of  topics  such  as  ex-ante  conditionalities,  benchmarking,  transport 
models, etc.115. 
Source: Authors’ classification based on assignment titles in JASPERS’ JADE database, 
December 2018. 

Close to 60% of country specific horizontal and strategic support is concentrated in 3 
Member States (Romania, Poland and Croatia). 

At  country  level,  we  find  that  close  to  60%  of  country  specific  assignments  are 
concentrated in three Member States: Romania, Poland and Croatia, while the remaining 
assignments are distributed across other 12 countries. Table A4.5 in Annex 4 presents the 
distribution of these assignments by country.  

More than 60% of activities for capacity building and horizontal and strategic advice 
were started at the beginning of (or before) the programming period 2014-2020.  

Over  time,  64%  of  all  assignments  for  capacity  building  and  horizontal  and  strategic 
advice were completed during the period 2012-2015 since many were organised in view 
of  the  preparation  for  the  new  programming  period  2014-2020.  In  our  view,  this  helps 

115  Some country specific HSS assignments for workshops and seminars are not included here but they are taken into 

account in the classification for capacity building. 

108 

 
 
  
  
  
 
 
 
 
                                                 
explain  also  the  prevalence  of  the  horizontal  support,  as  many  of  these  assignments 
focused on the preparation of operational programmes, identification of project pipeline, 
preparation  for  fulfilment  of  ex-ante  conditionalities,  sectoral  analyses  for  planning 
investments etc.  

Furthermore,  in  addition  to  the  activities  recorded  in  the  JASPERS’  database,  we  also 
looked  at  JASPERS’  Networking  Platform  online116.    Created  in  2013,  the Networking 
Platform  includes  information  on  JASPERS’  past  and  upcoming  events,  annual 
programmes starting with 2013 to date, and a library for JASPERS guidance and capacity 
building documents. The library includes around 60 publications, with more than 60% of 
them  covering  topics  such  as  cost  benefit  analysis,  knowledge  economy,  state  aid,  and 
waste management. Similarly, as with the activities, a large majority of these publications 
(75%) were published during 2012-2014, in preparation for the upcoming programming 
period.  

CEF Mandate 

For  JASPERS’s  CEF  mandate  we  distinguish  between  the  two  mandates  (CEF1.1  and 
CEF1.2) described in Chapter 2, with the evaluation looking only at the first mandate.  

Technical advisory for CEF projects 

For the first pilot (CEF1.1), JASPERS advisory assisted 29 projects, of which 23 were 
approved for CEF financing.  

Based on the final annual report of CEF1.1 mandate, we learn that JASPERS carried out 
31 assignments in support of preparation of 29 CEF projects,117 with a large majority of 
them in the rail sector.  Table 5.1.13 presents the details.  

Table 5.1.13 JASPERS’s assignments for CEF1.1 mandate 

Sector 
Rail 
Roads 
Inland waterways 
Total 

Assignments 
25 
5 
1 
31 

CEF projects 
assisted 

23 
5 
1 
29 

Source : Final Report for the CEF1.1 mandate 

At  country  level,  the  initial  agreement  for  the  first  CEF  pilot  (CEF1.1)  was  that 
JASPERS  would  support  projects  chosen  from  the  list  of  58  projects  in  7  cohesion 

116  www.jaspersnetwork.org 
117  Additional 3 projects supported during CEF1.1 were postponed for CEF1.2. Therefore, we do not include them in 

the CEF1.1 mandate. 

109 

 
 
 
 
 
 
 
 
                                                 
countries  included  in  Annex  I  of  the  Special  Grant  Agreement.  The  actual  number  of 
projects  supported  was  determined  by  the  budget  available  for  the  pilot  for  JASPERS 
assistance118.  Overall,  the  projects  supported  by  JASPERS  by  the  end  of  the  first  CEF 
pilot represents 55% of the projects listed as potential candidates in the SGA for CEF1.1. 
They were submitted for 3 CEF calls for the cohesion envelope  during 2014-2016, and 
23 of them were approved for CEF financing.  

As regards the contribution  of this first pilot to  the overall portfolio of CEF projects  at 
the  EU  level,    we  find  that  the  projects  assisted  by  JASPERS  represent  approximately 
11% of all the 265 CEF projects approved for cohesion and blending actions until August 
2019.  Out  of  14  cohesion  Member  States  with  projects  approved  for  financing,  6 
requested  JASPERS’  technical  support  for  project  preparation  in  proportions  ranging 
between  11%  and  47%  of  the  number  of  CEF  projects  at  country  level.  Table  A4.7  in 
Annex 4 presents  the distribution  of CEF projects  and JASPERS’ advisory  for CEF by 
country.  

Compared with all projects approved for CEF financing, the CEF projects assisted by 
JASPERS  advisory  are  more  concentrated  on  works  (rather  than  studies),  and  they 
tend to be much larger in terms of eligible costs. 

In  terms  of  type  of  projects,  81%  of  projects  supported  by  JASPERS  support  are  for 
works  or  mixed  (works  and  studies),  compared  with  64%  for  the  CEF  overall  project 
portfolio.  

As regards project size, we calculated the median total costs and CEF funding for all the 
projects  included in  the  Commission’s database  by  August  2019. Table  5.1.14 presents 
the details by type of project while also distinguishing between projects assisted and not 
assisted by JASPERS advisory. 

Table 5.1.14 Size of CEF projects assisted by JASPERS, relative to non assisted (median 
values) 

Type project 

Works 
Works + studies 
Studies 

Actual eligible costs (million 
euro) 

Actual CEF funding 
(million euro) 

Projects 
assisted by 
JASPERS 
114 
65 
4.4 

Projects not 
assisted by 
JASPERS 
35 
29 
3.7 

Projects 
assisted by 
JASPERS 
97 
55 
3.7 

Projects not 
assisted by 
JASPERS 
12 
22 
3.2 

118  The  budget  of  EUR  1.5  million  allocated  to  the  first  pilot  for  the  CEF1.1  mandate  was  first  decided  by  the 
Steering  Committee  of  CEF,  and  subsequently  allocated  by  country  according  to  pre-determined  national 
allocation keys.  

110 

 
 
 
 
 
 
 
                                                 
Note : data covers projects approved for CEF cohesion and blending actions (26 
projects assisted, and 239 projects not assisted).  
Source : DG MOVE data on CEF projects approved, August 2019 

On this basis, we learn that, relative to the ones not assisted by JASPERS, the projects 
assisted by JASPERS tend to be much larger in terms of median values of actual eligible 
costs and actual CEF funding for actions for works and mixed (works plus studies). For 
actions  including  only  studies,  the  median  costs  and  EU  grants  are  similar  for  the  two 
groups.  

Furthermore, from Jaspers we learnt also that 6 assignments for studies out of a total of 
15  were  dedicated  to  feasibility  studies  prepared  for  investment  projects  intended  for 
CEF financing119. 

In  a  sample  of  9  assignments,  frequent  topics  for  advisory  include  project  technical 
specification,  cost  benefit  analysis,  environmental 
issues,  procurement  and 
implementation, and grant application. 

In order to understand the type of support provided by JASPERS advisory to the assisted 
projects, we analysed the project development reports for a sample of 9 CEF projects for 
works assisted120. We classified JASPERS input to these projects as a contribution to the 
project on a given topic if there was any evidence in the assignment document of advice 
or comments on the topic. For a majority of these projects, JASPERS’ main contribution 
related  to  technical  specifications  of  the  project,  cost-benefit  analysis,  environmental 
issues, procurement and implementation aspects, and the grant application.  

As  regards  the  CEF1.2  mandate  started  in  2018,  the  timing  of  the  evaluation  overlaps 
only  with  a  few  months  into  this  mandate,  and  therefore  there  is  limited  scope  for  an 
assessment.  

Furthermore,  in  addition  to  the  assignments  for  technical  advisory,  the  JADE  database 
includes also two assignments for horizontal advice, but the number of is too limited to 
draw conclusions for an evaluation.  

IPA Mandate 

Technical advisory for IPA projects 

Relative to the previous period, JASPERS’ IPA mandate has expanded and diversified 
in the period 2014-2020. 

119  And 5 of these assignments for studies were still ongoing by end 2018.  
120  As one of these assignments was administratively closed, the relevant document reviewed was the action closure 

note.  

111 

 
 
 
 
 
 
 
 
 
 
                                                 
For  the  IPA  mandate  in  the  current  period,  JASPERS’  portfolio  includes  102 
assignments,  compared  with  6  assignments  completed  for  IPA  in  the  previous  period. 
The  IPA  mandate  now  covers  all  types  of  JASPERS  services,  except  for  the  review 
services which apply only to the ESIF projects. 

Almost three quarters of assignments in JASPERS’ portfolio for the IPA mandate are 
for advisory for project preparation. Projects for Water and wastewater represent 42% 
across sectors. 

JASPERS’ IPA portfolio is distributed in 72% assignments for support to preparation of 
projects, and 22% for activities for capacity building and horizontal and strategic advice. 
The  remaining  assignments  provide  support  for  the  implementation  of  projects.  The 
distribution of these assignments by sector is presented in Table 5.1.15. 

Table  5.1.15  JASPERS’  portfolio  for  the  IPA  mandate  2014-2018,  by  sector  and  type 
assignment 

Sector 

Project 
preparation 

Capacity 
building and 
horizontal 

Implementation 

Total  % total 

2 

7 

15 

21 

Energy and solid 
waste 
Rail, air and 
maritime 
Roads 
Water and 
wastewater 
Multi sector 
Total 
% total 
Source : JASPERS’ JADE database, December 2018. 

3 
22 
22% 

73 
72% 

10 

33 

4 

23% 

27% 

5% 

42% 

3% 
100% 

23 

28 

5 

43 

3 
102 
100% 

6 

1 

7 
7% 

While  the  sector  of  water  and  wastewater  remains  dominant  for  the  IPA  portfolio,  for 
transport projects we note a higher proportion of assignments for Rail, air and maritime 
relative to Roads. We found no assignment for the sector of smart development and ICT 
in the IPA mandate until December 2018.  

More  than half  of  the  advisory  assignments  are  concentrated  in  two  beneficiary  IPA 
countries. 

112 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
At country level,  across  5 countries  with  advisory  support over the reference period121, 
more  than  half  (53%)  of  assignments  for  project  preparation  are  concentrated  in  North 
Macedonia  and  Serbia.  For  Montenegro,  for  example,  project  preparation  was  more 
limited, with 13 out of 17 assignments concentrated on capacity building and horizontal 
support.  Table  A4.8  in  Annex  4  presents  the  distribution  of  assignments  for  the  IPA 
mandate by country. 

Overall, projects assisted by JASPERS advisory and approved for financing represent 
7% of all IPA projects approved for financing until 2018. In most of these assignments 
JASPERS  helps  with  the  guidance  and  supervision  of  contracts  for  technical 
assistance.  

In  order  to  understand  what  happened  to  the  projects  after  being  assisted  by  JASPERS 
advisory for preparation, we matched the data from JASPERS’ portfolio with the data on 
approved projects from DG NEAR. The data on IPA projects from DG NEAR refers to 
projects  approved  and  started  during  2011-2019122.    Overall,  this  dataset  includes  253 
projects,  of  which  17  could  be  matched  with  JASPERS’  assignments  for  the  current 
period. This data is summarized in Table 5.1.16, showing also the distribution by type of 
IPA project.   

Table 5.1.16 IPA projects started during 2011-2019 

Type IPA project 

All projects 

JASPERS 
assisted 

Investment facility 
Investment grant 
Technical assistance 
Total 
Source : DG NEAR dataset, August 2019 

4 
33 
216 
253 

3 
14 
17 

Therefore,  on  this  basis  we  learn  that  the  IPA  projects  assisted  by  JASPERS’  advisory 
represent 7% of the total number of IPA projects approved in the current period.  

As regards the type of advice provided by JASPERS advisory to IPA projects, it depends 
on  the  type  of  the  project.  As  reflected  in  the  dataset  from  DG  NEAR,  we  learn  that 
JASPERS can advise projects for investment grants, but also for supervision of technical 
assistance contracts for studies or other preparatory steps in view of investment. From the 
assignments  matched  between  the  two  datasets  we  see  that,  in  line  with  the  general 
pattern  for  all  IPA  projects,  JASPERS’  assignments  were  primarily  concentrated  on 

121  Projects are also included for Croatia for the period before its accession to the EU.  
122  We took projects starting in 2011 since the first JASPERS’ assignment for project preparation in the portfolio for 

2014-2020 is for a project starting in 2011.  

113 

 
 
 
 
 
  
 
 
                                                 
technical  assistance,  but  we  cannot  generalize  this  conclusion  as  the  portfolio  data  in 
JADE does not distinguish between the these types of projects.  

In terms of size of projects, based on the data from DG NEAR on project financing, we 
compared the median values of total costs for projects assisted by JASPERS and projects 
not assisted for those assignments that could be matched between the two datasets. Table 
5.1.17 presents the details by type of IPA project.   

Table 5.1.17 Median total amounts for IPA projects (euro)  

Projects not assisted  JASPERS assisted 

Type IPA project 
Investment facility 
Investment grant 
Technical assistance 
Note: Data covers 236 IPA projects not assisted by JASPERS, and 17 IPA projects with 
JASPERS support. All projects are approved and started during 2011-2019. 
Source: DG NEAR data, August 2019.  

13,325,000 
7,999,200 
925,000 

28,375,540 
950,000 

On this basis we learn that, for investment grants, projects assisted by JASPERS TA are 
much larger, but the numbers of such projects for the two groups are very different (with 
only  3  for  projects  assisted  by  JASPERS  advisory).  Therefore  this  evidence  is  rather 
preliminary.  As  regards  advisory  for  technical  assistance,  the  median  costs  of  projects 
between two groups (assisted and not assisted) are similar.  

Stakeholder consultation 

So far in this section we presented the assignments included in JASPERS’s portfolio and 
the corresponding projects based on data collected from databases and documents from 
the  Commission  services  and  JASPERS,  including  also  comparisons  with  the  previous 
programming  period  whenever  relevant.  This  evidence  reflects  the  extent  and  type  of 
JASPERS’s  services  requested  by  beneficiary  countries  during  the  first  5  years  of  the 
current programming period. 

In  order  to  understand  also  the  stakeholders’  perceptions  of  their  experience  with 
JASPERS  services,  we  carried  out  a  targeted  consultation  for  this  evaluation.  The 
consultation  was  available  online  during  the  period  June  –  September  2019,  and  it 
gathered  replies  from  210  respondents.  Details  on  the  composition  of  the  pool  of 
respondents and the distribution of replies by survey question are included in Annex 1.  

The  respondents  to  the  consultation  come  with  diverse  perspectives  on  JASPERS’s 
services  as  they  include  Managing  Authorities,  beneficiaries  or  promoters  of  projects, 
professionals  from  consultancy  companies,  and  staff  from  the  European  Commission. 
Many  of  these  stakeholders  declared  that  they  had  experience  with  all  JASPERS’ 
services, and especially with technical support for preparation of major projects (87%).    

114 

 
 
 
 
 
 
 
 
 
Across  mandates,  although  more  than  80%  of  all  respondents  said  they  had  experience 
with JASPERS services for the ESIF mandate, we also find that one third of them also 
declared  that  they  experienced  JASPERS’  services  for  more  than  one  of  the  three 
mandates  (ESIF,  CEF,  IPA).  Therefore,  as  it  is  very  likely  that  the  opinions  elicited 
through  the  consultation  are  informed  by  experiences  across  mandates  and  types  of 
services  combined,  we  summarize  the  replies  to  the  consultation  at  the  level  of  all 
mandates, by type of respondent.  

Almost all of the 210 stakeholders responding to the consultation consider JASPERS 
relevant  for  support  to  increase  project  quality  and  for  assisting  the  alignment  of 
projects  with  EU  environmental  directives.  Further,  a  large  majority  perceive 
JASPERS as relevant for support in developing skills for cost benefit analysis.  

When  asked  for  their  opinion  on  the  relevance  of  JASPERS  services  for  their  needs,  a 
large majority of these respondents perceive JASPERS as being relevant or very relevant 
for  helping  increase  projects  quality  (97%  of  the  valid  replies),  for  assisting  with 
compliance with  EU environmental directives  (90%), and for developing skills  for cost 
benefit analysis (88%). Other needs such as speeding up the approval process, increasing 
capacity  to  prepare  projects  and  reviewing  applications  were  also  chosen  by  a  large 
majority  of  respondents.  Less  frequent  needs  relevant  for  JASPERS’s  services  are 
perceived to  refer to  compliance with  EU public procurement and compliance with  EU 
state aid. Table A1.7 in Annex 1 includes the details for this question. 

The  most  frequent  reasons  for  requesting  JASPERS’  services  are  perceptions  of 
increased chances for faster project approval.  

In  terms  of  reasons  for  requesting  JASPERS  services,  most  frequent  choices  were 
increased chances for approval of projects in terms of a positive decision and speed of the 
decision process. Lack of skills for project preparation or for the application process was 
also chosen as a reason for asking for JASPERS’ assistance, albeit less frequently. Table 
A1.6 in Annex 1 includes the details for this question. 

A large majority of stakeholders think that JASPERS has a high ability to respond to 
needs for improving project quality, for ensuring compliance with EU environmental 
regulations, and for assisting with the development of skills for cost benefit analysis.  

Respondents  were  also  asked  about  their  perceptions  of  the  extent  to  which  JASPERS 
could respond to specific needs with its services. Overall, a very large majority of replies 
(85-94%) indicated a high ability to respond to needs for improving project quality, for 
ensuring compliance with EU environmental regulations, for helping preparing projects, 
and for assisting the development of skills for cost benefit analysis. Less frequent replies 

115 

 
 
 
 
 
 
 
 
 
referred  to  compliance  with  EU  public  procurement  and  assistance  for  project 
implementation. Table A1.8 in Annex 1 includes the details for this question. 

Overall, a very large majority of stakeholders rated highly JASPERS’ services in terms 
of coherence with EU standards, quality of advice, effectiveness and timeliness. 

Finally,  in  terms  of  customer  satisfaction,  a  very  high  proportion  of  stakeholders  (84-
94%) rated highly JASPERS’ services in terms of coherence of their assistance with EU 
standards,  quality  of  advice,  expertise  and  deliverables,  achievement  of  the  assignment 
objectives, and efficiency and timeliness of service. Table A1.9 in Annex 1 includes the 
details for this question. 

5.2 

EFFECTIVENESS 

The main objective of Commission’s initiative with technical assistance from JASPERS 
is  to  support  Member  States  in  preparing  investment  projects  at  EU  standards  and 
compliant with EU regulatory requirements. The advisory function aims at ensuring that 
projects  of  good  quality  are  proposed  and  succeed  in  a  timely  manner  with  the 
application for EU funding so that they reach the stage of implementation as quickly as 
possible.  For  the  review  function,  JASPERS  is  to  provide  an  independent  expert 
assessment  of  major  projects  submitted  for  EU  financing.  And  for  the  function  of 
horizontal  advice  and  capacity  building,  the  objectives  include  assisting  in  the 
preparation  of  strategies  and  national  plans  for  investments  supported  by  EU  funds, 
helping  identify  viable  project  pipelines  for  the  operational  programmes,  and  enabling 
strengthened national technical capacity for project preparation and implementation.  

In this section we analyse the extent to which and how these objectives are met by type 
of  mandate  and  by  type  of  JASPERS’  service.  We  analyse  the  effectiveness  of 
JASPERS’ services with respect to the extent to which the projects assisted by JASPERS 
advisory are subsequently submitted for approval for EU financing, the quality of these 
projects  and  their  success  in  the  approval  process,  the  effectiveness  of  JASPERS 
advisory  and  review  functions  in  signalling  quality  issues  identified  later  during  the 
approval process, the chances of projects not being interrupted if they had been assisted 
by JASPERS during project preparation, and the correlation between JASPERS technical 
assistance  and  projects’  progress  in  implementation.  For  the  review  function,  we  also 
analyse  the  time  needed  for  JASPERS’  assessment  in  overall  context  of  the  two 
procedures for the assessment and approval of major projects in the current period.  

This  section  is  organised  by  mandate  and  type  of  JASPERS’  service,  and  it  concludes 
with the online stakeholder consultation on the effectiveness of JASPERS’ services. The 
main findings are summarised in the Text Box 5.2.1 below.  

116 

 
 
 
 
 
 
 
 
Text Box 5.2.1 Main findings for the effectiveness of JASPERS’ services 2014-2018 

ESIF MANDATE 

1)  Major  projects:  Of  all  advisory  assignments  for  major  projects  completed,  70% 

resulted in major projects adopted by the Commission by October 2019.  

2)  Major  projects:  Compared  with  the  first  five  years  of  2007-2013,  JASPERS 
completed fewer advisory assignments for major projects in the current period, but it 
helped mobilize 22% more EU resources with the projects assisted and approved. 

3)  Major  projects:  The  median  duration  of  JASPERS’  advisory  involvement  in  project 
lifecycle is of 20 months, although there is a high variability across assignments by 
sector. 

4)  Major projects: In 42% of cases, JASPERS’ assistance is requested during the stage 
of core project preparation, with a resulting median coverage of 16% of this stage by 
advisory in calendar terms. For 18% of assignments, JASPERS is involved from the 
very beginning of the core project preparation, with a full coverage of this stage in 
calendar terms. 

5)  Major projects: Involvement of JASPERS’ advisory support from the start of project 
preparation is positively correlated with a shorter duration of the preparation stage 
relative to the duration of the project lifecycle.  

6)  Major  projects:  Relative  to  the  previous  period,  there  has  been  a  significant 
reduction  from  82%  to  46%  in  the  overall  rate  of  interruptions  of  major  projects 
during the approval procedure based on Art 102.2.  

7)  Major  projects:  The  two  most  frequent  topics  for  EC  observations  in  interruption 
letters include cost  benefit  analysis  and feasibility study, with higher frequencies in 
non-assisted relative to assisted projects. 

8)  Major  projects:  Relative  to  assisted  projects,  the  projects  not  assisted  by  JASPERS 
advisory tend to have a higher number of critical issues identified in the PSA reports 
by JASPERS IQR. Nevertheless, there are no significant differences between the two 
groups of projects in terms of the number of observations raised by the Commission 
in interruption letters and the duration of interruptions.  

9)  Major projects: The effectiveness of JASPERS’ advisory to signal issues with project 
quality later identified during the approval process varies across topics. It is highest 
for  cost  benefit  analysis  and  feasibility  study,  but  much  lower  for  other  project 
features such as indicators, project description, or project timetable.  

10) Major projects: JASPERS’ advisory for project preparation has a positive effect on 
the likelihood that projects are not interrupted during the approval process based on 
Art 102.2. This effect is statistically significant when accounting for project features 
and country specific factors.  

117 

 
 
 
 
 
 
 
 
 
 
 
 
 
11) Major projects: JASPERS’ advisory support is positively correlated with progress in 
the  financial  implementation  of  major  projects,  but  only  for  projects  in  which 
JASPERS  intervenes  early  during  the  period  of  core  project  preparation  and  for 
younger, more recent new projects prepared.  

12) Non-major  projects:  Overall  effectiveness  of  JASPERS’  advisory  for  non-major 
projects  could  not  be  established  as  data  on  projects’  progress  with  approval  and 
implementation is not available at portfolio level. Preliminary evidence for a third of 
completed assignments, however, indicates an approval rate for such projects by the 
Managing Authorities of at most 60%.  

13) Non-major projects: The median  duration  of an advisory assignment  for non-major 
projects is 14 months, although there is significant variability across assignments.  

14) Non-major  projects:  Most  frequent  issues  raised  by  JASPERS  advisory  during 
project preparation relate to procurement and implementation, cost benefit analysis, 
and risk and sensitivity analysis. 

15) Review  function:  The  median  total  duration  of  the  process  of  assessment  and 
approval  of  major  projects  notified  for  Art  102.1  is  7  months.  Of  this,  the  IQR 
assessment  carried  out  by  JASPERS  IQR,  including  also  the  response  time  for  the 
Member States, has a median duration of 5 months.  

16) Review  function:  The  median  total  duration  of  the  process  of  assessment  and 
approval of major projects submitted for adoption based on Art 102.2 is 3 months for 
projects  not  interrupted,  and  6  months  for  projects  interrupted.  In  this  process, 
JASPERS IQR is highly effective in delivering timely PSA reports (within one month).  

17) Review  function:  Relative  to  the  period  2007-2013,  there  has  been  a  significant 
reduction  in  the  overall  time  taken  with  the  assessment  and  approval  of  major 
projects.  This  reduction  is  highest  for  the  procedure  based  on  Art  102.2  (with  a 
reduction  of  45%)  and  lower  for  the  procedure  based  on  Art  102.1  (with  a  12% 
reduction).  

18) Review function: The effectiveness of JASPERS IQR to signal issues of project quality 
in the PSA report is relatively high for most topics, and especially for issues related 
to project timetable, risk and sensivity analysis, feasibility study, and cost and benefit 
analysis. 

19) Capacity  building  and  HSS:  So  far,  while  a  large  majority  activities  for  capacity 
building were completed and achieved their objectives for delivering trainings for the 
Member  States,  such  activities  have  not  proven  sufficient  to  trigger  a  visible  ripple 
effect on the technical capacity of Member States to address effectively issues such as 
cost benefit analysis, climate change and environment in project preparation. 

20) Capacity  building  and  HSS:  JASPERS  has  proven  potential  to  promote  more 
strategic  and  innovative  approaches  for  building  technical  capacity,  but  such 

118 

 
 
 
 
 
 
 
 
 
 
 
 
approaches  were  not  yet  mainstreamed  at  the  portfolio  level  during  the  reference 
period.   

21) Capacity  building  and  HSS:  For  the  horizontal  and  strategic  support,  based  on  an 
analysis of sampled assignments, there are indications that JASPERS’ support helps 
compensate for  lower local  technical capacity for  guiding and supervising tasks on 
specialised topics of policy relevance. 

CEF1.1 MANDATE 

22) CEF1.1:    For  around  70%  of  the  CEF  assignments  for  works  completed  until  end 
2018,  JASPERS’  technical  assistance  was  requested  primarily  around  the  time  of 
project applications. Some of these projects, however, had been assisted by JASPERS 
for  project  preparation  during  the  previous  programming  period  (for  the  ESIF 
mandate). 

23) CEF1.1:  In  40%  of  cases,  the  studies  financed  by  CEF  and  assisted  by  JASPERS 
advisory  based  on  the  CEF1  pilot  mandate  were  feasibility  studies  prepared  for 
investment projects intended for the CEF financial support.  

IPA MANDATE 

24) IPA: More than 70% of the assignments included in JASPERS’ IPA portfolio are for 
technical  advisory  work.  So  far,  close  to  20%  of  these  assignments  resulted  in  IPA 
projects  approved  for  EU-funding  due  to  a  combination  of  factors  external  to 
JASPERS’ services.  

25) IPA:  Based  on  an  analysis  of  sampled  assignments,  there  is  indication  that  the 
effectiveness of JASPERS’ advisory support is limited in cases when the requests for 
support arrive late in the project preparation cycle.  

26) IPA: Beneficiaries appreciate JASPERS’ horizontal and strategic support especially 
as  it  helps  raise  awareness  and  familiarize  national  staff  with  respect  to  the 
management and appraisal of EU funded projects.  

ONLINE CONSULTATION 

27) More  than  80%  of  the  stakeholders  responding  to  the  consultation  considered  that 
JASPERS  had  a  significant,  and  even  decisive,  effect  on  project  technical 
specifications and its documentation. They also rated highly JASPERS’ assistance for 
aligning  projects  with  EU  and  national  policies,  and  for  enabling  faster  project 
preparation and approval.  

For each of these evaluation findings the supporting evidence is as follows.  

119 

 
 
 
  
 
 
 
 
 
 
 
 
 
 
ESIF Mandate 

Technical advisory for major projects 

Of  all  advisory  assignments  for  major  projects  completed,  70%  resulted  in  major 
projects adopted by the Commission by October 2019.  

As presented earlier, for the ESIF mandate, a significant proportion of JASPERS’ work 
is  dedicated  to  providing  technical  assistance  for  major  projects  with  the  objective  to 
assist  Member  States  in  getting  these  projects  ready  to  apply  for  EU  funds.  Over  the 
period 2014-2018, JASPERS’ portfolio of assignments for technical assistance included 
414  assignments  for  major  projects,  with  178  of  these  assignments  completed  during 
2018  and  additional  48  completed  during  2019.  Of  the  completed  assignments,  158 
(70%)  of  them  resulted  in  major  projects  adopted  by  the  Commission  by  October 
2019123. The adopted projects assisted by JASPERS cumulate a total EU funding of 23.9 
billion  euro  (ERDF  and  Cohesion  Fund  combined),  representing  65%  of  EU  resources 
mobilised by all major projects adopted by the Commission during the period analysed. 
Three  quarters  of  the  projects  assisted  by  JASPERS  are  new  major  projects,  while  the 
remaining projects are phased from 2007-2013. 

Compared with the first five years of 2007-2013, JASPERS completed fewer advisory 
assignments for major projects in the current period, but it helped mobilise 22% more 
EU resources with the projects assisted by advisory and approved by the Commission.  

By  comparison,  during  the  previous  programming  period  until  2011,  JASPERS 
completed  343  assignments  for  major  projects,  resulting  in  205  major  projects  adopted 
by  the  Commission  by  October  2012.  The  adopted  projects  assisted  by  JASPERS 
cumulated  a  total  of  19.6  billion  euro  (ERDF  and  Cohesion  Fund  combined), 
representing  65%  of  EU  resources  mobilised  by  all  projects  adopted  over  the  first  five 
years of the programming period.  

Therefore, when comparing the two periods, we see that in the current period JASPERS 
advisory  completed  fewer  assignments  for  major  projects  by  the  5th  year  of  the 
programming  period,  resulting  also  in  fewer  projects  adopted  by  the  Commission. 
Nevertheless, In terms of EU financing, while their relative contribution of 65% remains 
the  same  over  the  two  periods,  the  projects  assisted  by  JASPERS  and  adopted  by  the 
Commission in  the current  period tend to  be larger in  size, and therefore  they mobilise 
more EU resources (by 4 billion euro) even if fewer in number. 

123  Additional 3 projects for which the advisory assignment had been administratively closed were also subsequently 
adopted  by  the  Commission  until  October  2019.  Overall,  there  are  160  projects  which  had  been  assisted  by 
JASPERS advisory and were subsequently adopted/ approved by the Commission until October 2019. 

120 

 
 
 
 
 
 
 
 
                                                 
The  median  duration  of  JASPERS’  advisory  involvement  in  project  lifecycle  is  of  20 
months, although there is a high variability across assignments by sector. 

In terms of duration of JASPERS’s assignments for technical assistance, we find that the 
median  duration  of  completed  assignments  is  20  months  (and  average  27  months), 
compared  to  an  average  of  16  months  in  period  2007-2013.  We  also  find  that  there  is 
significant variability across assignments in this regard. In Table 5.2.1 we summarise the 
data on duration of completed assignments for technical assistance for major projects by 
sector. 

Table 5.2.1 Duration of advisory assignments for major projects (months) 

Sector 

Number 
assignments 
23 
72 
36 

Median 
duration 
19 
19 
17 

Minimum 
duration 
2 
0.2 
1 

Energy and solid waste 
Rail, air and maritime 
Roads 
Smart development and 
ICT 
Water and wastewater 
All advisory assignments 
Note: Data covers all assignments completed by December 2018 
Source: JASPERS JADE database, December 2018 

23 
24 
178 

1 
8 
0.2 

15 
49 
20 

Maximum 
duration 
60 
76 
129 

95 
94 
129 

At  sector  level,  we  learn  that,  while  for  most  of  the  sectors  assisted  by  JASPERS  the 
median duration is  between 15 and 20 months, the assignments for the sector of Water 
and  wastewater  tend  to  take  much  longer.  High  variability  across  assignments  for  all 
sectors, however, is indicated by the fact that duration across assignments can vary from 
very short (a minimum of 6 days for Rail, air, and maritime) to very long (a maximum of 
129 months for Roads) across sectors.  

In  order  to  understand  better  the  role  of  JASPERS  advisory  in  the  process  of  project 
preparation  we  compared  the  duration  of  assignments  with  the  duration  of  project 
preparation in Member States. For this purpose, we collected SFC2014 data on dates for 
project  preparation  and  for  the  overall  (planned)  duration  of  all  new  major  projects 
submitted to the Commission until October 2019. This data is summarised in Table 5.2.2. 

121 

 
 
 
 
 
 
 
 
 
Sector 

16 
67 
76 

Number 
projects 

Median 
project 
duration 
89 
101 
166 

Median 
project 
preparation 
44 
59 
135 

Table 5.2.2 Lifecycle of new major projects from preparation to completion (in months) 
Median ratio: 
Preparation / Total 
duration 
50% 
58% 
85% 

Energy and solid waste 
Rail, air and maritime 
Roads 
Smart development and 
ICT 
Water and wastewater 
All new projects 
submitted 
69% 
Note: Data covers 210 (assisted and non-assisted) new major projects submitted to 
SFC2014, based on project timetables.  
Source: European Commission, SFC2014, October 2019 

60% 
53% 

67 
117 

40 
64 

125 

210 

82 

26 

25 

Durations reported in Table 5.2.2 are calculated based on the project timetables reported 
in the application to the Commission. These timetables include the start and end dates for 
10 stages of project development124. For the calculation of overall duration of a project 
we considered the earliest and latest dates reported in the project timetable, while for the 
preparatory phase of the project we considered the earliest and latest dates covering all 
core stages for preparation (feasibility study, cost benefit analysis, environmental impact 
assessment and design studies) up to, but not including, tender documentation.  

On this basis we find that, from cradle to completion, major projects can last a very long 
time, with longest total durations for Roads (14 years median value). The data also shows 
that  the  preparation  of  projects  has  different  relative  weights  in  total  duration  across 
sectors. For Roads, for example, the project preparation takes around 85% of the overall 
project  duration,  while  for  Rail,  air,  maritime  and  for  Smart  development  and  ICT  the 
preparatory phase accounts for up to 60% of the overall duration of the project.  

In  42%  of  cases,  JASPERS’  assistance  is  requested  during  the  stage  of  core  project 
preparation,  with  a  resulting  median  coverage  of  16%  of  this  stage  by  advisory  in 
calendar  terms.  For  18%  of  assignments,  JASPERS  is  involved  from  the  very 
beginning of the core project preparation, with a full coverage of this stage in calendar 
terms. 

In this context, it may seem that the time taken by JASPERS’ assignment for assistance 
with  preparation  of  a  project  is  about  a  quarter  of  the  total  preparation  time.  This, 

124  These  stages  include:  1.  feasibility  study/  business  plan,  2.  cost-benefit  analysis,  3.  environmental  impact 
assessment, 4. design studies, 5. preparation of tender documentation, 6. tender procedures, 7. land acquisition, 8. 
development  consent,  9.  construction  phase,  10.  operational  phase.  For  project  implementation  (the  operational 
phase), the timetables include expected completion dates. 

122 

 
 
 
 
 
 
                                                 
however, may not be the case in situations where JASPERS’ support is  requested later 
during  the  preparation  phase,  or  when  the  assignment  extends  also  after  the  core 
preparation is completed. For this reason, we also calculated the overlap between period 
of JASPERS’s assignment and the period for the project preparation as follows.  

As  explained  in  the  section  for  relevance,  for  the  new  major  projects  assisted  by 
JASPERS  advisory  and  submitted  to  the  Commission  we  used  the  data  on  project 
timetables  and  the  calendar  duration  of  assignments  to  identify  three  groups  of 
assignments: Group 1: assignments created from the start of the preparatory stage of the 
project or earlier; Group 2:  assignments created after the preparatory stage has started; 
and  Group  3:  assignments  created  after  the  preparatory  stage  ended.  For  the  first  two 
groups we calculated the overlap in % terms between the duration of the assignment and 
the  duration  of  the  preparatory  stage  as  defined  above.  This  gives  the  coverage  of  the 
preparatory stage by the assignment, with higher values indicating a higher coverage of 
core project preparation by advisory in terms of (calendar) duration. This data is reported 
in Table 5.2.3.  

Table 5.2.3 Overlap between advisory assignment and project preparation 

Group assignments 

Number 
assignments 
Min 
34% 
21 
Group 1 (from start) 
0.10% 
89 
Group 2 (after start) 
Group 3 (after completion) 
0 
10 
Note: Data covers 120 assignments for new major projects assisted by JASPERS and 
submitted to the Commission125.  
Source: European Commission SFC2014, October 2019; JASPERS’ JADE database, 
December 2018 

Coverage preparatory by assignment 
Max  
100% 
95% 
0 

Median  
99% 
16% 
0 

This data indicates that 89 of these assignments were created after the start of the project 
preparation  and,  for  a  majority  of  these  projects,  they  covered  around  16%  of  the 
preparation  time.  Data  also  indicates  that  there  is  significant  variability  across  these 
assignments, as some could last for almost the full period of the core preparatory stage. 
Further, we also found a group of 21 assignments which started from the very beginning 
of  the  preparatory  stage  (or  earlier)  and  a  majority  of  them  lasted  for  full  core 
preparation. Finally, 10 of the assignments analysed had no overlap as they started after 
the  core  preparatory  stage  had  been  completed.  Overall,  we  could  infer  that  around  a 
quarter  of  the  assignments  analysed  were  created  at  early  or  very  early  stages  of  the 
preparatory phase for projects, possibly with higher chances to advise on critical features 
of project design. 

125  There  are  139  assignments  for  advisory  for  new  major  projects  which  were  subsequently  submitted  to  the 
Commission until October 2019. For this analysis we used all projects with complete data both in JADE and in 
SFC. 

123 

 
 
 
 
 
 
                                                 
Involvement  of  JASPERS’  advisory  support  from  the  start  of  project  preparation  is 
positively  correlated  with  a  shorter  duration  of  the  preparation  stage  relative  to  the 
duration of the project lifecycle.  

Next  we  also  analysed  the  correlation  between  the  duration  of  the  stage  of  core 
preparation (relative to total duration of project’s lifecycle) and the stage of involvement 
of  JASPERS’  advisory  in  this  process.  The  duration  of  project  preparation  can  be  a 
function  of  several  co-founding  factors  such  as  projects  features  (size  and  sector)  and 
country specific effects. For this reason, we carried out linear regression models for the 
ratio of the duration of core preparation in total project lifecycle for which we tested two 
main sets of variables for JASPERS’ assistance: 1) JASPERS’ advisory assistance at any 
stage in project preparation relative to support after core preparation or no assistance, and 
2)  JASPERS’  involvement  in  project  preparation  from  start  and  after  start  of  core 
preparation,  relative  to  involvement  after  core  preparation  or  not  at  all.  The  analysis 
covers  79%  of  all  new  major  projects  submitted  to  the  Commission  until  October 
2019126. Table A2.2.4 in Annex 2 includes the results. 

 On this basis, we learn that JASPERS’ advisory support is positively correlated with a 
shorter duration of core preparation relative to project lifecycle only when the support is 
requested  from  the  start  of  the  preparation  stage.  This  effect  is  statistically  significant 
when  controlling  for  project  features  (size  and  sector)  and  country  fixed  effects. 
Moreover, we also find some indication that later involvement of JASPERS’ advisory in 
project  preparation  is  slightly  correlated  with  a  longer  duration  of  project  preparation. 
This effect, however, is statistically significant only at 10% level, and therefore weaker.  

Relative  to  the  previous  period,  there  has  been  a  significant  reduction  (from  82%  to 
46%) in the rate of interruptions of major projects during the approval procedures.  

Next, we look at what happened after advisory was completed and at the extent to which 
projects  assisted  by  JASPERS  advisory  were  sufficiently  prepared  for  the  adoption  / 
approval in the Commission, in comparison with projects not assisted by JASPERS. This 
analysis  covers  all  150  new  major  projects  submitted  to  the  Commission  for  Art  102.2 
during  the  reference  period  (whether  assisted  or  not),  with  a  focus  on  the  projects 
interrupted  in  the  approval  process  in  the  Commission127.  Data  for  assisted  and  not 
assisted projects and rates of interruptions by sector is presented in Table 5.2.4. 

126  These are projects for which we could find data for project timetables and all co-founding factors. 
127  Projects submitted for Art 102.1 are not included since the IQR assessment process is more informal and has no 

comparable interruption procedure.  

124 

 
 
 
 
 
 
 
 
 
                                                 
Table 5.2.4 New major projects interrupted by the Commission by sector and support 
Not assisted by JASPERS 
Assisted by JASPERS 
advisory 
advisory 

Sector 

% interrupted 

Number 
projects 
11 
34 
21 

55% 
38% 
52% 

Energy and solid waste 
Rail, air and maritime 
Roads 
Smart development and 
12 
ICT 
14 
Water and wastewater 
Total 
92 
Note: data covers all 150 new major projects submitted for Art 102.2. 
Source: European Commission SFC2014, October 2019.  

50% 
57% 
48% 

6 
0 
58 

Number 
projects 
2 
6 
44 

% interrupted 

100% 
83% 
30% 

83% 
n.a. 
43% 

Overall, we find that 69 of the 150 new major projects submitted by Member States for 
EU financing were returned for modification or, to a much lesser extent, withdrawn by 
the Member State for quality issues. This amounts to an overall rate of interruptions of 
46%.  Further,  If  we  consider  separately  the  two  groups  of  projects  (assisted  and  not 
assisted by JASPERS advisory), we note that the rate of interruption for projects assisted 
by  JASPERS  advisory  was  48%,  and  it  was  43%  for  the  non-assisted  projects.  As  we 
shall see later, however, the higher rate of interruption for projects assisted by JASPERS 
is  not  attributable  to  JASPERS  advisory  but  rather  to  project  features  and  country 
specific factors.  

By  comparison,  based  on  an  analysis  of  208  major  projects  submitted  (of  which  168 
assisted  by  JASPERS  advisory)  for  2007-2013,  we  learn  that  the  overall  rate  of 
interruption  was  much  higher  (82%  of  all  projects  submitted),  with  the  similar  rates  of 
interruption (82%) for the two groups: assisted and not assisted. Therefore, relative to the 
previous  period,  there  has  been  a  significant  reduction  in  the  rates  of  interruption  of 
major projects. Nevertheless, given that the extent of this reduction appears to be roughly 
in  a  comparable  range  for  the  two  groups  of  projects  (assisted  and  not  assisted),  we 
interpret  this  improvement  between  the  two  periods  as  primarily  due  to  the  overall 
changes  in  the  regulatory  and  administrative  set-up  for  the  assessment  and  approval  of 
projects instituted by the Commission for 2014-2020128.  

As regards the distribution by sector in the current period, we note that, compared with 
the non-assisted ones, projects assisted by JASPERS advisory tend to have lower rates of 
interruption for all sectors, except for Roads. For the sector of Water and wastewater, all 
projects  submitted  had  been  assisted  by  JASPERS  advisory,  and  therefore  the 

128  During the current period, there has also been a significant reorganization of activities overseeing the approval of 
major projects in DG REGIO, notably with the creation of a dedicated horizontal unit coordinating these activities 
at the service level.  

125 

 
 
 
 
 
                                                 
comparison is not feasible. Finally, within the group of assisted projects  across sectors, 
we learn that the rate of interruption tends to be higher for Energy and solid waste and for 
Water and wastewater.  

Finally,  when  comparing  rates  of  interruption  for  assisted  and  not  assisted  projects  at 
country level, we find that the projects assisted by JASPERS advisory tend to have zero 
or lower rates of interruption for 5 countries, similar rates of interruption for 3 countries, 
and a higher rate for only one country. Table A4.6 in Annex 4 includes the details.  

The  two most  frequent  topics  for  EC  observations  in interruption letters  include  cost 
benefit analysis  and feasibility study, with higher frequencies in non-assisted relative 
to assisted projects. 

In  order  to  understand  better  why  projects  are  still  interrupted  when  submitted  to  the 
Commission,  we  analysed  also  the  reasons  for  interruptions.  For  this  analysis,  we 
collected structured data on Commission’s observations made in  the observation letters 
by  topic  for  all  69  projects  interrupted  and/or  withdrawn  until  Oct  2019.  This  data  is 
presented in Table 5.2.5. 

Table 5.2.5 Topics of EC observations for major projects submitted for Art 102.2 

Interrupted projects 

Frequency 
in assisted 

Number 
projects 

% in total 
interrupted 

Topic EC observations 

(1) 

% in all 
for Art. 
102_2 

(2) 
31% 
27% 
25% 
24% 
23% 

Frequency 
in not 
assisted 
(3) 
72% 
72% 
60% 
44% 
48% 

(4) 
66% 
50% 
52% 
57% 
52% 

(5) 
47 
40 
38 
36 
35 

(6) 
68% 
58% 
55% 
52% 
51% 

48% 

21% 

Cost-Benefit analysis 
Feasibility study 
Environmental issues 
Project description 
Total eligible costs 
Risk and Sensitivity 
analysis 
Financial plan 
Indicators 
Capacity beneficiary 
Project timetable 
State aid 
Project not in OP 
Not a major project 
Note: Data covers all 69 projects interrupted, of which 44 assisted by JASPERS.  
Source: European Commission’s interruption letters and SFC2014 for projects submitted 
until October 2019. 

19% 
18% 
9% 
8% 
7% 
1% 
1% 

32% 
28% 
16% 
8% 
12% 
4% 
4% 

45% 
45% 
20% 
23% 
18% 
2% 
0% 

41% 
39% 
19% 
17% 
16% 
3% 
1% 

28 
27 
13 
12 
11 
2 
1 

43% 

45% 

31 

126 

 
 
 
 
 
 
Table  5.2.5  reads  as  follows.  Column  (2)  includes  the  share  of  projects  with 
Commission’s  observations  for  the  respective  topic  in  all  projects  submitted  for  Art. 
102.2  during  the  period  analysed.  For  Cost-Benefit  analysis,  for  instance,  31%  of  the 
projects  submitted  to  the  Commission  for  Art.  102.2  had  observations  related  to  this 
topic.  Column  (6),  on  the  other  hand,  refers  only  to  project  interrupted  and  gives  the 
share  of  projects  with  observations  on  the  respective  topic  in  all  projects  interrupted. 
Therefore,  of  all  69  projects  interrupted,  68%  had  observations  related  to  Cost-Benefit 
Analysis.  Finally,  the  intermediate  columns  (3)  and  (4)  indicate  the  frequency  of 
observations related to a given topin in the assisted, respectively non-assisted, projects.  

On this basis, we learn that around a third of projects submitted to the Commission for 
Art.  102.2  were  interrupted  and  had  observations  related  to  cost-benefit  analysis  and 
feasibility  study  in  the  interruption  letters  (column  2  in  Table  5.2.5).  Other  frequent 
reasons  for  Commission’s  observations  include  also  environmental  issues,  project 
description,  total  eligible  costs,  risk  and  sensitivity  analysis  (all  above  20%)  and,  to  a 
lesser extent, the financial plan and indicators in the project. The frequency of issues for 
cost  benefit  analysis,  feasibility  study  and  environmental  issues  tends  to  be  rather  high 
within the group of interrupted projects (as indicated in column 6).  

When we compare topics of interruption for assisted and not assisted projects (columns 3 
and  4  in  Table  5.2.5)  we  learn  that  for  three  of  these  topics  (cost  benefit  analysis, 
feasibility  studies  and  environmental  issues)  the  frequency  of  their  presence  in 
observation  letters  tends  to  be  higher  for  the  non-assisted  projects.  For  the  remaining 
topics,  however,  the  frequency  varies  between  assisted  and  not  assisted,  with  several 
topics (project description, total eligible cost, financial plan etc.) being more frequent for 
the assisted projects. This may also be due to the fact that these  issues are identified in 
the  project  applications,  which  sometimes  are  submitted  by  Member  States  with 
incomplete information.  

By  comparison,  for  period  2007-2013,  highest  rates  of  interruptions  were  related  to 
environmental  issues,  financial  plan,  and  project  design  (all  above  30%  in  projects 
submitted and analysed). Table A3.9 in Annex 3 presents these details.  When comparing 
the  rates  by  topic  for  the  two  periods,  we  note  reductions  in  the  frequency  of 
Commission’s  observations  especially  for 
the  financial  plan, 
environmental  issues,  and  implementation  timetable  and  structures,  thus  indicating  a 
learning  process  in  the  Member  States  in  this  regard  between  the  two  periods.  This 
comparison, however, is rather inferential as the data for the two periods is not perfectly 
comparable due to slightly different classifications of topics for interruption129.   

topics  related 

to 

129  Also, the regulatory requirements for major projects were further developed in the current period, relative to 2007-

2013.  

127 

 
 
 
 
 
 
 
                                                 
Relative  to  assisted  projects,  the  projects  not  assisted  by  JASPERS  advisory  tend  to 
have  a  higher  number  of  critical  issues  identified  in  the  PSA  reports  issued  by 
JASPERS  IQR.  Nevertheless,  there  are  no  significant  differences  between  the  two 
groups of projects in terms of the number of observations raised by the Commission in 
interruption letters and the duration of interruptions.  

Next,  for  a  further  understanding  of  the  maturity  of  projects  submitted  to  the 
Commission,  as  proxies  for  the  quality  of  projects  interrupted,  we  also  calculated  the 
number of observation topics and the number of observations raised by the Commission 
in the observations letter per project. Our conjecture is that fewer topics or issues raised 
would indicate a higher maturity of project preparation at adoption130. On this basis we 
find  that  the  median  number  of  observation  topics  per  project  is  4  (out  of  13),  and  the 
median  number  of  observations  raised  per  interrupted  project  is  6,  with  no  significant 
differences between assisted and non-assisted projects for both medians values.  

Further, we analysed this issue also based on the number of critical issues identified by 
JASPERS  IQR  in  the  PSA  reports.  On  this  basis,  we  learn  that  the  median  number  of 
critical  issues  identified  by  JASPERS  IQR  for  the  non-assisted  projects  is  4,  while  the 
median  number  of  critical  issues  identified  for  the  projects  assisted  by  JASPERS 
advisory is 1, and the difference in medians is statistically significant.  

Another possible indication of the maturity of projects arriving to the Commission is also 
the  time  it  takes  to  resolve  the  issues  raised  in  the  Commission’s  assessment.  For  all 
project interrupted and subsequently adopted, we calculated the time difference between 
the date when the project was adopted and the date of first interruption (or withdrawal). 
For the 47 projects adopted after having been interrupted over the reference  period, we 
find  that  the  median  total  duration  for  interruptions  is  4.4  months  (with  values  ranging 
between  1  to  22  months),  and  that  there  are  no  significant  differences  in  the  median 
values  for  the  assisted  and  non-assisted  projects.  Therefore,  interrupted  projects  in  the 
two  groups  (assisted  and  not  assisted)  tend  to  have  similar  median  durations  of 
interruption.  

The  effectiveness  of  JASPERS’  advisory  to  signal  issues  with  project  quality  later 
identified during the approval process varies across topics. It is highest for cost benefit 
analysis  and  feasibility  study,  but  much  lower  for  other  project  aspects  such  as 
indicators, project description, or project timetable.  

The  evidence  presented  so  far  indicates  that,  for  a  third  of  the  projects,  JASPERS 
advisory assistance does not seem to be sufficient to ensure that the projects advised and 
submitted  to  the  Commission  are  of  sufficient  quality  to  warrant  adoption  without 

130  These  proxy  measures  are,  however,  imperfect  as  the  observations  included  in  interruption  letters  refer  to  both 

critical and non-critical issues with project quality.  

128 

 
 
 
 
 
 
 
                                                 
interruption. There could be several reasons why this is observed. First, it is possible that 
projects  are  interrupted  for  issues  which  had  not  been  under  the  scope  of  JASPERS’ 
advisory  assignment.  Second,  it  is  also  possible  that  even  if  JASPERS  advised  the 
beneficiary  on  specific  topics,  its  comments  were  not  taken  on  board  by  the  time  the 
project  was  submitted  to  the  Commission.  Third,  it  is  possible  that  JASPERS  advisory 
reviewed  the  respective  topics  but  did  not  raise  the  issues  subsequently  raised  by  the 
Commission in its observation letters during the adoption procedure. 

In  order  to  distinguish  among  these  three  cases,  for  the  projects  assisted  by  JASPERS 
advisory  and  interrupted  by  the  Commission,  we  compared  the  topics  on  which  the 
Commission raised observations with the contribution and comments documented in the 
action completion notes of the corresponding advisory assignments. This analysis covers 
37 major projects interrupted by the Commission during the procedure for Art 102.2 and 
which  had  been  supported  by  JASPERS  advisory.  These  projects  represent  84%  of  all 
new  major  projects  advised  by  JASPERS  for  project  preparation  and  subsequently 
interrupted during the adoption procedure131. 

From  the  ACNs  issued  by  JASPERS  advisory  upon  completion  of  an  assignment  we 
collected  structured  data  on  the  type  of  JASPERS  assistance  to  the  project  as  follows. 
Following the structure of the ACN, we coded the assistance by topic as: 1) Significant 
issue, if JASPERS experts raised at least one significant issue for the respective topic, 2) 
Comment, if JASPERS experts included suggestions for improvements in the comments 
for  the  respective  topic,  3)  Contribution,  if  no  significant  issue  or  suggestions  for 
improvements  were  found,  but  there  was  an  explicit  mention  by  the  experts  that  they 
contributed to the issue during the assignment, 4) Positive assessment, when we found a 
positive assessment of the topic but  no evidence for 1) to  3), and 5) Review, when the 
experts provided a description for the topic without an assessment. This data is reported 
in  Table  5.2.6  for  the  projects  that  had  observations  from  the  Commission  on  the 
respective topic during interruptions.   

Table  5.2.6  JASPER’  assessments  in  ACNs  for  interrupted  projects  by  topic  of 
interruption 

131  ACNs provided by Member States together with the application for major projects for Art. 102(2) are available in 

SFC2014 for 88% of all projects assisted by JASPERS advisory over the reference period.  

129 

 
 
 
 
 
                                                 
Note: Data covers 37 projects assisted by JASPERS, interrupted by the Commission, and 
for which we found the ACN provided by the Member State in SFC2014 together with the 
project documentation.  
Source:  JASPERS’  ACNs,  European  Commission’s  SFC2014  and  observation  letters, 
October 2019. 

Table  5.2.6  reads  as  follows.  For  example,  for  cost  benefit  analysis,  in  column  (7)  we 
learn  that  22  projects  had  observations  from  the  Commission  on  issues  of  cost  benefit 
analysis,  and  they  had  also  been  assisted  by  JASPERS  advisory  on  the  topic.  For  8  of 
these 22 projects JASPERS’ experts raised significant issues with cost benefit analysis in 
the ACN, for 11 of them JASPERS included comments for improvements without raising 
significant  issues,  and  for  the  remaining  3  the  experts  stated  they  contributed  to  the 
analysis of the projects.  

On this basis, we calculate that JASPERS advisory had signalled issues in this regard in 
the ACN either by raising significant issues or by including suggestions for improvement 
for 86% of projects with observations on cost benefit analysis from the Commission. In 
terms  only  of  significant  issues  raised  by  JASPERS  advisory,  however,  the  proportion 
reduces to 36%. The same reasoning applies to all remaining topics.  

From this data we infer the following. First, we note that for the first 5 topics in the table 
(from  cost  benefit  analysis  to  capacity  of  beneficiary)  JASPERS’  experts  signalled 
changes  needed  in  projects  for  more  than  57%  of  the  projects  that  later  received 
observations  from  the  Commission  on  the  respective  topics  (column  8).  This  suggests 
that,  even  if  experts  included  significant  issues  and  comments  in  their  reports,  in  some 
cases the issues were not resolved by the Member State before submitting the project to 
the Commission.  

Nevertheless,  based  on  the  proportion  of  significant  issues  signalled  by  JASPERS 
(column  9),  we  also  infer  that,  in  some  cases,  the  experts’  message  on  issues  which 
required further  attention might  not have been sufficiently strong  for the beneficiary to 
understand that changes were needed. This is based on the fact that many of the projects 

130 

Significant issueCommentContribution, no commentPositive assessmentReviewTotal(1)(2)(3)(4)(5)(6)(7)(8)(9)Cost benefit analysis81132286%36%Feasibility study5761867%28%State aid43757%57%Risk and sensitivity analyis35151457%21%Capacity beneficiary43757%57%Financial plan52641741%29%Climate issues25651839%11%Costs333531735%18%Project timetable2133933%22%Project description235531828%11%Indicators13671724%6%Environmental issues122140%0%TopicJASPERS advisory assessment in ACN (only projects interrupted for the topic)% ACNs signalling comments and significant issue% ACNs signalling significant issues 
 
 
 
 
 
 
assisted  by  JASPERS  were  interrupted  for  topics  on  which  the  ACNs  did  not  raise 
significant issues but included comments.  

And third, there are also projects which were interrupted and received observations from 
the  Commission,  but  for  which  the  ACN  included  a  positive  assessment,  with  no 
suggestions  for  improvement.  This  is  the  case,  for  instance,  for  the  assessment  of 
environmental  issues,  indicators,  and  project  description  for  which  two  thirds  of  the 
projects  had  no  indication  in  the  ACN  that  some  aspects  might  have  needed 
improvements.  

JASPERS’  advisory  has  a  positive  effect  on  the  likelihood  that  projects  are  not 
interrupted  during  the  approval  process  for  Art  102.2.  The  effect  is  statistically 
significant when controlling for project features and country specific factors.  

Finally,  as  regards  interruptions,  we  also  analysed  the  probability  that  a  project  is  not 
interrupted as a function of the stage in which JASPERS’ support is requested for project 
preparation and other co-founding factors such as project size, sector and country specific 
factors. For JASPERS’ advisory support, we tested a variable with the following values: 
1 – if JASPERS advisory assisted the project preparation from the beginning or at least 
during the first half of the core preparation period (early involvement), 2 – if JASPERS 
advisory  assisted  the  projects  in  the  second  half  of  the  core  preparation  period  (later 
involvement),  and  3  –  if  JASPERS  assisted  the  project  after  the  end  of  the  core 
preparation  period  (late  involvement).  As  explained  earlier,  core  preparation  period  is 
defined  as  including  all  stages  until  preparation  of  tender  documentation  and  it  is 
calculated  based  on  project  timetables  and  period  of  assignment.  The  ordered  logistic 
regression models run for this analysis are presented in Table A2.2.6 in Annex 2.  

Covering 95% of all new major projects submitted to the Commission for Art 102.2 until 
October  2019,  this  analysis  indicates  that  major  projects  are  more  likely  not  to  be 
interrupted during the approval / adoption process if they had been advised by JASPERS 
advisory during the preparation of the projects. This effect is higher and more statistically 
significant  when  JASPERS  provides  assistance  from  the  beginning  of  the  preparation 
period or at least during its first half, when compared with later involvement in project 
preparation.  

JASPERS’  advisory  support  is  positively  correlated  with  progress  in  the  financial 
implementation of major projects, but only for projects in which JASPERS intervenes 
early during the period of core project preparation and for younger, most recent new 
projects prepared.  

Next,  we  assessed  the  effectiveness  of  the  advisory  function  also  in  relation  with  the 
progress  in  project  implementation  after  its  adoption/  approval.  For  this  purpose  we 

131 

 
 
 
 
 
 
 
 
collected  the  data  on  financial  and  physical  progress  for  166  new  major  projects 
submitted  to  the  Commission132,  as  reported  by  Member  States  in  the  annual 
implementation  reports  for  2019  (or  for  2018,  if  2019  missing).    First,  the  data  on 
physical implementation is reported in Tables 5.2.7.  

132  These projects represent 93% of all 178 new major projects adopted/ approved by the Commission until October 

2019.  

132 

 
 
 
 
 
                                                 
Table 5.2.7 Physical implementation of new major projects adopted 

Not assisted  
Number  % in total 

JASPERS TA assisted 
Progress implementation  
(reported by Managing Authority) 
% in total 
Number 
25% 
27 
Advanced construction 
12% 
13 
Completed/ In operation 
56% 
61 
Construction 
3% 
3 
Design 
5% 
5 
Procurement 
Total 
100% 
109 
Note:  Data  covers  166  projects  submitted  to  the  Commission  and  with  implementation 
data. 
Source:  European  Commission,  SFC2014,  AIRs  2018-2019,  downloaded  September 
2020. 

47% 
21% 
30% 
2% 
0% 
100% 

27 
12 
17 
1 
0 
57 

The large majority (81%) of projects assisted by JASPERS advisory were in construction 
or  advanced  construction  by  end  2019,  and  12%  were  already  completed  and/or  in 
operation.  By  comparison,  for  the  non-assisted  projects,  77%  were  in  construction  or 
advanced  construction,  and  21%  completed.  A  minority  were  still  in  design  and 
procurement phases for both groups.  

The  financial  progress  in  implementation  until  end  2019  is  based  on  the  percentage  of 
certified  eligible  expenditure  declared  by  Member  States  at  project  level,  and  it  is 
reported in Table 5.2.8. 

Table 5.2.8 Financial implementation of new major projects adopted 

Assisted 

Not assisted  
Number  % in total 

Number 
20 
18 
37 
17 
13 

% in total 
18% 
17% 
34% 
16% 
12% 

Financial progress  
(% certified expenditure) 
1_No progress 0% 
2_Progress up to 10% 
3_Progress in 10% -50% 
4_Progress in 50%-80% 
5_Progress in 80%-100% 
6_Progress more than 
100% 
Total 
Note:  Data  covers  166  projects  submitted  to  the  Commission  and  with  implementation 
data. 
Source:  European  Commission,  SFC2014,  AIRs  2018-2019,  downloaded  September 
2020 

12% 
4% 
18% 
11% 
35% 

7 
2 
10 
6 
20 

4% 
100% 

21% 
100% 

4 
109 

12 
57 

In terms of certified eligible expenditure, 32% of projects assisted by JASPERS advisory 
had  certified  at  least  50%  of  the  eligible  expenditure  of  the  project  by  end  2019, 
compared with  67% of the non-assisted projects.  This  indicates that, over the reference 

133 

 
 
 
 
 
 
the  non-assisted  projects  are  relatively  more  advanced  with  financial 

period, 
implementation relative to assisted projects.   

Progress  in  implementation,  however,  can  be  the  result  of  several  co-founding  factors 
such as project size, sector, year of start of implementation, and country specific factors. 
For  this  reason,  for  the  same  sample  of  166  projects,  we  analysed  whether  JASPERS 
advisory  had  an effect  on the likelihood of higher financial and/or physical  progress  in 
implementation  while  controlling  for  projects  features  and  country  fixed  effects.  For 
JASPERS  advisory  we  tested  three  variables:  1)  JASPERS  support  at  any  stage  during 
core preparation of projects, 2) JASPERS support at early stages of core preparation of 
projects,  and  3)  JASPERS  support  at  early  stages  of  core  preparation  of  young  new 
projects,  where  young  projects  are  defined  as  projects  which  started  implementation  at 
earliest  in  2017.  The  method  used  is  ordered  logistic  regression  and  the  results  for  the 
models of financial progress in implementation are presented in Table A2.2.8 in Annex 
2.  

Based on this analysis we find that there is a positive correlation of JASPERS advisory 
with  the  financial  implementation  of  new  major  projects  adopted  only  when  JASPERS 
assistance  is  requested  during  early  stages  of  project  preparation,  and  only  for  young 
projects. We do not find such an effect for JASPERS assistance in general and not even 
only  for  early  assistance  without  accounting  for  the  projects’  start  of  implementation. 
One reason for this result is the fact that 71% of the assignments with early involvement 
of JASPERS are for young projects starting implementation by 2017. For the other two 
groups (i.e. with later involvement of JASPERS in preparation or with no involvement at 
all), the frequency of such projects is much lower (at most 35%). Overall we find that, for 
almost  a  majority  (47%)  of  the  more  recent  new  projects  analysed,  Member  States 
requested  JASPERS  advisory  at  early  stages  of  preparation  and  this  is  positively 
correlated with the subsequent financial implementation of these projects after approval 
for EU funds.   

As regards physical implementation, the effect of JASPERS advisory is not statistically 
significant but we believe that this is due to the fact the classification of implementation 
stages self-reported by the Member States is not sufficiently granular to enable a similar 
analysis  by  stage  of  JASPERS  advisory  and  effective  maturity  of  physical 
implementation133.    

Therefore,  on  this  basis,  we  conclude  that  the  evidence  collected  indicates  that,  in 
addition to being correlated with a higher probability of projects not being interrupted by 
the Commission  (therefore indicating better project  quality), JASPERS advisory is  also 

133  For  physical  progress  in  implementation  none  of  the  three  variables  for  JASPERS  advisory  are  statistically 

significant. Therefore, the corresponding model results are not reported.  

134 

 
 
 
 
 
 
                                                 
positively  correlated  with  the  financial  implementation  of  the  assisted  projects  if  the 
support was requested early during project preparation, and for younger projects. 

135 

 
 
 
 
Technical advisory for non-major projects 

Overall  effectiveness  of  JASPERS’  advisory  for  non-major  projects  cannot  be 
established  as  data  on  projects’  progress  with  approval  and  implementation  is  not 
available.  Preliminary  evidence  for  a  third  of  completed  assignments,  however, 
indicates an approval rate of at most 60%.  

The possibility to evaluate the effectiveness of JASPERS’ advisory work for non-major 
projects is much more limited due to lack of data. As JASPERS does not collect any data 
on  what  happens  with  the  projects  once  the  advisory  work  is  completed,  we  requested 
this information from 6 Member States sampled for the evaluation, but only two of them 
responded.  Therefore,  except  for  the  data  on  duration  of  assignments  which  covers  the 
full  portfolio,  all  other  data  reported  in  this  sub-section  should  be  interpreted  as  rather 
illustrative. 

The  2014-2020  portfolio  includes  140  advisory  assignments  for  non-major  projects,  of 
which  47%  were  completed  by  December  2018.  For  the  completed  assignments,  we 
could collect data for 74% them, i.e. from 49 JASPERS’ ACNs issued upon completion 
of the advisory work. On this basis, we learnt that these projects have a cumulated total 
cost  of  close  to  2  billion  euro,  but  we  cannot  tell  to  what  extent  these  resources  were 
mobilised  through  the  programmes  as  we  do  not  know  which  of  these  projects  were 
eventually approved by Managing Authorities, and with which costs. For example, from 
the first country which responded to our request  for information, we learnt that 58% of 
the non-major projects with advisory work completed by December 2018 were approved 
by the Managing Authorities until autumn 2019. For the second country which responded 
to our request, one out of two projects with completed advisory was approved. These two 
countries together, however, represent only 30% of the completed assignments for non-
major projects.  

By comparison, in the period 2007-2013, the number of advisory assignments completed 
by JASPERS during the first five years of the period was 101, representing 76% of the 
133 advisory assignments completed for non-major projects over the whole period. Also 
for  the  previous  period,  however,  we  do  not  know  how  many  of  these  projects  were 
eventually implemented in the operational programmes for the period. 

The median duration of an advisory assignment for non-major projects is 14 months, 
although there is significant variability across assignments.  

As  regards  the  duration  of  advisory  assignments  for  non-major  projects,  based  on  data 
from JASPERS’ database, we learn that the median duration across all assignments is 14 
months,  with  higher  durations  for  Roads  and,  to  a  lesser  extent,  for  Water  and 
wastewater. Table 5.2.9 presents the details.  

136 

 
 
 
 
 
 
 
 
Table 5.2.9 Duration advisory assignments for non-major projects (in months) 

Number 
assignments 
8 
5 
3 

Duration assignment non-major projects 
Max 
45 
26 
59 

Median 
13 
13 
41 

Min 
4 
4 
5 

Sector 

Energy and solid waste 
Rail, air and maritime 
Roads 
Smart development and 
ICT 
Water and wastewater 
All completed assignments 
Note: data covers all completed assignments 
Source: JASPERS’ JADE database, December 2018 

22 
28 
66 

10 
19 
14 

58 
72 
72 

1 
4 
1 

Nevertheless,  this  data  also  indicates  that  there  is  a  significant  variability  across 
assignments, with duration ranging from one month to 72 months. This is consistent with 
the earlier finding that, depending on the scope of the assignment, JASPERS’ support is 
requested by Member States at various stages during project preparation.  

For  the  overall  duration  of  project  preparation,  however,  we  could  only  present 
illustrative  examples  as  relevant  data  across  all  assignments  is  not  available.  Based  on 
our review of the 49 ACNs for completed assignments for non-major projects, we found 
data  on  projects’  timetables  only  for  6  of  them.  As  for  major  projects  earlier,  also  for 
these  non-major  projects  we  calculated  the  duration  of  core  project  preparation  and 
classified them by the stage at which JASPERS advisory is requested. We found that the 
median duration of core project preparation for these 6 projects ranged between 13 and 
99 months, and for 4 of them JASPERS’ assistance started from the beginning of project 
preparation, or even earlier. For the remaining two projects,  JASPERS advisory started 
during project preparation.  

Further,  we  calculated  also  the  overlap  in  calendar  time  between  JASPERS’  advisory 
assignment  and  core  project  preparation.  For  the  4  assignments  with  early  support,  the 
record  in  JADE  indicates  that  all  assignments  lasted  for  the  full  duration  of  the  core 
preparation  of  the  projects,  extending  also  to  the  subsequent  stages  for  preparation  of 
tenders and for construction134. For the remaining two assignments, for which JASPERS’ 
assistance  was  requested  after  the  start  of  project  preparation,  the  median  overlap  in 
calendar time with project preparation was 33%. 

134  We  found also that, according to JADE, they started earlier than the project preparation specified in the project 
timetable. This can happen, for instance, if the project was assisted also for the previous programming period.  

137 

 
 
 
 
 
 
 
                                                 
Most frequent issues raised by JASPERS advisory during project preparation relate to 
procurement  and  implementation,  cost  benefit  analysis,  and  risk  and  sensitivity 
analysis. 

In terms of the content of JASPERS’ assistance for non-major projects, as with the major 
projects, we collected structured data on the type of contribution reflected in the ACNs of 
the  49  completed  assignments  analysed.  Data  collection  followed  the  same  method  of 
classifying JASPERS assistance by topic into 5 groups explained earlier:  1) significant 
issues  raised  in  the  ACN,  2)  comments  with  suggestions  for  improvement,  3) 
contribution mentioned by JASPERS experts in ACN, 4) positive assessment when 1) to 
3) do not apply, and 5) review of the topic with no explicit assessment and when 1) to 4) 
do not apply. This data is summarised in Table 5.2.10. 

138 

 
 
 
 
 
 
Table 5.2.10 JASPERS advisory assessment in ACNs by topic of interruption 

Note: Data covers 49 non-major projects assisted by JASPERS  
Source: JASPERS’ ACNs 

From this analysis we learn that JASPERS advisory made comments with suggestions for 
improvement and, to a much lesser extent, raised significant issues for almost half of the 
non-major  projects  analysed  for  issues  related  to  procurement  and  implementation.  For 
cost benefit analysis, risk and sensitivity analysis and demand analysis, comments were 
included  for  around  40%  of  the  projects.  For  the  remaining  topics,  the  extent  of 
contribution  varies,  also  due  to  the  fact  that  the  scope  of  these  assignments  was  more 
limited  for  those  projects  where  JASPERS  advisory  issued  a  short  ACN  (as  explained 
earlier in the section on relevance). 

Nevertheless,  when compared with  the  analysis for major projects,  for the advisory  for 
non-major projects the evidence on effectiveness cannot be conclusive as no sufficiently 
representative data is available on the success of these projects in the process of approval 
and, if applicable, implementation.  

139 

Significant issueCommentContribution, no commentPositive assessmentReviewTotal(1)(2)(3)(4)(5)(6)(7)(8)(9)Procurement and implementation11611433549%3%Cost benefit analysis1813554144%0%Risk and sensitivity analyis1221412941%0%Demand analysis1516634038%0%Technical solution2812253826%5%Financial plan26411113424%6%Option analysis1825924520%2%Projects costs34711103520%9%Capacity beneficiary522153315%0%Project description24102264414%5%Indicators2671513%0%State aid432143213%0%JASPERS advisory assessment in ACNs for non-major projects% ACNs signalling comments and significant issue% ACNs signalling significant issuesTopic 
 
 
 
 
 
 
Review function 

The newly introduced JASPERS’ review function for the ESIF mandate has the objective 
to ensure consistency in the assessment of major projects submitted for EU financing and 
reduce the time of assessment and approval of these projects.  

As  explain  earlier,  there  are  two  ways  in  which  the  assessment  of  major  projects  takes 
place :  1)  a  complete  assessment  carried  out  directly  by  JASPERS  IQR  (the  procedure 
based  on  Art  102.1),  and  2)  the  assessment  carried  out  by  the  Commission  with  the 
assistance of JASPERS IQR through the PSA service (the procedure based on Art 102.2). 
As  regards  the  approval/adoption  of  major  projects,  the  procedure  based  on  Art  102.1 
requires Member States to notify the Commission of a new major project by submitting 
the project application and an IQR report from JASPER IQR confirming that the project 
is of sufficient quality for approval. For the procedure based on Art 102.2, the Member 
State  submits  the  full  project  documentation  to  the  Commission,  the  Commission 
assesses  the  quality  of  the  project  and,  if  it  considers  that  the  project  is  sufficiently 
mature,  the  Commission  issues  a  decision  to  adopt  the  project.  If  the  quality  is  not 
considered sufficient, the project is interrupted.  

For  both  these  procedures,  we  calculated  the  time  it  takes  to  assess  and  subsequently 
approve  a  project  and  considered  the  contribution  of  JASPERS  IQR  in  the  process  as 
follows. 

The median total duration of the process of assessment and approval of major projects 
notified  for  Art  102.1  is  7  months.  Of  this,  the  IQR  assessment  carried  out  by 
JASPERS  IQR,  including  also  the  response  for  the  Member  States,  has  a  median 
duration of 5 months.  

For the procedure based on Art 102.1, we calculated the assessment time by considering 
the difference between the date when the IQR assessment was requested by the Member 
State and the date when the final IQR was issued by JASPERS IQR. The approval time 
for  these  projects  is  calculated  as  the  calendar  time  between  the  date  when  the  project 
application  and  the  IQR  report  are  notified  to  the  Commission  and  the  date  when  the 
project is approved. Table 5.2.11 below summarizes the results by sector. 

Based on the analysis of 50 major projects approved based on Art 102.1, we find that the 
median duration of IQR assessment carried out by JASPERS IQR division is of 5 months 
across  all  projects,  varying  between  1  and  15  months  at  project  level135.  This  duration 
includes also the time needed by the Member State to reply to the first assessment sent by 

135  Starting with the Omnibus Regulation adopted in July 2018, the regulatory requirement is for the IQR assessment 

to be completed within 6 months.  

140 

 
 
 
 
 
 
 
 
                                                 
Jaspers, and therefore it represents the time elapsed from the submission of IQR until the 
IQR report is issued.  

Table  5.2.11  Assessment  and  approval  time  for  major  projects  –  Art  102.1  (IQR)  (in 
months) 

Source : JASPERS’ JADE database (December 2018) and European Commission 
SFC2014 (October 2019) 

At  sector  level,  we  find  that  the  median  duration  for  the  IQR  assessment  is  longer  for 
Rail, air and maritime and for Roads (6 months for both).  

Further in the process, once a JASPERS IQR issues a positive IQR assessment and the 
project is notified to the Commission, then it takes 3 months for its approval136.  

We  also  calculated  the  overall  duration  of  this  process  from  the  request  for  the  IQR 
assessment  to  JASPERS  IQR  and  until  the  project  approval  in  the  Commission  (thus 
including  also  the  time  taken  by  Member  State  to  submit  the  project  application  after 
IQR) and found that it took around 7 months overall, with a mininum of 4 months and a 
maximum of 18 months. As shown already, longer durations in this process are primarily 
driven by the duration of the IQR assessment, as the Commission takes at most 3 months 
for approval for all projects submitted for Art 102.1.  

Furthermore,  when  testing  the  duration  of  the  IQR  assessment  for  projects  assisted  by 
JASPERS advisory relative to projects not assisted, we find that the median duration is 
higher  for  projects  not  assisted  by  JASPERS  advisory  (6  months)  relative  to  projects 
assisted (4 months). 

The median total duration of the process of assessment and approval of major projects 
submitted  for  Art  102.2  is  3  months  for  projects  not  interrupted,  and  6  months  for 
projects  interrupted.  In  this  process,  JASPERS  IQR  is  highly  effective  in  delivering 
timely PSA reports.  

For the procedure based on Art 102.2, as assessment and approval are carried out by the 
Commission  in  one  procedure,  we  calculated  the  calendar  time  it  takes  for  the  overall 

136  The  period  of  3  months  is  the  regulatory  deadline  for  tacit  approval.  Quicker  approval  is  possible  only  in 

exceptional situations. 

141 

MedianMaxMinMedianMaxMinEnergy and solid waste3161333Rail, air and maritime226151331Roads116141331Smart development and ICT5573331Water and wastewater9241331All projects505151331Duration approval ECSectorNumber projectsDuration IQR assessment 
 
 
 
 
 
 
 
 
 
                                                 
process, from the date when the Member State submits the project documentation to the 
Commission  for  the  first  time  and  until  the  date  when  the  project  is  adopted.  This 
calendar  time  includes  also  the  interruption  periods  for  projects  found  of  insufficient 
quality by the Commission in its assessment. This data is summarised in Table 5.2.12. 

Table 5.2.12 Duration for procedure Art 102.2 until adoption (in months) 

Projects by interruption 

Duration Art 102.2 

Max  Min 

Number 
projects 
78 
47 
125 

Median 
3 
6 
3.3 

Projects not interrupted 
Projects interrupted 
All projects for Art. 102.2 
Source : JASPERS’ JADE (December 2018) and European Commission SFC2014 (Oct 
2019) 

1 
1.3 
1 

4 
23 
23 

For this procedure, we find that projects which are not interrupted need around 3 months 
for the process  of assessment  and  adoption in  the Commission.  The projects  which are 
interrupted,  on  the  other  hand,  have  a  median  duration  of  6  months,  with  the  longest 
interruption  leading  to  up  23  months  for  the  overall  duration  in  one  case.  When 
aggregated  for  all  projects  submitted  for  Art  102.2  the  median  duration  is  3.3  months. 
Furthermore, we also tested whether the process duration differs for projects assisted by 
JASPERS advisory relative to non-assisted projects, but found no statistically significant 
difference. 

Across sectors, when analysing the median duration of the full process for Art 102.2 by 
sector, we find a higher duration for projects in Smart development and ICT (5 months) 
relative to all other sectors (3 months).  

In  this  context,  for  both  types  of  projects  (interrupted  and  not  interrupted),  the 
Commission  requests  JASPERS  IQR  to  issue  a  PSA  report  including  an  assessment  of 
the quality of the project. We find that, in line with the agreement with the Commission, 
JASPERS IQR takes around one calendar month (and slightly more during peak periods 
or holidays) to issue the report. Furthermore, once the report is sent to the Commission, 
JASPERS IQR also participates in a screening meeting with the Commission services – a 
meeting  where  the  quality  issues  for  the  project  are  discussed.    For  the  projects 
interrupted,  at  the  request  of  the  Commission,  JASPERS  IQR  issues  also  updated  PSA 
reports  addressing  the  replies  from  the  Member  State  to  Commission’s  observations. 
Overall, we assess that JASPERS IQR is highly effective in this process. 

Relative to the period 2007-2013, there has been a significant reduction in the overall 
time  taken  with  the  assessment  and  approval  of  major  projects.  This  reduction  is 
highest for the procedure based on Art 102.2 (with a reduction of 50%) and lower for 
the procedure based on Art 102.1 (with a 13% reduction).  

142 

 
 
 
 
 
 
 
By comparison, for the previous period, the REGIO evaluation of JASPERS carried out 
in 2012 informs us that the average duration of approval for major project was around 9 
months,  and  the  minimum  duration  recorded  was  6  months.  Therefore,  for  the  current 
period,  we  find  significant  gains  in  terms  of  shortening  the  time  of  assessment  and 
approval  of  major  projects  achieved  for  the  procedure  based  on  Art  102.2    (with  a 
reduction of 50% in the average duration of the process)137 and, to a lesser extent, for the 
procedure  based  on  Art  102.1  (with  a  reduction  of  13%  in  the  average  duration  of  the 
overall process)138. In our assessment, for both these procedures, JASPERS IQR plays an 
instrumental role in ensuring the assessment of the project quality on time in the current 
period. 

The effectiveness of JASPERS IQR to signal issues of project quality in the PSA report 
is relatively high for most topics, and especially for issues related to project timetable, 
risk and sensitivity analysis, feasibility study, and cost and benefit analysis. 

Next,  we  also  looked  at  effectiveness  of  JASPERS  IQR  in  terms  of  signalling  quality 
issues in the projects for which it issues PSA reports. For this analysis, we collected data 
from PSA reports for 66 major projects submitted to the Commission and interrupted for 
quality issues. For these projects we compared the issues raised by JASPERS in the PSA 
reports  with  the  Commission’s  observations  in  interruption  letters  sent  to  the  Member 
States. This data is reported in Table 5.2.13. 

Table  5.2.13  Projects  with  quality  issues  raised  in  PSAs  and  interrupted  by  the 
Commission 

Note : Data covers 66 major projects interrupted and with PSA reports. 
Source : European Commission observation letters and JASPERS’ PSA reports. 

Table 5.2.13 reads as follows. First, we identified the number of major projects for which 
the  Commission  included  observations  on  a  given  topic  in  interruption  letters.  For 
example, as regards the project timetable, we identified 12 interrupted projects for which 
the Commission made observations related to the topic. Next, we looked at for how many 
of these projects JASPERS IQR had raised critical (column 2) or non-critical (column 3) 

137  Average duration for Art 102(2) is 4.5 months. 
138  Average duration for Art 102(1) is 7.8 months. 

143 

Critical issuesNon critical issuesAll issues(1)(2)(3)(4)(5)(6)Project timetable 2121212100%Risk and sensitivity analysis1724303197%Feasibility study2426384095%Cost benefit analysis3239444794%Capacity of beneficiary49121392%Project description1318263672%Financial plan617202871%Total eligible costs1717223563%Projects with issues raised in PSAProjects with EC observations on the topicTopic% projects with issues signalled in PSA 
 
 
 
 
 
 
                                                 
issues on the same topic. For project timetable, JASPERS IQR raised critical issues in 2 
projects,  and  non-critical  issues  in  all  of  these  projects.  The  percent  ratio  calculated  in 
column (6) gives an overall proxy for the effectiveness of JASPERS IQR as it indicates 
how  often  the  PSA  report  helps  the  Commission  identify  issues  of  project  quality  by 
topic139.  For  project  timetable,  for  instance,  we  find  that,  for  all  projects  for  which  the 
Commission  raised  observations  on  the  topic,  the  PSA  reports  also  reflected  issues  of 
project quality in this regard. The same reasoning applies to all topics. 

On  this  basis,  we  conclude  that  the  effectiveness  of  JASPERS  IQR  to  signal  issues  of 
project  quality  in  the  PSA  report  is  relatively  high  for  most  topics,  and  especially  for 
issues related to project timetable, risk and sensitivity analysis, feasibility study, and cost 
and benefit analysis. For a majority of projects, the topics on which JASPERS IQR raises 
critical and non critical issues are also reflected in Commission’s observation letters.  

Capacity building and horizontal and strategic advice 

As  mentioned  in  the  previous  section,  JASPERS’  portfolio  of  assignments  for  capacity 
building and horizontal and strategic advice for 2014-2020 includes 220 assignments, of 
which  75%  are  for  horizontal  and  strategic  advice,  and  the  remaining  25%  are  for 
capacity building.  

Starting with capacity building activities, according to JASPERS’ Quality Manual140, the 
objective  with  these  assignments  is  to  provide  trainings,  workshops,  and  online  forums 
on  topics  relevant  for  the  preparation  and  implementation  of  projects,  as  well  as  to 
facilitate the dissemination of guides and studies. In terms of approach, the trainings and 
events  organised  for  capacity  building  can  be  either  multi-country  or  tailored  to  the 
specific needs of a country.  

As  explained  in  the  section  for  relevance,  the  majority  of  assignments  for  capacity 
building  are  multi-country  assignments,  with  the  most  frequent  trainings/  workshops 
organized  for  topics  related  to  climate  change  and  environment,  state  aid,  and  cost-
benefit analysis. 

So  far,  while  a  large  majority  activities  for  capacity  building  were  completed  and 
achieved their objectives for delivering trainings for the Member States, such activities 
have not proven sufficient to trigger a visible ripple effect on the technical capacity of 
Member  States  to  address  effectively  issues  such  as  cost  benefit  analysis,  climate 
change and environment in project preparation.  

139  We  do  not  distinguish  between  critical  and  non-critical  issues  in  the  overall  assessment  since  Commission’s 

observations usually cover both types of issues.  

140  JASPERS’ Quality Manual (2018). 

144 

 
 
 
 
 
 
 
 
                                                 
When we compare the topics of the capacity building activities completed until end 2018 
with  the  quality  gaps  identified  in  project  preparation  during  the  advisory  work  with 
JASPERS and during the assessment of major projects  in  the Commission,  we identify 
two main issues. First, despite a relatively frequent focus of capacity building activities 
on  issues  of  cost  benefit  analysis,  climate  change  and  environment,  from  the 
Commission’s  interruption  letters  we  learn  that  these  issues  are  still  among  the  most 
frequent  topics  for  observations  to  major  projects  interrupted.  Furthermore,  from  the 
analysis of the ACNs issued by JASPERS advisory, we also see that JASPERS provides 
support on cost benefit analysis to 95% of the projects assisted, and on climate issues to 
60%  of  these  projects.  Therefore,  on  the  this  basis,  we  conclude  that  the  activities  for 
capacity  building  organised  on  these  topics  have  not  yet  proven  sufficient  to  trigger  a 
visible ripple effect on building an autonomous technical capacity for Member States to 
address sufficiently these issues in project preparation.  

Second,  both  from  the  documentation  of  advisory  work  and  from  the  Commission’s 
assessments of major projects, we learn that there is still a significant number of projects 
in  need  of  advisory  support  also  for  other  topics  such  as  feasibility  study  and  risk  and 
sensitivity  analysis,  while  the  activities  organised  for  capacity  building  on  these  topics 
are not that frequent141.  

Furthermore,  at  country  level,  as  noted  earlier,  we  find  that  countries’  reliance  on 
JASPERS’  advisory  assistance  for  project  preparation  has  not  been  reduced  relative  to 
the previous period, therefore suggesting modest capacity building for project preparation 
in between the two periods. We find that, of 14 Member States supported by JASPERS 
advisory  for  project  preparation  during  the  previous  period,  only  two  Member  States 
(Poland  and  Czechia)  have  reduced  their  relative  reliance  on  JASPERS’  assistance  for 
project  preparation.  In  addition,  if  quality  of  project  preparation  is  an  indication  of 
technical capacity, then significant remaining gaps in capacity can be inferred also from 
the  rates  of  project  interruptions  for  most  countries142.  For  9  out  of  13  Member  States 
submitting  major  projects  for  Art  102.2,  the  rate  of  interruption  is  at  least  50%,  as 
summarized  in  Table  A4.6  in  Annex  4.  Therefore,  also  the  evidence  across  countries 
tends  to  suggest  that  activities  for  capacity  building  do  not  seem  to  have  had  a  visible 
ripple effect on countries’ technical capacity to prepare projects of higher quality without 
relying on extensive advisory support.  

141  An informal argument among stakeholders consulted suggested that capacity building is better strengthened with 
hands-on  experience  while  working  directly  with  JASPERS  on  the  projects  rather  than  through  trainings  and 
workshops.  While,  in  principle, we  see  the  merit  of  this  argument,  we  also  do not  see  any  evidence  that  this  is 
reflected  in  actual  sustainably  better  preparation  of  projects  across  sectors  and  countries.  This  may  be  due  to 
higher turnover of the personal in national authorities, but also due to the fact that often JASPERS interacts with 
the  consultants  contracted  for  the  preparation of  project documentation  and therefore  the  learning  effect  for  the 
staff of national authorities is limited. 

142  Due  to  lack  of  data,  however,  we  could  not  really  establish  whether  the  persons  who  participated  in  capacity 

building events were also the ones working on project preparation. 

145 

 
 
 
 
 
                                                 
JASPERS  has  the  proven  potential  to  promote  more  strategic  and  innovative 
approaches for technical capacity, but such approaches were not yet mainstreamed at 
the portfolio level during the period analysed.   

Nevertheless, while at portfolio level we could not identify evidence of effectiveness of 
activities  for  capacity  building,  based  on  in-depth  analysis  at  assignment  level  we 
identified several  cases in which the approach or the method of delivery of the activity 
may  have  a  significant  potential  to  contribute  more  visibly  to  overall  effectiveness,  if 
mainstreamed strategically across Member States, topics of high interest, and time.  

As  a  first  example  of  good  practice,  we  identified  the  strategic  planning  for  capacity 
building adopted by Croatia in 2018 which, in our view, has the potential to increase the 
effectiveness  of  such  activities  as  they  target  the  most  pressing  gaps  in  expertise  for 
project  identification  and  preparation  in  the  country.  This  experience  with  JASPERS 
support in Croatia is presented briefly in Text Box 5.2.1. 

Text Box 5.2.1 Strategic approach of capacity building in Croatia 

By  end  December  2018,  JASPERS  had  included  30  assignments  for  horizontal  and 
strategic  advice  and  capacity  building  in  its  portfolio  for  Croatia.  Half  of  these 
assignments,  however,  were  subsequently  administratively  closed.  In  2017  the  State 
Audit  Office  of  Republic  of  Croatia  carried  out  a  performance  audit  on  the  impact  of 
JASPERS initiative on the projects prepared for application for EU co-financing143, and 
concluded  that  the  added  value  of  horizontal  and  capacity  building  support  was  rather 
limited due to deficient planning and absence of a clear assessment of needs. Following 
the  recommendations  from  the  audit,  beginning  with  2018  and  in  cooperation  with  the 
European  Commission  and  JASPERS  NCC,  the  Managing  Authority  established  a 
capacity building plan based on two main sources : 1) an assessment of needs carried out 
by the Managing Authority with the intermediate bodies; and 2) gap analyses carried out 
by JASPERS’ experts for 6 sectors of importance for the implementation of EU funds in 
the country. This plan is being implemented during the current period.   

Further for Croatia, as part of this strategic approach at country level, more recently we 
also  learnt  that  the  country  is  working  with  the  European  Commission  and  with 
JASPERS  to  establish  a  national  IQR  service  for  review  and  quality  assessment  of 
projects prepared for EU funds. 

A  second  example  illustrates  the  “train  the  trainers”  mode  of  delivery  for  capacity 
building trainings. First recommended for JASPERS in the EIB evaluation carried out in 

143  Republic  of  Croatia  State  Audit  Office  (2017)  –  Impact  of  the  JASPERS  programme  on  submitted  project 
proposals by the Republic of Croatia for co-financing from EU funds. Report of Performance Audit, Zagreb, May 
2017. 

146 

 
 
 
 
 
 
 
 
                                                 
2010,  the  “train  the  trainers”  approach  focuses the training on local  trainers who  could 
subsequently  disseminate  the  know  how  to  larger  audiences  and  repeatedly  over  time 
within  and  across  countries.  This  approach  was  applied  by  JASPERS  for  some  of  the 
assignments of capacity building in the current period, but it has not been mainstreamed 
at  portfolio  level.  In  our  view  this  approach  also  has  the  potential  to  enhance  the 
effectiveness  of  capacity  building  activities  as  it  can  help  strengthen  the  local 
competences  and  reach  much  larger  audiences.  An  example  of  an  assignment  applying 
the “train the trainers” approach in Romania is presented in Text Box 5.2.2. 

Text Box 5.2.2 “Train the trainers” mode of delivery in Romania 

For the programming period 2007-2013, JASPERS provided support for the development 
of  a  professional  training  on  environmental  impact  assessment  addressed  to  national 
officials. The “train the trainers” approach adopted for this assignment entailed a 2-days 
training  of  a  nucleus  of  around  30  trainers  who  subsequently  provided  15  training 
sessions to additional 400 trainees from regional and local environmental authorities. The 
training  was  positively  assessed  by  beneficiaries  as  it  ensured  a  high  outreach  and 
dissemination of knowledge on the topic.  

Finally, as a third example we identify as good practice illustrates the economies of scale 
that could be achieved by coordinated training across countries on a specific EU priority 
for the implementation  of the funds. The  example refers to  nine training  workshops  on 
EU  Sustainable  Urban  Mobility  Planning  (SUMP)  organised  by  JASPERS  in  eight 
countries. The Text Box 5.2.3 provides the details. 

Text  Box  5.2.3  Coordinated  local  workshops  on  EU  Sustainable  Urban  Mobility 
Planning (SUMP) 

During  the  period  September  2016  –  February  2017,  at  the  request  of  Commission 
services DG REGIO and DG MOVE, JASPERS organized SUMP trainings for local and 
regional  authorities  in  eight  countries:  Bulgaria,  Cyprus,  Greece,  Latvia,  Lithuania, 
Poland,  Slovenia,  and  Spain.  Initially,  JASPERS  surveyed  the  Member  States  on  their 
interest  in  capacity  building  training  on  this  topic  and,  when  realizing  that  there  was  a 
broader interest  across countries, it decided to  merge  all trainings in  one  assignment in 
order  to  ensure  consistency  and  efficiency  of  delivery.  In  this  context,  the  training 
sessions  were  tailored  to  countries’  needs  by  applying,  for  instance,  the  “train  the 
trainers”  approach  in  Slovenia,  while  in  Cyprus  and  Greece  the  focus  was  more  on 
awareness  raising  for  decision  makers.  The  one-day  workshops  were  provided  by  a 
private consultancy company selected through the EIB’s Advisory Services Framework 
contract. 

For  the  horizontal  and  strategic  support,  based  on  the  analysis  of  sampled 
assignments, there are indications that JASPERS’ support helps compensate for lower 

147 

 
 
 
 
 
 
 
  
 
 
local  technical  capacity  for  guiding  and  supervising  tasks  on  specialised  topics  of 
policy relevance.  

Next, for the horizontal and strategic support, JASPERS’ Quality Manual informs us that 
the  main  objective  for  these  assignments  is  to  support  national  strategies  on  specific 
issues, river basin flood management plans or integrated urban development. This type of 
support is usually not project related, and it can be either sector specific or cross-sectoral. 
As presented earlier in the section for relevance, however, we find that these assignments 
are  highly  diverse  in  their  specific  objectives  (ranging  from  horizontal  tasks  to  support 
for  national  strategies  and  plans),  and  therefore  less  amenable  for  an  assessment  of 
overall  effectiveness.  Instead,  for  this  type  of  support,  we  sampled  10  country  specific 
assignments and analysed them in more detail. 

Of  the  10  assignments  sampled,  7  were  completed  and  3  administratively  closed  by 
December 2018. From the completed assignments, we selected three assignments which 
can  help  illustrate  the  nature  and  methods  of  JASPERS’  work  in  this  context.  The 
following presentation of these examples is based on the action completion notes issued 
by JASPERS for the respective assignments.  

First, in an assignment for Bulgaria, JASPERS was requested by the national authority to 
provide a peer review of several regional Water and Wastewater Master Plans prepared 
in  the  context  of  a  larger  Master  Planning  project  funded  by  the  World  Bank  in  the 
country. The national authority had contracted consultancy services for the preparation of 
the regional master plans, and JASPERS’s task was to review this work and convey its 
conclusions  to  the  national  authority.  Upon  the  revision  of  the  first  master  plan, 
JASPERS developed a model that was subsequently replicated by the consultants for all 
the remaining plans. The assignment was carried out during 2012-2014, and it resulted in 
several  guidance  notes  issued  by  JASPERS  when  reviewing  the  regional  plans. 
Stakeholders’  interviewed  for  this  assignment  rated  positively  JASPERS’  work, 
emphasizing  the  value  added  of  JASPERS’  advice,  especially  as  it  relied  on  its 
experience in collaborating with the European Commission. 

A  second  example  is  an  assignment  from  Romania  for  supporting  the  planning  of  EU 
investments  in  smart  grids.  For  this  assignment,  JASPERS  was  requested  to  ensure 
guidance and supervision for the  consultancy service contracted to  design a framework 
and methodology for the assessment of investments in smart grids. The outcomes of this 
work  include  a  complete  methodology  for  project  assessment  for  smart  grids,  with  a 
definition  of  eligibility  criteria  for  projects  applying  for  EU  funds  in  this  domain,  and 
mechanisms  to  support  the  implementation  of  these  investments.  The  assignment  was 
carried  out  during  December  2013  –  February  2014,  and  JASPERS’  input  was 
appreciated especially as regarded advice concerning the alignment of the methodology 
with EU requirements and standards.  

148 

 
 
 
 
 
 
Finally,  a  third  example  coming  from  Malta  illustrates  JASPERS’  assistance  for  the 
implementation of the national strategy for Research and Innovation (R&I) launched the 
context  of  the  ex-ante  conditionality  for  smart  specialisation  for  EU  cohesion  policy. 
JASPERS’  task  was  to  assist  in  refining  the  sub-criteria  for  the  appraisal  of  projects 
applying  for  EU  funding  for  R&I.  As  part  of  this  assignment,  JASPERS  provided  the 
Managing Authority with  examples  of evaluation criteria and sub-criteria for R&I state 
aid schemes from other Member States with a similar R&I profile, and it helped re-orient 
the criteria towards project deliverables and outcomes based on the introduction of result 
indicators. This work was carried out during the period October 2015 – December 2017, 
and  it  was  appreciated  by  the  beneficiary  especially  as  it  also  helped  ensure  alignment 
with the EU state aid legislation. 

In  sum,  based  on  the  sampled  assignments,  we  conclude  that  JASPERS  succeeded  in 
achieving the objectives at least for 70% of these assignments144, and its support helped 
compensate  for  lower  local  technical  capacity  for  guiding  and  supervision  tasks  on 
specialised  topics  of  relevance  for  EU  cohesion  policy.  Moreover,  based  on  interviews 
with the stakeholders involved in these assignments, we learnt that JASPERS is valued 
by  beneficiaries  especially  for  its  experience  acquired  across  Member  States  with  the 
planning and  assessment of projects  for the policy, and  for its advice on  aligning these 
investments with EU legislation and standards. 

CEF Mandate 

Technical advisory for CEF projects 

As concerns CEF, as presented in the previous section on relevance, we learnt that 79% 
(23 out of 29) of the projects supported by JASPERS during its first pilot mandate with 
CEF were successful in the CEF competitions for the cohesion envelope.  

For  around  70%  of  the  CEF  assignments  for  works  completed  until  end  2018 
JASPERS’  technical  assistance  was  requested  primarily  around  the  time  of  project 
application.  Some  of  these  projects,  however,  had  been  assisted  by  JASPERS  for 
project preparation during the previous programming period (for the ESIF mandate).  

For the analysis of effectiveness, as regards the assignments for advising the preparation 
of  these  projects,  we  first  looked  at  their  duration  and  calculated  the  time  it  takes  to 
complete an advisory assignment for a CEF project for 21 assignments completed until 
December 2018 for the first pilot CEF1.1. On this basis, we found that, while the median 
duration is around 3 months, the assignments vary significantly in this respect. This data 
is reported by sector in Table 5.2.14, and by type of project in Table 5.2.15.  

144  The issue of administratively closed assignments is further analysed in the next section on efficiency. 

149 

 
 
 
 
 
 
 
                                                 
Table 5.2.14 Duration of JASPERS’ advisory for CEF projects by sector 

Sector 

Number 
assignments 
18 
3 
21 

Duration assignment (months) 
Max  
Median  
20 
2 
Rail 
14 
5 
Roads 
Total 
20 
3 
Note: Data covers CEF assignments started in 2015 (earliest) and with completion date 
in JADE. 
Source: JASPERS’ JADE database, December 2018. 

Min 
1 
4 
1 

Table 5.2.15 Duration of JASPERS’ advisory for CEF projects by type  

Type project 

Number 
assignments 
13 
6 
2 
21 

Duration assignment (months) 
Max  
Median  
15 
2 
20 
8 
3 
3 
20 
3 

Min 
1 
2 
3 
1 

Works 
Mixed 
Studies 
Total 
Note: Data covers CEF assignments started in 2015 (earliest) and with completion date 
in JADE. 
Source: JASPERS’ JADE database, December 2018. 

From  this  data  we  learn  that  the  advisory  assignments  for  CEF  tend  to  last  longer  for 
projects  in  Roads,  relative  to  Rail.  From  the  classification  by  type  of  assignments, 
however,  we  see  that  much  of  the  variability  is  generated  by  the  difference  between 
support  for  studies  and  support  for  works.  The  median  duration  is  highest  for 
assignments  combining  support  for  works  and  studies,  and  it  is  lower  for  assignments 
focused either on studies or on works. Nevertheless, compared with studies, we note also 
that the assignments mixing studies and works have a wide distribution of duration.  

Next,  in  order  to  understand  better  JASPERS’  work  in  the  CEF  context,  we  analysed 
more  in-depth  10  sampled  assignments  for  the  CEF1.1  pilot,  representing  one  third  of 
JASPERS’ portfolio for this mandate. Table A4.10 in Annex 4 presents the details.  

First, in terms of types of projects in the sample, 9 of these projects are for invesments 
(works in CEF terminology) and one for a feasibility study. As regards the projects  for 
works, two thirds proved successful in the CEF competition. And so did also the project 
for the study.  

Based on the analysis of the documentation for these assignments we found that, except 
for one project in Slovakia, for all the other projects for works JASPERS’ assistance was 
requested for review of CEF application and project documentation, and usually around 

150 

 
 
 
 
 
 
 
the period of submission and assessment of the application145. Moreover, for half of these 
assignments,  when  interviewed,  stakeholders  stated  that  JASPERS’  support  was  asked 
rather late in the application process and therefore had limited scope for added value (this 
was especially the case for the unsuccessful projects).  And given that around 70% of the 
19  CEF  assignments  for  works  in  JASPERS’  portfolio  completed  until  end  December 
2018 have a duration of at most 5 months over the period for submission and assessment 
of CEF applications, we infer that for a majority of assignments for CEF1.1 it is likely 
that  there  was  little  to  no  scope  for  JASPERS’  advisory  to  have  an  impact  on  project 
quality in terms of project technical design due to late involvement in the process.    

Nevertheless,  while  the  finding  above  refers  to  the  technical  assistance    provided  by 
JASPERS  in  the  context  of  the  CEF  mandate,  we  also  learnt  that  7  of  the  investment 
projects  assisted by JASPERS at  application stage for CEF competitions  had also  been 
assisted  by  JASPERS  technical  service  for  the  preparation  of  these  projects  during  the 
course of the previous programming period in the context of the ESIF mandate.  

In 40% of cases, the studies financed by CEF and assisted by JASPERS advisory based 
on  the  CEF1  pilot  mandate  were  feasibility  studies  prepared  for  investment  projects 
intended for the CEF financial support.  

As regards the technical advisory support provided by JASPERS for projects for studies, 
we  found  that  some  of  them  refer  to  feasibility  studies  prepared  for  CEF  investment 
projects. More specifically, this is the case in 6 out of all the 15 assignments (completed 
and  ongoing)  for  studies  or  mixed  projects  (i.e.  works  and  studies)  prepared  with  the 
intention to be submitted for CEF financing.  

Finally, from the interviews with the stakeholders, we learnt that, given the scope of the 
assignments, they appreciated JASPERS’ input as regards suggestions for improvement 
of  project  application,  and  especially  in  terms  of  alignment  with  EU  legislation  and 
standards.  

IPA Mandate 

Technical advisory for IPA projects 

As  mentioned  in  the  previous  section,  JASPERS’  portfolio  for  IPA  included  102 
assignments by end December 2018. Of these, 38% had been completed, 50% were still 
ongoing, and the rest were administratively closed. Table 5.2.16 presents these details.  

145  For two of these projects we also found that JASPERS had assisted them in the past for project preparation when 

they were being planned for the ESIF mandate.  

151 

 
 
 
 
 
 
 
 
 
 
                                                 
Table 5.2.16 JASPERS’ IPA assignments by status 

Source: JASPERS’ JADE database, December 2018 

For the IPA mandate, 18% of assignments resulted in IPA projects approved for EU-
funding.  

The advisory work for project preparation constitutes the bulk of this mandate, with  73 
assignments completed, closed or on-going for advisory by end December. Nevertheless, 
when matching the data on JASPERS’ portfolio with data on approved IPA projects from 
the Commission,  we find that only 13 of these portfolio assignments could  be matched 
with  17  approved  IPA  projects146.  This  implies  that,  so  far,  only  18%  of  all  advisory 
assignments  (completed  or  ongoing)147  in  JASPERS’  portfolio  resulted  in  IPA  projects 
approved.  

As  regards  the  submission  of  the  assisted  projects  for  EU  funding,  although  it  is 
informative as regards the contribution of JASPERS’ services to the mobilization of EU 
funds,  it  cannot  be  interpreted  as  a  close  proxy  for  the  effectiveness  of  JASPERS’ 
technical  assistance  due  several  factors  as  follows.  First,  it  is  possible  that,  at  the  time 
when  the  feasibility  study  is  carried  out,  the  national  authorities  reach  the  conclusions 
that current conditions for the implementation are not optimal and therefore the project is 
postponed.  Second,  even  if  the  project  is  feasible,  it  is  yet  to  be  included  in  an  IPA 
programme  and  apply  for  EU  funding.  Third,  JASPERS  explained  that  there  were  also 
situations  when  the  national  authorities  eventually  decided  to  fund  the  project  from 
national sources only. And fourth, JASPERS explained also that there could be a time lag 
between the moment when the project is proposed for EU funding and the time when the 
decision is made.   

Next  we  calculated  the  duration  of  the  completed  advisory  assignments  for  project 
preparation. Table 5.2.17 presents the details by sector.  

146  For the IPA mandate, one assignment can cover several projects, and one project can be covered by more than one 

assignments.  

147  Most of the assignments matched with approved IPA projects appear with the status “ongoing” in JADE by end 

December 2018.  

152 

Type assignmentCompletedOngoingAdministratively closedTotal% in totalAdvisory preparation projects283787372%Advisory implementation projects25077%Capacity building20022%Horizontal and strategic support7942020%Total395112102100%% in total38%50%12%100% 
 
 
 
 
 
 
 
 
 
                                                 
Table 5.2.17 Duration of assignments for project preparation 

Sector 

Number 
assignments 

Duration assignment 
(months) 
Max 

Median 

Min 

12 

14 

Energy and solid 
waste 
Rail, air and maritime 
Roads 
Water and wastewater 
All assignments 
Note: Data covers completed advisory assignments. 
Source: JASPERS’ JADE data, December 2018 

1 
1 
12 
28 

8 
28 
19 
18 

55 

8 
28 
57 
57 

3 

8 
28 
15 
3 

We find that, while the median duration of an advisory assignment for the preparation of 
IPA projects is 18 months, some of these assignments could continue for as long as 57 
months. The sectors with longest median durations are Roads (28 months) and Water and 
wastewater (19 months).  

From the section on relevance, however, we recall that these assignments are not only for 
investment  projects,  but  they  can  also  assist  national  authorities  in  guiding  and 
supervising contracts for technical assistance. Nevertheless, since data in JADE does not 
inform  on  the  type  of  assignment  from  this  perspective,  we  cannot  distinguish  further 
what type of assignments required more time in this regard.  

Based on an analysis of sampled assignments, there is indication that the effectiveness 
of JASPERS’ advisory support is limited in cases when the requests for support arrive 
late in the project preparation cycle.  

In order to understand better the work carried out by JASPERS for advisory assignments 
for  project  preparation  for  IPA,  we  sampled  6  such  assignments  for  further  analysis148. 
All  the  projects  assisted  were  for  investments  in  a  new  road,  and  in  waste,  water  and 
wastewater management systems. Their total costs range from 4 to 45 million euro. Two 
of these projects were approved for IPA financing, with the remaining 4 were either not 
submitted (2 projects) or not approved due to cuts in IPA financing.  

For these projects we learn primarily that JASPERS advisory can make a difference for 
project quality when its support is requested early in the process of project preparation. 
This  was  the  case  for  2  of  the  assisted  projects.  For  other  3  assignments  completed149, 
JASPERS’s support was requested at final stages of project preparation and, despite its 

148  There are 36 assignments completed and administratively closed for advisory for project preparation for the IPA 

mandate. Therefore, the sampled assignments represent 17% of the total.  

149  The  6th  assignment  was  administratively  closed  as  the  beneficiary  did  not  make  any  progress  with  project 

preparation.  

153 

 
 
 
 
 
 
 
                                                 
assessment of significant outstanding issues with project quality, no substantial changes 
could be made any longer.  

Therefore,  we  interpret  this  evidence  as  suggesting  that,  at  least  for  the  sampled 
assignments,  the  effectiveness  of  JASPERS  support  proved  limited  for  a  majority  of 
cases,  although  this  is  not  attributable  to  JASPERS  itself  to  the  extent  that  the  main 
reasons  were  late  request  for  advisory  support  and  reductions  in  IPA  financing  for 
projects. Under these circumstances, however, in terms of customer satisfaction, we also 
found that the stakeholders generally appreciated working with JASPERS and valued its 
advice  especially  as  regards  the  clarification  of  EU  requirements  and  alignment  of 
applications with EU standards.  

IPA horizontal and strategic support 

IPA beneficiaries appreciate JASPERS’ horizontal and strategic support especially as 
it helps raise awareness and familiarize national staff with respect to the management 
and appraisal of EU funded projects.  

As  regards  the  horizontal  and  strategic  support  for  the  IPA  mandate,  of  the  20 
assignments  in  the  portfolio,  80%  had  been  completed  or  were  still  ongoing  in 
Montenegro, North Macedonia and Serbia by December 2018. Half of these assignments 
were  created  for  workshops  and  advisory  support  for  cost  benefit  analysis  and  for 
assistance  with  national  strategies  and  master  plans.  The  remaining  assignments  had  a 
variety of objectives such as preparation of project pipeline, analysis and advice at sector 
level, preparation for the upcoming programming period (2014-2020) etc. 

In terms of distribution by sector, 10 of these assignments are for Water and wastewater, 
and 6 for Rail, air and maritime. As for the duration of assignments completed, it ranges 
between  8  and  22  months.  Further  analysis  of  the  effectiveness  of  these  assignments, 
however, is not feasible due to lack of data. 

Nevertheless, while we could not gather sufficient evidence for an overall assessment of 
the  effectiveness  of  this  type  of  more  horizontal  support,  we  include  some  examples 
analysed in more detail by way of illustration as follows.  

First, we looked at a sampled assignment from Serbia, where JASPERS provided support 
for  the  review  of  terms  of  reference  for  the  programme  on  energy  efficiency  in  public 
buildings.  The  assignment  was  part  of  JASPERS’  mandate  of  support  for  the 
prioritisation and preparation of projects to be financed under IPA 2014-2020 in Serbia. 
When  interviewed,  the  national  authority  using  JASPERS’  assistance  stated  that  they 
valued  highly  this  support  and  especially  JASPERS’  advice  as  regards  alignment  with 
EU requirements and standards.  

154 

 
 
 
 
 
 
 
 
 
A  second  example  refers  to  two  workshops  organised  in  Montenegro  on  project 
management  and  cost  benefit  analysis.  For  these  workshops,  part  of  a  larger  plan  for 
capacity  building  in  the  country,  we  learnt  that  the  beneficiaries  found  this  type  of 
support useful as it helped raise awareness and familiarize national staff with respect to 
the management and appraisal of EU funded projects.  

Stakeholders consultation 

In the online consultation, the stakeholders were asked also for the opinion on the extent 
to which JASPERS had an effect at project level and on horizontal issues. The replies to 
this question are summarized in Tables A1.10 and A1.11 in Annex 1.  

A large majority of stakeholders  consider that JASPERS had a significant, and even 
decisive,  effect  on  project  technical  specifications  and  its  documentation.  They  also 
rated  highly  JASPERS’  assistance  for  projects’  alignment  with  EU  and  national 
policies, and for faster project preparation and approval.  

At project level, more than 80% of stakeholders considered that JASPERS had a decisive 
or  significant  effect  on  project  technical  specifications  and  for  improving  project 
documentation. Other aspects appreciated in this regard by  the majority of stakeholders 
include also JASPERS’ effects on project alignment with EU and national policies, and 
on faster project preparation and approval.  

On  other  project  related  aspects,  however,  stakeholders’  opinions  on  the  extent  of 
JASPERS’  effects  are  more  heterogeneous.  Around  half  of  the  stakeholders  perceived 
significant  or  decisive  effects  of  JASPERS  on  helping  achieve  cost  savings  at  project 
level, or on improving project implementation. Still for implementation, more than 50% 
of respondents perceived JASPERS’ effects as limited or none. 

As regards effects on horizontal aspects, around 60% of stakeholders perceived decisive 
or significant effects of JASPERS’ support on increased administrative capacity, and on 
improved and faster development of national strategies and plans.   

5.3 

EFFICIENCY 

For  the  criterion  of  efficiency  we  analysed  the  planning  process  for  JASPERS’ 
assignments as reflected by country plans in order to understand the extent to which these 
activities are planned and implemented in a timely manner. Further, we also explored the 
work intensity per assignment, calculated relative to the duration of assignments, and its 
differentiation  by  type  of  assignment.  This  section  includes  also  an  analysis  of  the 
developments over time of the financial and human resources for JASPERS’ initiative.  

155 

 
 
 
 
 
 
 
 
 
 
The section is organised in two main parts. The first part focuses on the analysis of the 
planning process and the implementation of assignments over time, while the second part 
includes the analysis of financial and human resources over time. The main findings are 
reported in Text Box 5.3.1. 

Text Box 5.3.1 Main findings for efficiency of JASPERS’ activities during 2014-2018 

PLANNING AND MANAGEMENT OF ASSIGNMENTS 

1)  Planning:  The  monitoring  system  and  country  plans  for  JASPERS’  support  have 
improved over time, but some work still remains to be done as regards consistency of 
data included in the country plans.  

2)  Planning: A majority of assignments could not meet the initially expected completion 
dates, and for 80% of them the expected duration had to be extended for significant 
periods of time. 

3)  Planning: In a third of the countries supported by JASPERS significant proportions 
of assignments are administratively closed instead of completed due to a combination 
of  factors.  These  assignments  tend  to  be  protracted  for  long  periods  of  time  before 
being eventually closed.  

4)  Planning:  The  work  intensity  per  assignment  varies  significantly  by  type  of 
assignment,  with  highest  median  work  intensity  (60%)  for  the  PSA  support  for  the 
Commission.  Lowest  work  intensity  is  observed  especially  for  long  assignments 
which are administratively closed. 

5)  Planning: Overall, there is scope for a more structured approach of the planning and 
monitoring  of  assignments,  with  well-defined  and  targeted  objectives,  clear  and 
realistic  timelines,  and  milestones  that  enable  assessing  the  progress  over  time 
during implementation. 

HUMAN AND FINANCIAL RESOURCES 

6)  The  total  budget  for  JASPERS’  ESIF  mandate  for  the  period  2014-2018  was  175 
million euro, with the Commission’s average contribution of 81% of the total costs. 
The ESIF mandate represents 97% of the total resources allocated for the initiative.  

7)  Relative  to  the  previous  period,  JASPERS  has  developed  significantly  its  pool  of 
human  resources,  with  the  number  of  professional  and  support  staff  increasing  by 
55% relative to 2013.  

8)  Preliminary evidence on developments over time indicates a need for further in-depth 
reflection on possibilities to optimise allocation of work effort and rationalise costs. 

For each of these evaluation findings the supporting evidence is as follows.  

156 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Planning and management of assignments 

For  the  analysis  of  the  planning  and  management  of  assignments  we  screened  the 
quartely country plans for three sampled countries: Croatia, Poland and Romania. These 
three  countries  together  represent  49%  of  all  relevant  assignments  in  JASPERS’ 
portfolio150.  

Country plans are rolling documents in which the Member States and JASPERS agree to 
include  assignments  planned  for  JASPERS’  support  over  time  as  needs  arise.  They 
include  assignments  for  technical  advisory  for  projects,  and  activities  for  capacity 
building  and  strategic  and  horizontal  support.  These  documents  are  updated  regularly, 
and  JASPERS  issues  quarterly  updated  versions  of  country  plans  for  all  countries 
supported. For the current analysis, we analysed 33 quarterly country plans available for 
the three sampled countries over the period 2014-2019151. The main results presented in 
the following paragraphs are summarised in Table A4.11 in Annex 4.  

The monitoring system and country plans for JASPERS’ support have improved over 
time,  but  some  work  still  remains  to  be  done  as  regards  consistency  of  data  in  the 
country plans.  

In a first instance, we collected all assignments included in the country plans over time 
across  all  available  plans  over  the  reference  period.  In  this  way  we  learn,  for  example, 
that  Croatia  has  included  153  assignments  overall  over  the  period  2014-2019.  For 
Poland, the plans include a total of 138 assignments, while for Romania 231 until 2019.  

Of  these  assignments,  we  find  that  a  large  majority  of  them  are  also  included  in 
JASPERS’ JADE database, but not all. For Croatia and Poland, we find that around 85% 
of  the  assignments  included  in  the  country  plans  are  also  included  in  JADE,  while  for 
Romania 74% of assignments could be matched between the country plans and JADE. A 
number of assignments (7% in Croatia and Poland, and 17% in Romania) were present in 
the country plans, but not included in JADE152. When looking at developments over time 
in the country plans, however, we find that the assignments missing in JADE are for the 
early  years  of  the  programming  period.  Therefore,  on  this  basis,  we  conclude  that  the 
monitoring of assignments was initially not fully adapted to ensure consistency between 
the  internal  database  and  the  country  plans,  but  that  the  situation  was  resolved  in  the 
more recent years. 

150  Country plans do not include the assignments for the review function (IQR and PSA) for ESIF.  
151  Availability across countries differs slightly. For example, for Poland, the earliest country plan available was from 

April 2016. The analysis, however, also takes into account the coverage of the time period by the plans.  

152  For a number of early assignments we could not establish whether they were present both in country plans and in 
JADE due to availability of country plans. For Croatia, for example, the country plans available for analysis start 
in the 2nd quarter of 2015. Therefore, for all assignments created and completed in JADE earlier than this period 
we could not determine whether they we present also in country plans. The analysis, however, takes into account 
this aspect.   

157 

 
 
 
 
 
 
 
                                                 
Further  as  regards  consistency  of  information  included  for  assignments  in  the  JADE 
database  and  in  country  plans,  for  each  assignment  we  compared  the  data  on  its  status 
and timeline from JADE with the developments in terms of its presence and status in the 
quarterly plans over time. On this basis, we find that data in JADE are consistent with the 
data in country plans for a large majority of assignments. Nevertheless, for a proportion 
of  14-21%  of  assignments  (depending  on  the  country)  we  found  a  number  of 
inconsistencies.  For  example,  there  are  assignments  which  appear  and  disappear  in 
country  plans  over  time  without  a  clear  apparent  reason  while  they  are  maintained  in 
JADE  as  on-going  over  the  same  period.  There  are  also  cases  of  assignments  reported 
repeatedly  as  completed  in  country  plans,  even  two  quarters  after  the  completion  date 
recorded  in  JADE153.  Overall,  however,  when  checking  developments  over  time  in 
country  plans, we find that  such inconsistencies   are no longer present for assignments 
created starting 2018 in Croatia and Poland, but less so in Romania.  

A majority of assignments could not meet the initially expected completion dates, and 
for  80%  of  these  the  expected  duration  had  to  be  extended  by  significant  periods  of 
time. 

Next, still for the assignments that are present both in the country plans and in JADE, we 
analysed the extent to which these assignments are on track relative to the initial planning 
established  when  first  created.  For  this  purpose  we  compared  the  initial  expected 
completion date with the date of completion/ closure of the assignment for assignments 
finalised or, if still on-going, with their status by December 2018. Overall, this could be 
done  for  a  large  majority  of  assignments  as  country  action  plans  normally  include  an 
expected  date  of  completion.  Nevertheless,  we  learnt  that  the  extent  to  which  this 
information  is  included  in  country  plans  also  varies  across  countries :  for  Poland,  for 
example,  98%  of  the  assignments  in  the  country  plans  also  have  an  expected  date  of 
completion, while this proportion is lower for Romania (92%) and Croatia (87%).  

On this basis, relative to initial planning, less than half of the assignments included in the 
country  plans  are  on  time.  More  precisely,  around  44%  of  assignments  had  been 
completed/ closed on time (or earlier) or were within the planned period if still on-going 
by  end  December  2018.  For  Croatia,  however,  this  proportion  is  much  lower,  at  27%,  
thus  indicating  that  the  initial  planning  was  highly  imprecise  for  a  large  majority  of 
assignments. The median duration of delays relative to the initially expected completion 
dates  ranges  in  12-20  months,  although  there  is  a  high  variability  across  assignments. 
Across countries, longer delays are observed especially for the on-going assignments.  

Further  for  the  analysis  of  planning  over  time,  we  also  analysed  for  how  many  of  the 
assignments which are already past their initial expected completion date there has been a 

153  Normally, once an assignment is completed or closed, it is no longer included in subsequent country plans. 

158 

 
 
 
 
 
 
 
                                                 
prolongation of the expected completion rate. We found that this was the case for close to 
80% of (the delayed) assignments in Croatia, 86% in Romania, and 90% in Poland. The 
median  length  of  the  prolongation  at  assignment  level  is  in  the  range  24-32  months, 
depending  on  the  country.  At  this  stage,  however,  we  cannot  tell  whether  these 
assignments are on track to meet the updated deadlines since the monitoring system does 
not include any indication on intermediate progress in implementation, and the updated 
deadlines were generally set for 2019-2020. 

In reply to our analysis, JASPERS explained that an important factor driving the need to 
extend  assignments  over  time  relates  to  the  delays  occurring  with  the  projects  on  the 
ground. 

In  a  third  of  the  countries  supported  by  JASPERS  significant  proportions  of 
assignments  are  administratively  closed  instead  of  completed  due  to  combination 
factors. These assignments tend to be protracted  for long period of time before being 
closed.  

Further in the evaluation, an indication of sub-optimal planning is also represented by the 
proportion  of  administratively  closed  assignments.  These  are  assignments  initially 
planned for JASPERS’ support and which were closed before JASPERS could complete 
the assignment and issue an action completion note (or a progress development report for 
CEF).  Overall,  9%  of  the  assignments  in  JASPERS’  portfolio  were  administratively 
closed, with more than half of them in Croatia, Poland, and Romania. At country level, 
however, 8 in 25 countries supported by JASPERS have a proportion of administratively 
closed  assignments  higher  than  10%  in  their  number  of  assignments.  Table  A4.12  in 
Annex 4 presents this distribution across countries. 

In order to  understand better the administratively closed assignments, we also  analysed 
their duration by type of assignment.  Table 5.3.1 below presents the details. 

The median duration for administratively closed assignments is highest for an assignment 
for project implementation, followed by assignments for horizontal and strategic support, 
and  for  advisory  for  major  projects.  In  fact,  across  all  these  assignments,  we  find  that 
25%  of  them  had  a  duration  of  more  than  50  months,  thus  indicating  that  they  were 
eventually closed after a long period of time. Moreover, based on the analysis of country 
plans  above,  we  also  found  that  a  large  number  of  the  administratively  closed 
assignments  had  no  initial  expected  date  for  completion  or,  if  they  had  one,  they  were 
delayed  relative  to  initial  planning.  On  this  basis,  we  conclude  that  the  planning  and 
management of administratively closed assignments were rather approximate. 

159 

 
 
 
  
 
 
 
 
 
 
Table 5.3.1 Duration administratively closed assignments 

Type assignment 

Assignments 

43 
10 
3 
4 

Advisory major project 
Advisory non-major project 
Review function 
Capacity building 
Horizontal and strategic 
support 
Implementation 
All assignments 
Source : JAPERS’ JADE database, December 2018 

38 
1 
99 

36 
43 
25 

Duration assignment (months) 
Max 
Median 
76 
31 
50 
19 
19 
8 
14 
9 

Min 
1 
2 
4 
4 

56 
43 
76 

2 
43 
1 

As  regards  distribution  by  sector,  we  find  that  the  median  duration  of  administratively 
closed assignments is longest for Rail, air and maritime (40 months), followed by Water 
and  wastewater  treatment  (35  months).  Across  mandates,  we  find  that  the  assignments 
administratively  closed  with  longest  median  duration  are  for  the  IPA  mandate  (52 
months), followed by the CEF mandate (37 months).  

In  terms  of  reasons  for  administrative  closure,  we  identified  three  main  reasons  as 
follows:  a)  an  agreement  between  JASPERS  and  beneficiary  country  to  combine  the 
respective assignment with other on-going assignments, b) closure of the assignment due 
to  change in  context  (for example, projects  prepared for ESIF but  eventually submitted 
for CEF), and c) a decision of the beneficiary country to remove the assignment from the 
country plan if deemed no longer feasible.  

in-depth  analysis  of  10  sampled  assignments  which  were 
Based  on  a  more 
administratively closed, we find that 3 cases could be assimilated to type a) and b) above 
in the sense that either the assignment was eventually combined with another one, or that 
JASPERS had provided significant support for the respective project but the assignment 
had  to  be  closed  due  to  a  change  in  context  (as  was  the  case  for  a  project  initially 
prepared  for  ESIF  and  subsequently  submitted  for  CEF).  For  the  remaining  7 
assignments, however, we found that, once the assignment was accepted, there had been 
little  follow  up  on  the  assignment  by  the  national  authorities  and  eventually  the 
assignment  was  administratively  closed,  although  after  being  maintained  in  the 
monitoring system for a long period of time (21 to74 months). 

The  work  intensity  per  assignment  varies  significantly  by  type  of  assignment,  with 
highest median work intensity (60%) for the PSA support for the Commission. Lowest 
work intensity is  observed especially for long assignments  which  are administratively 
closed. 

160 

 
 
 
 
 
 
 
Next, we analysed the duration of all completed assignments in JASPERS’ portfolio over 
the  reference  period  until  end  2018.  Assignment  durations  by  type  of  assignment  are 
presented in Table 5.3.2 for all mandates at portfolio level. 

Table 5.3.2 Durations of assignments completed until end December 2018 

218 
77 
50 
125 
32 

Type assignment 

Assignments 

Duration assignment (months) 
Median 
18 
14 
5 
3 
14 

Advisory major projects 
Advisory non-major projects 
Review IQR 
Review PSA 
Capacity building 
Horizontal and strategic 
advice 
Implementation 
All assignments 
Note : Data covers completed assignments for all mandates. Data for review IQR and 
PSA is from SFC2014, and for the remaining assignments from JADE.  
Source : JASPERS’ JADE database December 2018, and European Commission’s 
SFC2014 October 2019. 

Max 
129 
72 
15 
23 
57 

81 
7 
590 

87 
67 

21 
21 

Min 
0.2 
1 
1.0 
1 
3 

2 
7 

The  median  duration  of  assignments  is  shortest  for  the  review  tasks  carried  out  in  the 
context  of  procedures  based  on  Art  102.2  (PSA)  and  Art.  102.1  (IQR)  for 
approval/adoption of major projects. The advisory work for major projects, on the other 
hand,  can  take  around  18  months  (median  value)  for  half  of  the  projects,  but  was 
extended even as long  as 10  years in  one case.  In fact,  for 25% of the  assignments for 
major  projects,  the  duration  is  above  35  months.  Another  type  of  assignment  with  an 
even longer duration is the horizontal and strategic advice. With a median duration of 21 
months, this support lasted for more than 38 months in 10% of cases.  

In  terms  of  advisory  support  by  sector,  we  find  that  the  longest  median  duration  is 
primarily  for  Water  and  wastewater  (22  months),  followed  by  Roads  and  Energy  and 
solid  waste  (18  months).  Across  mandates,  the  longest  median  duration  for  completed 
assignments is for ESIF (19 months), followed by IPA (15 months), and CEF (3 months).  

From the efficiency perspective, however, it is important to understand that the calendar 
duration  of  an  assignment  does  not  mean  full  time  work  for  JASPERS’  experts 
throughout the period. From the in-depth analysis of sampled assignments, and especially 
for  advisory  work,  we  learnt  that  it  can  happen  that  an  assignment  is  accepted  and 
recorded  in  the  system  at  a  given  date  and  the  actual  work  starts  much  later  when  the 
beneficiary is ready to send the project documentation for review to JASPERS.  

161 

 
 
 
  
 
 
 
 
An  example  of  a  very  long  duration  is  an  assignment  for  project  preparation  in  Roads 
present  in  the  monitoring  system  for  8  years,  and  which  got  seriously  delayed  due  to 
protracted  negotiations  with  landowners  in  the  country  which  needed  to  be  finalised 
before  the  project  application  could  be  prepared  and  submitted.  For  this  assignment, 
JASPERS could assist with project preparation at the beginning of the process, but then 
had to wait a long time for the review of project application until negotiations at national 
level were finalised.  

Furthermore,  we also  learnt that it is not  uncommon  that several  aspects  of support are 
combined into one long assignment. An example is an assignment which also lasted very 
long (6 years) which entailed supporting the national authorities for the development of a 
national  master  plan  for  transport  that  would  also  ensure  compliance  with  the  ex-ante 
conditionality  for  cohesion  investments  in  the  sector.  In  this  case,  JASPERS’  advisory 
effort was distributed over years, starting with the production of a background paper for 
strategic planning, and following with several guidance notes at the following stages of 
the work until the strategy was finalised.  

As regards the actual time worked per assignment, JASPERS has started monitoring this 
aspect  in  2017,  recording  the  numbers  of  hours  worked  per  assignment  completed  or 
administratively  closed.  For  our  analysis,  JASPERS  shared  with  us  data  for  190 
assignments started and completed or closed during the period January 2017 – February 
2019154. This dataset includes 60% assignments for the review function (IQR and PSA), 
and the rest for advisory work and horizontal assignments.  

Using  the  hours  worked  per  assignment,  we  calculated  the  intensity  of  work  per 
assignment  as  follows.  First,  starting  from  the  calendar  duration  of  the  assignment,  we 
assumed  that  70%  of  this  duration  represents  working  days  and  calculated  a  maximum 
potential working period per assignment155. Second, based on the hours worked reported 
by  JASPERS  per  assignment,  we  assumed  a  standard  working  day  of  8  hours  and 
calculated the number of standard days worked per assignment156. Finally, we computed 
the intensity of effective work as the ratio between the number of standard days worked 
and the length of the potential working period for the each assignment. This gives us the 
proportion  of time worked in  the maximum working time available for the assignment. 
Table 5.3.3 below presents these results by type of assignment. 

154  94% of the assignments in this dataset are for ESIF, 4% for IPA, and 2% for CEF. 
155  This  assumption  is  based  on  the  fact  that  a  calendar  year  of  365  days  usually  includes  250-255  working  days. 

156 

Therefore we made a round approximation of 70% working days within a calendar year.  
In  addition  to  reported  working  hours  per  assignment,  JASPERS  estimates  that  the  time  needed  for  ex-ante 
preparation of assignments and other horizontal tasks amounts to 12% of the overall working time.  

162 

 
 
 
 
 
 
 
 
                                                 
Table 5.3.3 Intensity of work per assignment in total working time157 

Note: Data covers 98% of assignments started and completed/ closed over the period Jan 
2017-Feb 2019.  
Source : JASPERS’ data on hours worked per completed assignment.  

For  presentational  purposes,  in  Table  5.3.3  we  include  only  the  median  values  for  the 
number of days worked and for the maximum number of working days available for an 
assignment, while presenting in more details the distribution of the intensity of work per 
assignment.  

From  this  data  we  learn,  for  instance,  that  across  the  45  assignments  for  advisory  for 
major projects, for a majority of them JASPERS worked up to 51 days per assignment, 
while  the  maximum  duration  of  working  days  for  a  majority  of  assignments  was  195 
days. The median value or the intensity of work for this type of assignment amounts to 
30%, although we see that there is a high variability across assignments in this group. At 
a minimum, there is one assignment for which the actual work represents around 1% of 
the  total  working  time  during  the  assignment.  The  is  the  case  of  one  assignment  that 
lasted for almost 16 months, and the days of work reported are around 9. The assignment 
was eventually administratively closed. At the higher end, we also have assignments with 
a  very  high  intensity  of  work.  For  advisory  of  major  projects,  the  assignment  with 
maximum intensity of 152% lasted little more than two months, and the numbers of days 
worked reported is 75. This is possible, for instance, if several experts work in parallel on 
the same assignment during a given (usually short) period.   

Across  types  of  assignments  we  learn  that  the  assignments  for  PSA  have  the  highest 
intensity of work, a result which can be explained by the fact that the period available to 
experts to  review and assess  a major project  for  the PSA service to  the  Commission  is 
around  30  calendar  days.  Therefore,  the  work  effort  needs  to  be  concentrated  in  this 
relatively  shorter  period  of  time.  Further,  also  for  the  PSA,  we  see  that  there  can  be 
assignments  with  low  intensity,  but  these  are  usually  in  a  minority.  More  than  80%  of 
assignments for PSA reports have a work intensity of at least 30%.  

For  the  remaining  types  of  assignments  a  similar  interpretation  applies,  although  the 
reliability of the summary values differs with the number of assignments. For instance, 

157 

Intensity of work can be zero for assignments administratively closed, and higher than 100% if more experts work 
at the same time on the assignment.  

163 

MedianMaxMinAdvisory major projects455119530%152%1%Advisory non-major projects201110412%53%0%Review IQR34267334%207%3%Review PSA80142060%262%8%Capacity building32742814%38%6%Horizontal and strategic support62820920%125%4%Advisory project implementation25519038%72%5%Total assignments190Intensity of work per assignment (%)Max working period in days (median value)Days worked (median value)Type assignmentAssignments 
 
 
 
 
 
 
                                                 
for  capacity  building,  horizontal  support  and  implementation,  there  are  very  few 
completed  assignments  included  in  the  dataset  analysed,  and  therefore  the  values  are 
more  illustrative.  Nevertheless,  across  assignments,  for  the  advisory  work  we  find  that 
actual work is more concentrated over a calendar period for ESIF assignments for major 
projects  (with  median  work  intensity  of  33%),  followed  by  CEF  advisory  assignments 
(23%), ESIF advisory for non-major projects (16%), and IPA advisory (10%). 

Overall,  there  is  scope  for  a  more  structured  approach  for  the  planning  of 
assignments,  with  well-defined  and  targeted  objectives,  clear  and  realistic  timelines, 
and milestones that enable assessing the progress over time during implementation. 

Further,  when  analysing  the  intensity  of  work  in  relation  to  the  duration  of  the 
assignment,  we  find  that  there  is  little  correlation  between  them.  In  Figure  5.3.1  we 
illustrate the relationship  between the two variables for assignments for advisory work, 
distinguishing between major and non-major projects. 

Figure 5.3.1 Duration of assignments and intensity of work – advisory 

Source : Based on data for hours worked from JASPERS, Jan 2017-Feb 2019 

From  the  scatterplot  in  Figure  5.3.1,  we  see  that,  while  the  duration  and  intensity  of 
assignments  for most non-major projects  tend to  concentrate in  the area  with  relatively 
lower  values  for  both  duration  and  intensity,  for  major  projects  there  is  much  more 
heterogeneity. As already presented, we can identify assignments with a high number of 
hours  worked  during  a  relatively  shorter  period  of  time,  but  for  the  majority  of  the 
assignments there is not much correlation apparent between duration and work intensity 
since  there  are  many  assignments  with  varying  durations  at  lower  levels  of  work 
intensity. Jaspers explained that this is due to the fact that the evolution of assignments is 
highly dependent on the evolution of the projects on the ground.  

164 

 
 
 
 
 
 
 
A similar graph is presented in Figure 5.3.2, illustrating the distributions of the duration 
of  assignments  and  the  work  intensity  for  the  two  types  of  assignments  for  the  review 
function (IQR and PSA). 

Figure 5.3.2 Duration of assignments and intensity of work – review 

Source : Based on data for hours worked from JASPERS, Jan 2017-Feb 2019 

For the review function, the higher concentration of short assignments is clearly apparent 
for the PSA function, with the work intensity varying from low to very high. For the IQR 
version  of  the  review,  on  the  other  hand,  the  distribution  is  more  horizontal  across  the 
scale  for  duration,  as  assignments  range  from  short  to  rather  long  (up  to  10  months), 
while the work intensity also varies. With IQR we note that there is a tendency for lower 
work intensity to  be associated with  the longer  assignments, although the correlation is 
influenced by outliers.   

We  believe  that  two  possible  reasons  for  these  distributional  differences  between  the 
PSA  and  IQR  include  the  following.  First,  as  noted,  the  PSA  assignments  have  a  clear 
deadline of 30 days agreed between JASPERS and the Commission. This deadline was 
introduced  since  the  Commission  itself  is  bound  by  regulatory  deadlines  for  issuing  a 
decision on a major project submitted, and therefore there was a need for predictability as 
regarded  PSA.  As  regards  IQR,  however,  there  was  no  regulatory  deadline  until  July 
2018, and therefore the process tended to be prolonged for some assignments.  

Second,  while  the  PSA  service  is  provided  directly  to  the  Commission,  for  the  IQR 
JASPERS needs to run an interactive process of assessment with the Member State. This 
happens, for instance, due to the need to  clarify certain aspects of the projects with the 

165 

 
 
 
 
 
 
 
 
Member  State  during  the  assessment.  And  these  exchanges  do  take  time.  When  we 
compared  the  total  duration  of  the  IQR  process,  including  also  the  notification  and 
approval  in  the  Commission,  we  found  that  it  takes  around  7  months  for  a  majority  of 
projects,  and  therefore  four  months  more  than  for  non-interrupted  projects,  and  one 
month more than for interrupted projects in the case of Art. 102.2. In the future, however, 
we  expect  that  the  IQR  process  will  become  efficient  for  more  assignments  due  to  the 
regulatory deadline of 6 months introduced for it in the Omnibus regulation in 2018.  

To sum up, based on the analysis in this sub-section, we conclude that there is scope for 
improvement  and  rationalisation  of  the  planning  and  management  of  assignments 
included in JASPERS’ portfolio. First, for a third of the countries assisted we find high 
proportions  of  assignments  included  in  country  plans  and  subsequently  closed 
administratively, and this type of assignments have very low work intensities and tend to 
be  protracted  for  long  periods  of  time.  Nevertheless,  in  order  to  prepare    these 
assignments  for  approval  with  the  Commission,  JAPERS  needs  to  invest  resources  ex-
ante  and,  given  that  the  value  added  is  minimal  to  none  eventually,  this  is  a  source  of 
inefficiency.  Second,  as  regards  assignments  that  are  completed  and  on-going,  we  find 
that the planning for a majority of them in the sampled countries was rather imprecise, as 
they needed to be subsequently extended by significant periods of time. In this regard, we 
assess  that  there  is  a  need  for  more  structure  in  the  planning,  with  well-defined  and 
targeted objectives, clear and realistic timelines, and milestones that enable assessing the 
progress over time during implementation.  

Human and financial resources 

The  total  budget  for  JASPERS’  ESIF  mandate  for  the  period  2014-2018  was  175 
million  euro,  with  the  Commission’s  average  contribution  of  81%  to  the  total  costs. 
The ESIF mandate represents 97% of the total resources allocated for the initiative.  

The  financial  resources  mobilized  for  JASPERS’  operations  for  the  ESIF  mandate 
amount to 175 million euro over the period 2014-2018, representing 84% of the budget 
committed for the period. Of these, the Commission contribution covers 81% of the total 
costs. Additional 1.4 million euro were allocated for the CEF1.1 mandate over the period 
2015-2018, of which the Commission contribution covered 90%158. Finally, for the IPA 
mandates, the resources allocated amount to around 4 million euro over the period 2015-
2018, representing 64% of the total budget planned for the period159.  

Relative  to  the  previous  period,  JASPERS  has  developed  significantly  its  pool  of 
human  resources,  with  the  number  of  professional  and  support  staff  in  2018 
increasing by 55% relative to 2013.  

158  For the CEF1.2 mandate, the allocated EC contribution is EUR 3 million. 
159  Of these, the budget execution rate is higher for the Serbia mandate (75%), compared to IPA II (59%). 

166 

 
 
 
 
 
 
                                                 
In what follows we focus on the ESIF mandate as it represents 97% of the total resources 
allocated to  JASPERS initiative in  the current  period, and it also  enables a comparison 
with the first period (2007-2013) of JASPERS’ existence. Table 5.3.4 presents the main 
coordinates  in  terms  of  actual  human  and  financial  resources  allocated  to  the  initiative 
over the two periods of time.  

Table 5.3.4 Human resources and actual expenditure for JASPERS 2007-2018 

ESIF mandate 

Period 2007-
2013 
79 
85% 
15% 

Period 2014-
2018 
122 
85% 
15% 

Total staff (average FTEs / year), of which: 
Professional staff (% in total) 
Support staff (% in total) 
Cumulative total expenditure (million. euro), of 
which: 
Staff salaries (% in total) 
External consultancy (% in total) 
Travel and other (% total) 
Cumulative EC contribution (million. euro) 
Share EC contribution in total cost (%) 
Cumulative actual to planned total cost  (%) 
Note: Professional staff includes JASPERS’ management and the experts. 
Source: Based on data reported in JASPERS’ annual reports 2007-2018. 

186 
82% 
15% 
3% 
132 
71% 
n.a. 

175 
91% 
6% 
3% 
142 
81% 
84% 

When  comparing  the  resources  allocated  over  the  two  periods  of  time,  we  note  that 
JASPERS has grown significantly over time. If in 2007 it started with 56 employees, in 
2018  professional  and  support  staff  increased  to  140  -  therefore  2.5  times  higher.  As 
reflected  by  the  developments  over  time  explained  in  the  introductory  chapters,  this  is 
due to the need to adapt the initiative to the demand for JASPERS’ services, which have 
expanded and become more diversified in the current period.  

Moreover,  from  the  structure  of  the  budget,  we  also  note  that  there  has  been  a 
reorientation  in  JASPERS’  staff  policy  in  the  current  period,  in  that  the  initiative  now 
relies  on  own  staff  to  a  larger  extent.  In  the  previous  period  the  use  of  external 
consultancy services complementing JASPERS’ expertise was higher, as indicated by the 
larger  share  of  expenditure  with  such  services  contracted  by  JASPERS.  Consequently, 
we see that the bulk of JASPERS’s resources are currently invested in its people, with the 
share  of  staff  costs  increasing  from  82%  to  91%  between  the  two  periods.  As  regards 
conditions  of  employment,  the  FAFA  agreement  establishes  that JASPERS’s  personnel 
be employed under the EIB standard employment conditions. In terms of distribution of 
JASPERS experts across locations in year 2018, 21% were located in Luxembourg, 13% 

167 

 
 
 
 
 
 
in  Brussels,  and  the  remaining  67%  across  the  regional  offices  in  Bucharest,  Sofia, 
Vienna and Warsaw. 

As  regards  the  sources  of  financing,  data  indicates  an  increase  in  the  Commission’s 
contribution  to  the  total  budget  of  the  initiative  from  71%  to  81%  between  the  two 
periods  –  an  increase  which  served  primarily  to  cover  the  higher  staff  costs  incurred 
during the current period. 

Next, based on the two panels in Figure 5.3.3, we look at the dynamics of the human and 
financial resources, and of the number of assignments over time.  

Figure 5.3.3 Human and financial resources, and assignments over time, ESIF mandate 

Source: Based on data reported in JASPERS’ annual reports 2007-2018 

The  left  panel  in  Figure  5.3.3  illustrates  the  number  of  annual  full  time  equivalents 
(FTEs)  supported  in  JASPERS  and  the  annual  Commission’s  contribution  over  the 
period  2011-2018.  This  data  indicates  that,  starting  with  year  2014,  there  has  been  a 
systematic increase in  staff  every  year over the  period, while the Commission’s annual 
contribution  increased  primarily  in  2015-2016,  and  in  2017-2018.  Relative  to  2013, 
JASPERS’ personnel  was 55% higher  and Commission’s annual  contribution was 65% 
higher in 2018. 

Preliminary evidence on developments over time indicates a need for further in-depth 
reflection on possibilities to optimise allocation of work effort and rationalise costs. 

To  some  extent,  the  right  panel  in  the  figure  helps  us  understand  better  these 
developments,  as  it illustrates the dynamics in  the number of the on-going assignments 
by  year,  together  with  the  evolution  of  completed  and  administratively  closed 
assignments.  First,  as  regards  the  on-going  assignments,  we  note  an  almost  linear 
increase on a yearly basis until year 2017, while staff continued increasing significantly 
throughout the current period. The increasing annual number of assignments until 2017 
can  be  explained  by  the  rate  of  accepting  assignments  and  the  median  duration  of 
assignments  of  roughly  two  years.  Assignments’  completion/  closure  took  off  mostly 

168 

 
 
 
 
 
  
 
 
 
 
starting  with  year  2016.  The  number  of  on-going  assignments,  however,  is  only  an 
imperfect  proxy  of  the  actual  work  on  the  assignments  since,  as  explained  earlier,  the 
work intensity per assignment varies significantly across assignments, and it is not very 
high for a majority of assignments other than the PSA review function. Nevertheless,  a 
more detailed analysis of this aspect of efficiency is not feasible for this evaluation as it 
would require data on hours worked per assignment for all assignments and per calendar 
year– data which was not available for the period analysed in this evaluation. Therefore, 
we conclude with a partial assessment that this preliminary evidence indicates a need for 
further  in-depth  analysis  of  the  topic  in  order  to  help  identify  possibilities  to  optimise 
allocation of work effort and rationalise costs.  

5.4 

COHERENCE 

In  this  section  we  analyse  the  external  and  internal  coherence  of  JASPERS’  services. 
External coherence includes the alignment of JASPERS’ initiative with the EU priorities, 
with a focus on the objectives of Cohesion Policy and on the higher EU level objectives 
for  climate,  environment,  and  competition  in  the  Union.  Further  for  the  external 
coherence, we also explored the possibility of analysing the synergies between JASPERS 
and  other  centrally  managed  initiatives  for  technical  assistance.  As  regards  internal 
coherence,  we  analyse  the  consistency  of  approach  for  project  assessment  between  the 
two main divisions of JASPERS : advisory and IQR. The main findings are reported in 
Text box 5.4.1. 

Text Box 5.4.1 Main findings for coherence of JASPERS’ activities 2014-2018 

Coherence with EU priorities 

1)  EU  priorities:  JASPERS’  services  are  fully  coherent  with  EU  requirements  and 
standards  for  cohesion  policy,  and  with  the  higher  EU  level  objectives  for  climate, 
environment  and  competition.  Its services  contribute to  guiding project  preparation 
and to supporting the assessment of these projects during adoption in line with these 
policy priorities. 

Coherence with other advisory services at EU level 

2)  Other EU advisory services: The potential for synergies between JASPERS and other 

technical assistance initiatives had not been exploited until end 2018. 

Internal coherence between the advisory and review functions 

3)  Internal coherence: JASPERS IQR and JASPERS advisory are broadly consistent in 
their approach to project assessment, but some scope for further alignment remains. 
This  scope  is  limited  to  up  to  10%  of  the  projects  for  the  minimum  regulatory 
requirements  regarding  project  quality,  and  it  is  wider  for  non-critical  aspects 

169 

 
 
 
 
 
 
 
 
 
 
 
identified with project quality. 

For each of these evaluation findings the supporting evidence is as follows.  

170 

 
 
 
 
 
 
Coherence with EU priorities 

JASPERS’  services  are  fully  coherent  with  EU  requirements  and  standards  for 
cohesion  policy,  and  the  higher  EU  level  objectives  for  climate,  environment  and 
competition.  Its  services  contribute  to  guiding  project  preparation  and  to  supporting 
the assessment of these projects during adoption in line with these policy priorities. 

The  application  for  EU  financing  for  a  major  project  for  Cohesion  Policy  requires  the 
Member  States  to  ensure  the  alignment  of  the  project  with  EU  priorities  regarding 
objectives  of  Cohesion  Policy  and  the  higher  EU  level  objectives  of  mitigation  and 
adaptation  to  climate  change,  account  for  environmental  impacts,  and  state  aid 
compliance.  In  consequence,  all  JASPERS  services  are  organised  so  that  they  promote 
and align the advisory and review work with these EU priorities.  

We  analysed  the  coherence  of  JASPERS’  work  with  these  EU  priorities  based  on  the 
documentation  issued  by  its  services,  and  especially  the  documents  which  include  the 
assessments of the projects with respect to objectives of Cohesion Policy, environment, 
climate and  competition  law. Table 5.4.1 summarizes the information  on the document 
sections  which  reflect  these  assessments  in  the  IQR  and  PSA  reports  for  the  review 
function, and in the ACNs for the advisory function. 

Table  5.4.1  Sections  of  JASPERS’  documents  for  assessment  of  alignment  with  EU 
priorities  

Source: based on the templates established for IQR and PSA reports, and on JASPERS’ 
internal guidelines for action completion notes.  

For the IQR review, the quality review criteria for the assessment of a major project are 
established in Commission Delegated Regulation (EU) 480/2014 Annex II. On this basis, 
as regards the project’s alignment with the objectives of Cohesion Policy, in Section 3 of 
the  IQR  report,  JASPERS  IQR  summarises  the  information  on  project  objective  and 
assesses its consistency with the priority axes of the operational programme from which 
the resources  for investment are to be allocated. For the EU environmental and climate 
priorities, JASPERS IQR summarizes  the information  on the analysis of environmental 
impacts  including,  for  example,  the  implementation  of  polluter  pays  and  preventive 

171 

DocumentEU cohesion policyEU environmental policyClimate proofingEU state aidIQR ReportSection 3: consistency with the operational programme and contribution to its objectivesSection 4.2.1 Consultation on state aid issuesPSA ReportNot covered by PSA. Commission's assessmentNot covered by PSA. Commission's assessmentSection 2.6 Climate change and mitigation needs, and disaster resilienceNot covered by PSA. Commission's assessmentAction Completion NoteSection 3: consistency with the operational programme and contribution to its objectivesAnnex: Environmental information: SEA and EIA Directives, effects on Natura 2000 sites, project consistency with Community policy and legislationSection 13: Climate change mitigation and GHG emissions, climate adaptation and resilienceSection 9: State aidSection 7: Analysis of the environmental impact, taking into account climate change adaptation and mitigation needs, and disaster resilience 
 
 
 
 
 
 
 
action  principles,  compliance  with  the  EIA  Directive160,  ex-ante  conditionalities  of 
environmental legislation for Cohesion Policy, compliance with the Habitats Directive161 
etc.  

Similarly,  for  climate  adaptation  and  mitigation  measures,  JASPERS  IQR  provides  an 
assessment  of  project’s  contribution  to  the  objectives  of  the  EU  climate change  policy, 
and in particular to the Europe 2020 strategy, as well as measures to ensure resilience to 
climate change variability. For state aid issues, part of the working arrangements for IQR 
between  the  Member  States  and  JASPERS  IQR  is  the  possibility  to  ask  JASPERS’ 
experts  to  consult  the  Commission  on  state  aid  rules  applicable  to  the  project.  This 
information  is  included  in  the  IQR  report,  where  JASPERS  explains  whether  this 
consultation took place, and whether it considers that the project might involve state aid. 
Further,  in  case  the  project  does  involve  state  aid,  JASPERS  also  gives  its  opinion  on 
whether  the  state  aid  is  in  line  with  the  regulatory  requirements.  Finally,  for  all  these 
issues  JASPERS  IQR  needs  to  provide  clear  statements  whether  the  project  meets  the 
regulatory quality criteria together with supporting justifications.  

Further, upon receiving a project  notification including also  JASPERS’ IQR  report, the 
Commission has the possibility to reject the request for EU funding if it considers that a 
positive statement in the IQR report is not supported or it is contradicted by information 
available to the Commission162. This could be the case, for instance, if the Commission 
has  additional  information  on  complaints,  pilot  investigation,  infringements,  national 
proceedings,  Ombudsman  investigation  etc.  on  environmental  issues.  In  our  analysis, 
however, we did not find any such cases where notifications of major projects based on 
the IQR assessment were rejected by the Commission for non-fulfillment of these quality 
criteria.  

By comparison with the IQR process, the PSA assessment is more focused primarily on 
the technical and operational aspects of the project.  It does not cover the assessment of 
project’s alignment with the operational programme and neither assesses the compliance 
with the environmental regulations and EU state aid rules. These aspects are covered by 
the Commission through its internal inter-service consultation. As regards climate change 
aspects,  the PSA report  provides an  assessment  of the project’s alignment  with  climate 
proofing requirements established in the Memorandum of Understanding agreed upon by 
DG  REGIO  and  DG  CLIMA  in  2015.  This  agreement  between  the  two  Commission 
services ensures that JASPERS provides an assessment according to several criteria such 
as the national adaptation strategy and the project’s contribution to it, the GHG emissions 

160  Directive 2011/92/EU. 
161  Directive 92/43/EEC. 
162  Additional  possible  reasons  include  the  omission  of  statements  in  the  IQR  reports  or  statements  rendered 
unreliable due to changes that have occurred in the project between the finalization of the IQR and the notification 
to the Commission.  

172 

 
 
 
 
 
                                                 
entailed by the project, identification of project vulnerability as regards climate change, 
relevant measures to ensure resilience etc.  

Finally, in the course of advisory work, depending also on the scope of the assignment 
requested  by  the  Member  States,  JASPERS  advisory  provides  an  assessment  (or,  at  a 
minimum, a description) of the project’s alignment with the EU requirements on policy 
objectives and EU priorities for the environment, climate change, and competition rules. 
In  this  process,  it  can  raise  significant  issues  which  require  changes  in  project 
documentation in order to ensure alignment with EU requirements and standards or, for 
less  critical  issues,  it  can  make  comments  with  suggestions  for  further  improvement. 
Earlier in this chapter, in the section for relevance, we found that JASPERS’ advice on 
climate and environmental issues is rather frequent, for 54-60% of the projects assisted, 
while state aid issues are addressed in about a quarter of these projects. 

To  sum  up,  as  regards  coherence  with  EU  cohesion  policy  and  priorities  for  the 
environment,  climate,  and  competition,  we  conclude  that  JASPERS’  services  are  fully 
coherent  with  EU  requirements  and  standards,  and  its  service  contribute  to  guiding 
project preparation and to supporting the assessment of these projects during adoption in 
line with these policy priorities.  

Coherence with other advisory services at EU level 

The  potential  for  synergies  between  JASPERS  and  other  technical  assistance 
initiatives had not been exploited until end 2018. 

For  this  evaluation,  we  also  explored  the  possibility  of  assessing  the  coherence  and 
synergies  between  JASPERS  services  and  other  Commission’s  initiatives  for  technical 
assistance. Under the current Multiannual Financial Framework (MFF), there are several 
centrally managed initiatives for technical assistance, and most of them are for capacity 
building of horizontal nature and for project preparation. They include, for example, the 
European  Investment  Advisory  Hub  (EIAH),  the  European  Local  Energy  Assistance 
(ELENA), Connecting Europe Facility (CEF), European Energy Efficiency Fund (EEEF) 
etc.163.  Among these initiatives, JASPERS is one of the largest  - with 30% of the total 
cumulated  budget  of  more  than  700  million  euro  for  technical  assistance  centrally 
managed. 

 JASPERS’  Quality  Manual  mentions  the  possibility  for  JASPERS  to  cooperate  in 
sharing expertise with other technical assistance services. Moreover, in 2015, following 
the publication of the EFSI Regulation164, JASPERS analysed the scope for a cooperation 
with the European Investment Advisory Hub (EIAH) and even established procedures in 

163  A list of these initiatives is presented in ECA (2020). 
164  Regulation (EU) 2015/1017. 

173 

 
 
 
 
 
 
 
                                                 
this  regard.  In  our  analysis  of  JASPERS’  portfolio,  however,  we  did  not  find  any 
significant work involving such cooperation with other centrally managed initiatives for 
technical assistance over the reference period of the evaluation. Therefore, on this basis, 
we  conclude  that  the  potential  for  synergies  between  JASPERS  and  other  technical 
assistance initiatives has not yet been exploited during the period analysed165.   

Internal coherence between the advisory and review functions 

JASPERS  IQR  and  JASPERS  advisory  are  broadly  consistent  in  their  approach  to 
project assessment, but some scope for further alignment remains. This scope is limited 
to  up  to  10%  of  the  projects  for  the  minimum  regulatory  requirements  regarding 
project quality, and wider for non-critical aspects identified with project quality. 

Next,  we  analysed  also  the  coherence  between  the  advice  provided  by  JASPERS’ 
advisory to the assisted projects, as reflected in the ACNs issued for this work, and the 
project  assessments  carried  out  by  JASPERS  IQR  in  the  PSA  reports.  This  analysis 
covers  77  major  projects  which  were  first  supported  by  JASPERS  advisory  during 
project  preparation,  and  were  subsequently  submitted  to  the  Commission  for  approval 
based on the procedure of Art. 102.2 with PSA assessment.  

As explained earlier, we used the ACNs issued by JASPERS advisory to classify the type 
of contribution that JASPERS’ support had to the projects by topic. This classification is 
the  same  as  the  one  used  for  the  chapter  on  effectiveness,  and  it  groups  projects  as 
follows: 1) projects for which the ACN raises significant issues with project quality, and 
which require revision before the project is considered suitable for the application for EU 
financing; 2) projects with no significant issues, but with comments and suggestions for 
further improvement of  project  preparation;  in  such cases the issues  are  not  considered 
critical from the regulatory perspective of project quality; 3) projects with no significant 
issues and no comments, but with evidence that JASPERS had contributed to improving 
project quality, 4) projects for which JASPERS provides a positive assessment on a given 
topic,  but  for  which  we  could  not  find  evidence  on  1)  to  3),  and  5)  projects  for  which 
JASPERS provides a description of the topic, but no explicit assessment.  

Further,  for  the  same  projects  for  which  we  analysed  the  ACNs,  we  also  screened  the 
PSA  reports  issued  later  by  JASPERS  IQR  during  the  approval  process  in  the 
Commission. In these reports, JASPERS IQR emphasized critical and non-critical issues 
still pending with project quality by topic, if applicable. As explained earlier, we found 
that it is often the case that the assessment made by JASPERS IQR in the PSA reports is 
also  reflected  in  the  Commission’s  observation  letters  sent  to  the  Member  States  when 
project are interrupted.  

165  This is likely to change in the future as, for the next MFF, the Commission proposed the creation of the InvestEU 

Advisory – a centrally managed initiative which will include also JASPERS.    

174 

 
 
 
 
 
 
                                                 
For  the  analysis  of  coherence  of  the  two  rounds  of  JASPERS’  assessment  during  the 
advisory phase and in the PSA process, we analysed how frequent were the cases where 
critical issues for a given topic signaled by JASPERS IQR were also raised as significant 
issues, or comments, by JASPERS advisory. A summary of the data for this analysis is 
presented in Table 5.4.2.  

Table  5.4.2:  Comparison  of  PSA  reports  and  ACNs  for  projects  with  critical  issues  in 
PSA 

Topic with critical 
issue(s)  
in PSA report 

Number 
projects 

Significant 
issue 

Conclusions in ACN: 
Assessed, 
with 
comment  
7 
6 

Assessed, 
no issues 
raised 
1 
2 

8 
5 

16 
13 

Cost benefit analysis 
Feasibility study 
Risk and sensitivity 
analysis 
Project description 
Project costs 
Capacity of beneficiary 
Financial plan 
Project timetable 
Note: Based on analysis of 77 major projects submitted for Art. 102.2 
Source: PSA reports and ACNs 

0 
1 
3 
0 
0 
1 

7 
8 
9 
2 
2 
2 

3 
3 
2 
2 
1 
1 

4 
3 
4 
0 
0 
0 

% 
significant 
in critical 

50% 
38% 

43% 
38% 
22% 
100% 
50% 
50% 

Table  5.4.2  reads  as  follows.  For  cost  benefit  analysis,  for  example,  JASPERS  IQR 
signaled critical issues on this topic in its PSA reports for 16 out of 77 the major projects 
analysed.  When  looking  at  the  corresponding  ACNs,  we  find  that  JASPERS  advisory 
raised  significant  issues  with  cost  benefit  analysis  only  for  8  of  these  16  projects,  and 
included  comments  for  further  improvement  for  7  of  these  projects.  Therefore,  we 
conclude that JASPERS’ advisory was fully coherent with the PSA assessment in 50% of 
the cases, and less so for the remaining cases for which it provided comments but no firm 
warning on outstanding  issues. A similar interpretation  applies to  all topics included in 
the table.  

For a large majority of projects (above 90%), we assess full coherence between the two 
services  in  the  following  sense.  First,  for  projects  with  no  critical  issues  identified  in 
PSA,  there  are  cases  for  which  JASPERS  advisory  had  raised  significant  issues  during 
project preparation and the Member State addressed them  before submitting the project 
to the Commission. Therefore, these issues were no longer outstanding when the project 
reached  the  PSA  assessment,  and  the  two  types  of  assessment  are  coherent  even  if 
different. Second, for other projects for which the PSA did raise critical issues on a given 

175 

 
 
 
 
 
 
topic, JASPERS advisory also signaled significant issues during project preparation, but 
they  were  not  addressed  by  the  Member  State  by  the  time  the  project  reached  the 
Commission.  Also  in  this  case  we  consider  that  the  two  assessments  were  coherent,  as 
the outstanding critical issues were due to lack of improvements in the application by the 
Member State despite early warnings.  

Nonetheless,  this  analysis  also  indicates  partial  coherence  between  JASPERS  advisory 
and JASPERS IQR assessment in PSA reports for about 9-10% of major projects for the 
topics of cost benefit analysis, feasibility study, and projects costs, and to a lesser extent 
(6%) for risk and sensitivity analysis, and project description.  

Furthermore, we carried out  a similar analysis also  for the non-critical  issues identified 
by  JASPERS  IQR  in  the  PSA  report,  and  their  correspondence  with  the  support  from 
JASEPERS advisory. The data for this analysis is summarized in Table A4.13 in Annex 
4. In this case we focused more on the projects for which the ACNs did not signal any 
further scope for improvement (either in comments or significant issues) while they had 
non-critical issues identified in the PSA on a given topic, and we found that consistency 
in assessments differs significantly across topics between the two services. For example, 
as  regards  the  capacity  of  beneficiary,  the  PSA  assessment  differs  from  the  ACN 
assessment  in  38%  of  the  projects  analysed.  Similarly,  for  cost  benefit  analysis,  risk 
assessment and the project financial plan, the proportion ranges in 30-34%. Therefore, on 
this  basis,  we  conclude  that  there  is  more  heterogeneity  between  the  two  types  of 
assessments as regards the identification of non-critical issues in project quality.  

To sum up, as regards the internal  coherence between the assessments produced by the 
two  types  of  JASPERS’  services,  we  conclude  that  the  JASPERS  IQR  and  JASPERS 
advisory are broadly consistent in their approach to project assessment, while some scope 
for further improvement remains. This scope is limited to up to 10% of the projects for 
the minimum regulatory requirements regarding project quality, and it is wider for non-
critical aspects. 

5.5 

EU ADDED VALUE 

This  section  includes  the  main  findings  for  the  analysis  of  the  EU  added  value  of 
JASPERS  initiative.  It  relies  primarily  on  the  feedback  received  from  stakeholders 
responding  to  the  online  targeted  consultation  run  for  this  evaluation  during  summer 
2019.  In  addition,  we  also  present  highlights  from  the  perceptions  of  stakeholders 
interviewed  on  aspects  of  JASPERS’  EU  added  value  for  the  in-depth  analysis  of 
sampled  assignments  and  countries.  Finally,  the  section  also  includes  a  summary  of 
JASPERS’ own assessment of the added value of its advisory work. The main  findings 
for the EU added value are presented in Text Box 5.5.1. 

176 

 
 
 
 
 
 
 
Text Box 5.5.1 Main findings for the EU added value of JASPERS’ activities 2014-2018 

1)  Stakeholder consultation: A large majority of stakeholders consider that, relative to 
other  advisory  services,  JASPERS’  added  value  is  higher  especially  as  regards  its 
support  for  the  alignment  with  EU  priorities,  the  potential  for  knowledge  transfer 
among Member States, its hands-on experience, and its technical expertise.  

2)  Stakeholder  consultation:  A  large  majority  of  Managing  Authorities  and  project 
beneficiaries/ promoters consider that JASPERS’ services will still be needed in the 
future. Willingness to pay from own technical assistance for these services, however, 
is rather low for both types of stakeholders.  

3)  Sampled  assignments:  Further  evidence  from  sampled  assignments  indicates  an 
appreciation of JASPERS’ added value especially in terms of its technical expertise 
and  EU-wide  experience  acquired  while  working  in  many  countries,  and  with  the 
European Commission.  

For each of these evaluation findings the supporting evidence is as follows.  

For  the  analysis  of  the  EU  added  value  of  JASPERS  initiative,  we  first  asked  the 
stakeholders  for  their  opinions  on  JASPERS’  added  value  relative  to  other  advisory 
services with regards to several service aspects such as the quality of technical expertise, 
the hands-on experience with projects, the knowledge transfer from other Member States, 
the  knowledge  of  the  context  of  the  beneficiary  Member  State,  etc.166.  The  full  list  of 
aspects considered for the question and the distributions of replies by role of respondent 
(i.e. Managing Authorities, project beneficiary / promoter, consultancy company, or other 
type of stakeholder) are presented in Table A1.13 in Annex 1.  

A  large  majority  of  stakeholders  consider  that,  relative  to  other  advisory  services, 
JASPERS’  added  value  is  higher  especially  as  regards  its  support  for  the  alignment 
with  EU  priorities,  the  potential  for  knowledge  transfer  among  Member  States,  its 
hands-on experience, and its technical expertise.  

From  the  replies  to  this  question  we  learn  that  a  large  majority  (more  than  80%)  of 
stakeholders  appreciate  a  higher  added  value  for  JASPERS  services  especially  with 
respect  to  their  alignment  with  EU  requirements,  the  potential  of  knowledge  transfer 
from  other  Member  States,  its  hands-on  expertise  with  projects,  and  the  quality  of  its 
technical expertise. Slightly more heterogeneous opinions were stated for aspects such as 

166  The online consultation includes also a probing question, whereby stakeholders are asked about their familiarity 
and use of other advisory services such as the Structural Reform Support Service of the Commission, other EIB 
advisory  services,  the  World  Bank  advisory,  EBRD  advisory,  and  private  consultancy  services.  A  large  of 
majority of 90% of respondents stated that they had used at least one of these advisory services, and therefore the 
replies to the following questions are likely to be informed by their direct experience.  Table  A1.12 in Annex 1 
presents the details.   

177 

 
 
 
 
 
 
 
 
 
 
                                                 
flexibility in responding to needs and expertise on administrative and procedural issues. 
Also for costs it appears that that up to 60% of stakeholders with an opinion considered 
that  JASPERS  services  had  higher  added  value  relative  to  other  advisory  services,  
although in this case it is less clear how the respondents interpreted the question as, when 
compared  with  other  advisory  services,  JASPERS  support  is  free  of  charge  for  the 
beneficiary Member State.  

A  large  majority  of  Managing  Authorities  and  project  beneficiaries/  promoters 
consider that JASPERS’ services will still be needed in the future. Willingness to pay 
from own technical assistance for these services, however, is rather low for both types 
of stakeholders.  

Further, stakeholders were also asked to express their opinions on the need for JASPERS 
services  in  the  future.  The  question  asked  and  the  distribution  of  the  replies  from  the 
Managing Authorities and project beneficiaries/ promoters are reported in Table A1.14 in 
Annex 1.  

In this regard, we learnt that at least 70% of the Managing Authorities responding to the 
consultation  consider  that  JASPERS  services  will  still  be  needed  in  the  future,  even  in 
case when the Commission’s decision is no longer necessary for the approval of major or 
strategic  projects.  They  also  consider  that,  relative  to  the  review  function  or  horizontal 
advice,  the  advisory  support  will  still  be  needed  in  the  future,  at  least  in  some  areas. 
When asked about their willingness to pay for JASPERS services from own resources for 
technical assistance, however, only 19% agreed they would be willing to do so, while the 
response rate to the question was also lower.  

Similar  replies  to  this  question  were  provided  also  by  the  project  beneficiaries/ 
promoters.  Compared  to  the  Managing  Authorities,  even  relatively  more  respondents 
among project beneficiaries/ promoters agreed that JASPERS support will be needed in 
the future at  least  in  some areas, and that it will also  be needed  for assistance with  the 
alignment  of  national  strategies  to  the  EU  sectoral  policy  priorities.  The  willingness  to 
pay  for  JASPERS  services  stated  by  project  beneficiaries/  promoters  settled  at  28%  of 
respondents, while the response rate was lower also for this type of stakeholders.  

Further  evidence  from  sampled  assignments  indicates  an  appreciation  of  JASPERS’ 
added  value  especially  in  terms  of  its  technical  expertise  and  EU-wide  experience 
acquired while working in many countries, and with the European Commission.  

In addition, in the context of the in-depth analysis of sampled assignments from JAPERS 
portfolio  and  for  the  sampled  countries,  stakeholders  of  the  respective  projects 
(Managing Authorities, beneficiaries and staff from the European Commission) were also 
interviewed about their opinion on the added value of JASPERS services. The results are 

178 

 
 
 
 
 
 
 
 
very  similar  to  the  findings  from  the  online  consultation,  in  that  stakeholders  tend  to 
appreciate  JASPERS’  services  for  their  technical  expertise,  for  their  experience  with 
similar projects in other Member States, and for their experience in working directly with 
the European Commission for the alignment with EU priorities and project appraisal. In 
this  context,  stakeholders  explained  that,  in  many  respects,  JASPERS’  services  are 
complementary  to  other  advisory  services  available  on  the  market.  Compared  with  the 
private  consultancy  services,  for  example,  they  explained  that  JASPERS’s  role  is  to 
guide  and  supervise  the  work  of  private  experts  on  project  analysis  and  documentation 
such  as  cost  benefit  analysis  and  feasibility  studies.  JASPERS  often  assists  the 
beneficiaries  in  defining  the  terms  of  reference  for  contracting  private  consultancy 
services. Further, stakeholders also opined that the quality of JASPERS support is very 
difficult  to  replace  with  local  expertise  due  to  their  acquired  extensive  methodological 
expertise and sector experience across Member States – features less easily found on the 
local  market.  Finally,  from  the  operational  perspective,  the  stakeholders  often 
emphasized  that  the  use  of  JASPERS  services  does  not  require  public  procurement 
procedures, which is considered an advantage. 

From  the  staff  of  the  European  Commission  interviewed  for  this  evaluation,  we  learnt 
that they appreciate the high technical expertise of JASPERS’ experts especially for more 
complex  technical  issues  in  the  projects.  As  regards  the  PSA  service  for  the  review 
function,  they  recognised  the  visible  improvement  in  the  timeline  of  the  approval  of 
projects, which is enabled also by the effective project appraisal provided by JASPERS. 
In  addition,  the  use  of  the  same  service  for  the  appraisal  of  all  projects  helps  ensure 
consistency in the decision making process.  

Finally,  for  the  analysis  of  the  added  value,  we  also  considered  JASPERS’  own 
assessment on the added value to the projects assisted. For this purpose, we analysed the 
value added fiches issued by JASPERS upon completion of an assignment. In the fiches, 
JASPERS’ experts provide their views on a number of aspects of value added for their 
assignments167. We carried out this analysis for 30 non-major projects completed in the 
six  countries  sampled for this evaluation,  and learnt that, at  least  for these projects,  the 
most  frequent  aspects  of  value  added  chosen  by  JASPERS’  experts  refer  to 
improvements  in  project  quality,  increased  capacity  of  the  beneficiary  and  assistance 
with the removal of barriers to realize projects. The least frequent aspect of value added 
mentioned in the fiches is the facilitation of implementation. 

In sum, based on the feedback received from the stakeholders surveyed and interviewed, 
we conclude that the EU added value of JASPERS services is reflected primarily by their 
high  technical  expertise  informed  also  by  experience  with  projects  across  sectors  and 

167 

In the value added fiche, these aspects are classified as follows: increased capacity of counterparts, contribution to 
sector  strategy/  planning,  contribution  to  removal  of  barriers  to  realise  projects,  cost  savings,  improvements  of 
project quality, facilitation of project implementation, and other. 

179 

 
 
  
 
 
                                                 
Member States, by their experience in working with the European Commission, and by 
their ability to explain and support the alignment of national projects and strategies to EU 
policy priorities and standards.   

6. CONCLUSIONS 

In  this  evaluation  we  assessed  the  implementation  of  JASPERS’  services  and  its 
performance over the period 2014-2018 according to five evaluation criteria: relevance, 
effectiveness, efficiency, coherence, and EU added value. In the following paragraphs we 
summarize and discuss the main evaluation findings first succinctly at portfolio level, and 
then more in detail by mandate and type of JASPERS’ service.  

JASPERS’ portfolio of services 

At  portfolio  level,  relative  to  the  previous  period,  we  found  that  JASPERS’  activities 
expanded  both  in  scope  and  volume  during  20014-2018.  In  terms  of  scope,  while  the 
ESIF  mandate  has  the  highest  share  (close  to  90%)  of  the  overall  portfolio  in  terms  of 
numbers  of  assignments,  JASPERS’  services  are  now  supported  by  two  more 
Commission services, as DG MOVE and DG NEAR joined DG REGIO in financing the 
initiative  for  CEF  and  IPA  projects.  JASPERS’  services  are  now  available  for  all 
Member  States  and  IPA  countries.  Further  for  scope,  JASPERS  has  also  diversified  its 
portfolio by including a new function for the quality review  of ESIF major projects.  In 
volume, during the first five years of the current period JASPERS has already received 
more requests for service than in the entire period 2007-2013. Among these services, we 
noted  especially  an  increase  in  activities  for  capacity  building  through  multi-country 
trainings and workshops developed through JASPERS’ Networking Platform.  

Technical advisory for ESIF major projects 

For  technical  assistance  for  the  preparation  of  ESIF  major  projects,  we  found  that 
JASPERS’ advisory services remained relevant during the current period, as reflected by 
more  than  400  assignments  requested  by  the  Member  States  until  December  2018. 
Almost one year later, we learnt that 70% of these assignments resulted in major projects 
adopted by the Commission – projects which mobilized almost 24 billion euro worth of 
investments from ERDF and Cohesion Fund.  

When  compared  with  the  previous  programming  period  2007-2013,  we  find  that 
JASPERS’ portfolio has a very similar profile in 2014-2020 as regards the high (close to 
60%)  concentration    of  assignments  in  a  few  countries,  and  with  technical  assistance 
provided  to  larger  projects  (compared  with  the  non-assisted  projects).  As  regards  the 
distribution  across  sectors,  the  share  of  the  transport  sectors  in  terms  of  number  of 

180 

 
 
 
 
 
 
 
assignments is now highest in the portfolio, with the weight of the sector for water and 
wastewater treatment reduced by 11%.  

For  the  coverage  of  major  projects  by  JASPERS’  advisory  support,  relative  to  the 
previous  period,  the  share  of  projects  assisted  by  JASPERS  in  all  major  projects 
submitted to the Commission has increased from 50% to 58%. This is due to the fact that 
many Member States use JASPERS’ advisory service for a large majority of their major 
projects.  Only  two  Member  States  reduced  their  relative  reliance  on  technical  advisory 
for project preparation – Poland and, to a lesser extent, Czechia.  

In  terms  of  the  nature  of  support,  when  assisting  the  preparation  of  major  projects, 
JASPERS’  advisory  services  provide  guidance  and  supervision  most  frequently  for  the 
preparation  of  the  feasibility  study  and  cost  benefit  analysis  –  studies  which  are 
regulatory  requirements  for  accessing  EU  funding  for  major  projects  for  EU  Cohesion 
Policy. Further, we learnt also that the extent of JASPERS’ support across topics depends 
on its timing, with more extensive support provided in assignments requested early in the 
process of project preparation. 

The  timing  of  JASPERS’  involvement  in  project  preparation  for  major  projects  has  a 
significant effect on its effectiveness. First, the evidence indicates that, when compared 
to non-assisted projects, the projects assisted by JASPERS advisory have more chances 
to  be  adopted  without  interruptions  for  quality  issues.  Second,  JASPERS  advisory  is 
positively  correlated  with  a  shorter  duration  of  project  preparation  and  with  faster 
financial  implementation  of  the  projects  subsequent  to  their  approval.    The  effects  on 
project  duration  and  implementation,  however,  are  statistically  significant  only  for 
projects  which  had  been  supported  early  in  the  preparatory  stage.  According  to  our 
estimation, these early assignments are likely to cover around 40% of the major projects 
assisted168. For the remaining projects, assistance is likely to be requested at later stages 
of project preparation, or even at the application stage, with reduced effectiveness.  

Even  if,  ceteris  paribus,  the  projects  assisted  by  JASPERS  have  a  higher  chance  to  be 
approved  by  the  Commission  without  interruptions  for  quality  issues,  there  is  a  non-
negligible  proportion  (48%)  of  these  projects  submitted  for  Art  102.2.  which  are  still 
interrupted. We identify several reasons why this can happen. In some cases, the topics 
for  which  projects  are  interrupted  had  not  been  covered  by  JASPERS’  advisory 
assignments,  while  in  some  other  cases  the  Member  States  did  not  follow  up  on  the 
recommendations  for  improvement  indicated  by  JASPERS’  advisory  during  the 
preparation  of  projects.  Nevertheless,  there  is  also  evidence  that  the  effectiveness  of 
JASPERS’ advisory in signalling issues with project quality at the preparatory stage also 
varies  across  topics.  It  is  highest  for  the  cost  benefit  analysis  and  the  feasibility  study, 
and lower for topics such as indicators, project description and project timetable. Part of 

168  This estimate applies to major projects submitted for Art.102.2 and assisted by JASPERS advisory. 

181 

 
 
 
 
 
 
                                                 
this  more  limited  effectiveness  is  also  the  fact  that  sometimes  the  experts’  message  on 
certain aspects of the project is not sufficiently strong in order to signal effectively a need 
for change. 

Overall,  however,  when  compared  with  non-assisted  projects,  the  projects  assisted  by 
JASPERS  advisory  have  fewer  critical  issues  of  quality  raised  during  the  approval 
process in the Commission, while the period of interruption is rather comparable between 
the two groups of projects.  

Technical advisory for ESIF non-major projects 

Also  for  ESIF  non-major  project  the  relevance  of  advisory  support  is  indicated  by  the 
number of requests for such assistance received from the Member States. During the first 
five years of the current period, JASPERS created at least as many assignments for ESIF 
non-major projects as for the entire period 2007-2013.  

As regards the profile of the portfolio for non-major projects, for the distribution across 
countries,  also  for  non-major  projects  there  is  a  high  concentration  (around  70%)  of 
assignments across 3 main beneficiary countries, and this is roughly similar in magnitude 
with the experience from the previous period (although the countries concerned are not 
exactly the same). As regards the distribution across sectors, we noted an increase in the 
share of assignments for transport and a reduction in the share of the sector for water and 
wastewater  treatment,  while  the  latter  still  maintains  the  highest  proportion  of 
assignments across sectors.   

Further  for  non-major  projects,  the  evidence  gathered  points  towards  an  issue  with  the 
prioritisation  of  these  assignments.  While  the  initial  intention  with  the  JASPERS’ 
mandate for this type of service was to focus the activities on countries with fewer major 
projects, or on pilot projects and projects that could be replicated on a much wider scale 
in  the  country,  there  is  little  evidence  that  the  assignments  implemented  for  non-major 
projects were aligned with these priorities over the reference period.   

For the nature of JASPERS’ assistance, also for non-major projects, JASPERS advisory 
contributes  most  frequently  with  guidance  and  advice  for  the  feasibility  study  and  cost 
benefit analysis. In terms of issues still outstanding by the time the advisory assignment 
is  finalised,  JASPERS’  comments  for  improvements  are  most  often  related  to 
procurement and implementation, cost benefit analysis, and risk and sensitivity analysis.  

When compared to major projects, however, the possibility to evaluate the effectiveness 
of the advisory support provided to ESIF non-major projects was much more limited due 
to  difficulties  encountered  with  the  data  collection  for  these  projects.  For  example,  the 
non-major  projects  assisted  by  JASPERS  had  a  rate  of  success  in  the  range  50-58% 

182 

 
 
  
 
 
 
 
 
during  the  approval  process  with  the  Managing  Authorities  in  two  countries,  but  we 
cannot  generalise  this  result  since  the  data  collection  for  other  Member  States  did  not 
prove feasible. In this regard, over the reference period for this evaluation, JASPERS had 
very little follow up information on non-major projects assisted as it did not collect any 
data on what happened to them once the advisory work was completed. In our view, this 
would  be  necessary  in  the  future  in  order  to  have  the  possibility  to  assess  the 
effectiveness  of  such  assignments  and  identify  the  scope  for  a  better  calibration  of  this 
service.  

Technical advisory for CEF projects 

JASPERS’  technical  assistance  for  CEF  projects  was  initiated  as  a  pilot  in  the  current 
programming period. It started with a first mandate (CEF1.1) valid until March 2018, and 
it  continued  with  a  second  mandate  (CEF1.2)  until  the  end  of  2021.  This  evaluation 
covered primarily the first mandate, which applied to projects prepared with the intention 
to submit them for CEF competitions for cohesion countries.  

Within  the  scope  of  JASPERS’  first  CEF  mandate  until  2018,  JASPERS  supported  29 
CEF  projects  in  6  cohesion  countries,  with  the  number  of  project  supported  being 
determined by the budget allocated for this pilot. 

As regards the types of CEF projects supported by JASPERS advisory,  when compared 
with  the  non-assisted  CEF  projects,  the  assisted  projects  were  more  concentrated  on 
works (rather than studies), and tended to be much larger in terms of total eligible costs. 

Still for the CEF projects included in the first mandate, we concluded that, while 80% of 
the  supported  projects  proved  successful  in  CEF  competitions,  the  probability  that  the 
advisory had a significant effect on project quality beyond project documentation through 
these  assignments  is  not  high.  This  is  due  to  the  fact  that,  for  a  majority  of  them,  the 
advisory support was requested primarily around the time of CEF  competitions and the 
subsequent assessment of applications. JASPERS informed us, however, that in some of 
these cases the projects had been assisted by their advisory service for project preparation 
during  the  previous  programming  period,  at  a  time  when  the  beneficiaries  were 
considering  submitting  the  projects  for  financing  from  the  EU  Cohesion  Policy. 
Therefore,  we  can  infer  that,  at  least  for  these  cases,  the  incentive  to  request  further 
extensive support during the current period was reduced.  

Finally, during the final stages of this evaluation we also learnt that, following the CEF 
pilot  experience  with  JASPERS’  assistance,  DG  MOVE  has  decided  to  support  a 
renewed  mandate  for  JASPERS’  services  for  CEF  projects  during  the  programming 
period  2021-2027  –  a  mandate  with  a  significantly  higher  budget  and  therefore  higher 
potential for more extensive assistance to projects in all Member States. In this context, 

183 

 
 
 
 
 
 
 
JASPERS  assistance  for  CEF  projects  will  be  expected  to  prioritise  early  phases  of 
planning and project preparation in order to ensure maximum added-value.  

184 

 
 
Technical advisory for IPA projects 

The  IPA  mandate  was  piloted  during  the  last  years  of  the  previous  period  and 
subsequently developed into a fully-fledged line of advisory service in the current period. 
During the reference period 2014-2018, JASPERS received requests for advisory support 
for  around  70  assignments.  Of  these  assignments,  17  resulted  in  projects  approved  for 
IPA  funding,  representing  7%  of  all  IPA  projects  approved  until  summer  2019. 
Therefore, also for this mandate, we concluded that the scope for advisory services was 
lower than in the case of ESIF major projects. This is also reflected by the much lower 
budget allocated for this type of service within the overall budget of JASPERS.  

In terms of geographical and sectoral distribution of the advisory work for IPA projects, 
more than half of the assignments are concentrated in two countries, and they are most 
frequently  requested  for  the  sector  of  water  and  wastewater  (45%  of  all  IPA  advisory 
assignments).  

For the nature of support provided, in contrast to the mandates for ESIF and CEF where 
the  focus  is  primarily  on  investment  projects,  the  IPA  projects  supported  by  JASPERS 
and approved are mostly for technical assistance rather than works. This is in line with 
the overall distribution of IPA projects, whereby a large majority of them are financed for 
technical assistance intended to support the administrative capacity in these countries.  

As  regards  effectiveness,  based  on  an  analysis  of  sampled  assignments,  also  for  this 
mandate we found indication that the effects of JASPERS’ advisory support in terms of 
project  quality  and  beyond  improvements  in  project  documentation  are  likely  to  be 
limited, and for similar reasons. This analysis indicates that the requests for support came 
late in the process of project preparation, and mostly at application stage.  

In sum,  as regards the relevance  and effectiveness  of JASPERS’ advisory  function, we 
conclude that, while this line of service remains relevant especially for the ESIF mandate, 
there is also scope for further reflection on a more strategic prioritisation of this service. 
For  all  mandates,  JASPERS’  contribution  is  likely  to  be  effective  when  its  support  is 
started  at  early  stages  of  project  preparation  –  a  practice  which  is  not  yet  dominant  in 
JASPERS’ portfolio. Moreover, for the ESIF non-major projects, the support should be 
more  focused  on  the  priorities  established  initially  for  this  type  of  service.  We  also 
identified  a  need  for  better  monitoring  of  the  effectiveness  of  this  type  of  service  over 
time.  

185 

 
 
 
 
 
 
 
 
 
ESIF review function 

Introduced to respond to the new regulatory requirements for the approval of ESIF major 
projects  for EU  financing, the new review function for the ESIF mandate has proven a 
valuable support in the Commission’s decision making process for the approval of major 
projects. Overall, relative to 2007-2013, there has been a significant reduction (50%) in 
the  time  needed  for  the  approval  of  major  projects  –  a  process  in  which  JASPERS 
provides timely and effective support for project assessments.  

Of  the  two  main  strands  of  the  review  function  (IQR  and  PSA),  we  could  evaluate  the 
PSA process extensively due to the possibility to collect structured data on process and 
performance.  On  this  basis,  we  concluded  that  JASPERS  IQR  is  highly  effective  in 
identifying critical issues in projects submitted for EU financing (when applicable), and 
in providing its assessment to the Commission in a timely manner. This is reflected by 
the  close  correspondence  between  the  observations  raised  by  the  Commission  in  its 
interruption letters to the Member States and JASPERS’ PSA reports, by the active role 
of JASPERS IQR in the screening meetings with the Commission services, and by their 
to  Commission’s  requests  for  post-submission 
systematically  prompt  response 
assessments.  

As  regards  the  IQR  strand  of  the  review  function  -  it  also  contributes  to  an  overall 
reduction in the time needed for the assessment and approval of major projects, but we 
could  not  evaluate  further  its  effectiveness  in  terms  of  critical  issues  identified  for  the 
projects assessed. This is due to the fact that, over the reference period, the IQR process 
was  organised  in  a  much  less  formal  way  and  there  was  little  scope  for  collection  of 
structured data in this regard. In terms of more recent developments, in its comments for 
this  evaluation,  JASPERS  IQR  informed  us  that,  starting  with  late  2018,  also  the  IQR 
process has been formalised by introducing a structured template for its first assessments 
of projects submitted for the IQR quality review.  

Capacity building and horizontal and strategic support 

While  this  line  of  service  was  present  in  JASPERS’  portfolio  also  during  the 
programming  period  2007-2013,  for  the  current  period  it  has  expanded,  especially  as 
regards activities for trainings and workshops on topics highly relevant for EU Cohesion 
Policy. Between the two periods, the share of such assignments has  grown from  4% to 
25% in JASPERS’ portfolio of activities for capacity building, and primarily for the ESIF 
mandate.  

The  assignments  for  training  and  workshop  activities  are  organised  most  frequently 
(close  to  60%)  in  a  multi-country  format,  often  in  cooperation  with  Commission  staff, 
and for topics such as climate change adaptation and mitigation, environmental impacts, 
state aid, and cost benefit analysis. For country specific trainings, on the other hand, we 

186 

 
 
 
 
 
 
 
concluded  that  the  demand  was  more  limited  over  the  reference  period,  as  reflected  by 
the  relatively  lower  number  of  such  assignments  requested  by  the  Member  States  and 
their distribution across countries.  

As  regards  the  assignments  for  horizontal  and  strategic  support,  they  were  provided 
primarily  for  the  ESIF  and  the  IPA  mandates,  and  they  were  more  geared  towards 
country  specific  support  of  horizontal  nature  such  as  sectoral  analysis,  preparation  of 
project pipeline, preparation of operational programmes etc. For the assistance with more 
strategic  issues,  approximately  30%,  this  type  of  support  was  requested  by  Member 
States  for  preparing  national  and  regional  strategies  for  investments  for  EU  Cohesion 
Policy. 

Geographically,  based  on  the  distribution  of  country  specific  activities  for  capacity 
building  (including  both  training  activities  and  horizontal  assignments),  there  is  a  high 
concentration  of  activities  (60%)  in  three  Member  States  for  the  ESIF  mandate  –  a 
situation similar with the advisory function. 

In  terms  of  timing,  two  thirds  of  these  assignments  had  started  at  the  beginning  of  (or 
before)  the  programming  period  2014-2020,  as  they  were  meant  to  support  the 
preparation for the upcoming period.   

As regards effectiveness, we concluded that, for a large majority of activities for capacity 
building and horizontal and strategic support, their specific objectives at assignment level 
had been achieved. Nevertheless, we could not find any convincing evidence that, over a 
medium  to  longer  term,  these  activities  also  had  a  noticeable  ripple  effect  in  terms  of 
strengthened  technical  capacity  in  beneficiary  countries  for  identification,  preparation 
and  submission  of  projects  for  EU  funding.  On  the  contrary,  in  our  assessment,  the 
continuing  high  reliance  on  JASPERS’  advisory  assistance  across  many  of  the 
beneficiary  countries,  as  well  as  the  frequency  of  quality  issues  still  occurring  in  the 
process  of  approval  of  major  projects,  indicate  a  rather  limited  impact  of  capacity 
building  activities  on  enhanced  relevant  technical  capabilities  in  the  beneficiary 
countries.  

In  this  respect,  however,  we  also  noticed  what  appeared  to  be  a  paradox  for  the 
evaluation.  On  the  one  hand,  we  found  that  the  evidence  collected  did  not  support  the 
hypothesis  of  significant  impacts  of  JASPERS’  activities  on  technical  capacity  in  the 
beneficiary countries. On the other hand, however, we learnt from stakeholder interviews 
and  from  the  targeted  online  consultation  that  there  is  a  widespread  perception  among 
stakeholders 
technical  capacity. 
Stakeholders  also  tend  to  emphasize  the  value  of  JASPERS’  hands-on  support  for 
technical capacity during the assignments (relative to training).  

that  JASPERS’  support  does  help  strengthen 

187 

 
 
 
 
  
 
 
 
In  our  view,  this  apparent  paradox  can  be  explained  by  the  fact  that,  while  JASPERS’ 
support  does  help  alleviate  the  lack  of  sufficient  technical  capacity  in  the  beneficiary 
countries, this approach does not prove sufficient in ensuring a long lasting effect at local 
and national levels. This can be due to several factors. First, as regards the deployment of 
JASPERS’ services, there is no substantive evidence of an overall strategic approach for 
building  relevant  technical  capacity  across  Member  States  over  the  reference  period. 
Second, we learnt that, while carrying out  the assignments, JASPERS often guides and 
supervises the work of service providers for technical assistance – an approach which has 
less potential for a long term impact on capabilities in national authorities in charge with 
the  management  of  EU  funds.  Finally,  we  also  found  that  staff  turnover  in  national 
administrations  is  sometimes  also  a  factor  not  conducive  to  long  term  building  of 
technical  capacity  at  national  level.  In  reply  to  our  analysis,  Jaspers  explained  that,  for 
the future, the scope and the strategic approach for activities for capacity building will be 
strengthened.  

Nevertheless,  as  explained  next  for  the  overall  stakeholders’  perceptions  of  JASPERS’ 
services, we learnt that an undisputed and appreciated contribution of JASPERS with this 
type of service is their ability to explain and inspire the alignment of projects with the EU 
requirements and standards.  

Finally,  as  regards  support  for  capacity  building,  we  found  also  that  JASPERS  has 
proven potential for a more strategic and innovative approach to capacity building. Good 
practice  includes  examples  of  strategic  planning  of  such  activities  at  country  level,  of 
training  activities  organised  in  the  “train  the  trainers”  format  with  a  high  potential  for 
knowledge dissemination, and of effective organisation methods applied when organising 
training activities on topics of relevance for several countries. Therefore, we concluded 
that,  while  not  yet  mainstreamed  at  the  portfolio  level,  such  practices  have  a  high 
potential  for  a  more  impactful  support  for  technical  capacity.  In  terms  of  more  recent 
developments, in its comments to this evaluation, JASPERS informed us that efforts to 
mainstream such practices at portfolio level are currently under way.  

Stakeholders’ views on the relevance and effectiveness of JASPERS’ services 

For  this  evaluation,  stakeholders’  feedback  was  gathered  systematically  through  the 
online targeted consultation – a survey for which we gathered 210 replies from Managing 
Authorities,  projects  promoters  and  beneficiaries,  professionals  from  consultancy 
companies, and colleagues from  the Commission169. Additional  opinions on JASPERS’ 
activities  were  gathered  through  interviews  for  the  background  analysis  of  sampled 
assignments and countries.  

169  With this occasion, we would like to express our gratitude to all stakeholders in Member States and pre-accession 

countries, and to our Commission colleagues, who cooperated for this evaluation.  

188 

 
 
 
 
 
 
                                                 
On this basis, we learnt that JASPERS’ services are highly appreciated by a majority of 
stakeholders with varying experiences across mandates and types of service. Almost all 
of  these  stakeholders  consider  JASPERS  as  relevant  or  highly  relevant  for  providing 
technical support for increasing project quality, for assisting with project alignment with 
EU  requirements  and  standards,  and  for  enabling  the  development  of  skills  for  cost 
benefit  analysis  in  line  with  the  EU  guidelines.  Moreover,  the  stakeholders  surveyed 
consider that JASPERS has a high ability to respond to needs related to these aspects of 
project  preparation.  More  heterogeneous  replies  in  this  regard  were  noticed  for  issues 
related  to  compliance  with  EU  public  procurement  and  assistance  for  project 
implementation.  

Further, as regards the effects of JASPERS’ services on a variety of aspects, more than 
80% of stakeholders opined that JASPERS had a significant, and even decisive, effect on 
the  technical  specifications  and  documentation  of  projects.  They  also  rated  highly 
JASPERS’  effects  on  alignment  of  investment  projects  with  EU  and  national  policies, 
and  on  faster  project  preparation  and  approval.  More  heterogeneous  opinions  among 
stakeholders  were  expressed  as  regards  JASPERS’  effects  on  achieving  cost  savings  at 
project level, and as regards project implementation.  

Finally,  as  regards  customer  satisfaction,  a  very  large  majority  of  stakeholders  rated 
highly  JASPERS’  services  in  terms  of  their  consistency  with  EU  requirements  and 
standards, quality of advice, effectiveness and timeliness. 

Efficiency of planning and work allocation 

For the criterion of efficiency, the evaluation focused primarily on the planning process 
of  JASPERS’  work,  on  the  rate  of  completion  of  assignments  and  administrative 
closures,  on  the  allocation  of  actual  work  effort  during  the  calendar  duration  of 
assignments, and on the dynamics over time of human and financial resources allocated 
to the initiative.  

In terms of planning, we found that, similar to the situation in the previous programming 
period,  JASPERS’  portfolio  of  activities  develops  in  line  with  the  requests  from  the 
Member States. Assignments are included in the planning as they arrive, without a clear 
process  of prioritisation.  As explained by type of service above, there is  little evidence 
for a strategic approach in focusing the deployment of JASPERS’ services for activities 
with  highest  potential  leverage  in  terms  of  added  value  for  the  management  and 
implementation  of EU funds at  country level.  For example, this is  reflected by the fact 
that concentration of assignments in a few beneficiary countries emerges systematically 
across  periods,  mandates  and  types  of  services.  In  our  view,  for  such  countries  which 
tend  to  have  a  high  demand  for  all  types  of  JASPRS’  services,  a  more  strategic  and 
selective  approach  in  aligning  capacity  building  activities  and  advisory  support  could 

189 

 
 
 
 
 
 
have a higher potential for a more effective technical assistance at country level, possibly 
reducing also their reliance on assistance on longer term.  

This  approach  in  planning  also  has  consequences  for  the  evolution  of  JASPERS’  work 
over  time,  as  reflected  by  several  developments.  First,  we  found  that  the  proportion  of 
assignments  which  had  to  be  administratively  closed  without  fully  achieving  their 
objectives  (or  even  not  at  all)  is  relatively  high  in  a  number  of  Member  States.  This 
happens  most  likely  because  the  initial  planning  for  such  assignments  was  not 
sufficiently  defined,  and  they  were  included  in  the  country  action  plans  without  clear 
expected durations and milestones for monitoring the progress in the interim. As a result, 
they were protracted for long periods of time until a decision was eventually taken in the 
Member State to close them.  

Second, for the assignments which achieved their objectives, we found that a majority of 
them were delayed relative to their initial planning, and had to be extended for significant 
periods  of  time.  In  their  comments  to  this  evaluation,  JASPERS  explained  that  the 
assignments  for  advisory  usually  follow  the  timeline  of  projects  established  in  the 
Member  States.  Therefore,  the  delays  experienced  are  primarily  due  to  unexpected 
developments  in  the  Member  States  which  delay  the  projects  themselves.  Nonetheless, 
while  we  see  the  merit  of  this  argument,  we  also  consider,  however,  that  a  more 
structured  approach  to  planning,  with  an  effective  mechanism  that  ensures  timely 
implementation  of  assignments  would  benefit  the  overall  efficiency  of  the  initiative.  In 
this  regard,  in  terms  of  more  recent  developments,  JASPERS  informed  us  that  it  is 
currently  working  on  an  improved  system  for  planning  the  assignments  and  for 
monitoring the interim progress with the their implementation.  

And third, the impact of the tendency to accumulate assignments in JASPERS’ portfolio 
without  prioritisation  on  efficiency  can  also  be  inferred  from  the  analysis  of  the  work 
intensity per assignment. Based on a sample of assignments implemented over the period 
2017-2019,  we  learnt  that  the  work  intensity  per  assignment  relative  to  its  calendar 
duration varies significantly across assignments. We found that assignments for the PSA 
quality reviews are most efficient, as they are clearly targeted in terms of objectives and 
duration. For the remaining assignments, however, there is  significant  variability in  the 
work  intensity  per  calendar  duration  primarily  due  to  the  time  lags  in  the  interaction 
between JASPERS and their counterparts in the Member States, as explained above.  

As regards the assessment of the extent of the scope for improvement in the allocation of 
work  effort  across  assignments,  however,  we  could  not  produce  an  estimate  in  this 
evaluation  as  this  would  have  required  data  for  the  hours  worked  per  year  per 
assignment, for all assignments in JAPERS’ portfolio – data which was not available for 
the evaluation. We learnt, however, that, following the ECA performance audit published 
in 2018, JASPERS has engaged in a process of improving the monitoring system for its 

190 

 
 
 
 
 
 
activities,  including  also  tracking  the  work  effort  per  assignment.  Therefore,  in  our 
assessment, as more relevant monitoring data accumulates over time, an analysis of the 
distribution  of  the  work  effort  by  mandate,  type  of  service,  sector  and  country  would 
contribute significantly to identifying the scope and avenues for the optimisation of the 
planning and allocation of resources within the organisation.  

In terms of financial resources allocated to the initiative, relative to the previous period, 
the overall contribution of the Commission has increased from 71% to 81% of the total 
expenditure with the initiative, reaching close to 150 million euro for the three mandates 
over  the  period  2014-2018.  This  increase  in  financial  resources  has  a  direct 
correspondence  in  the  consolidation  of  the  pool  of  professional  expertise  employed  by 
JASPERS, with a higher emphasis on own expertise rather than contracted services. As 
regards the evolution  over time, the resources  allocated to  the initiative evolved in  line 
with the accumulation of assignments in JASPERS’ portfolio. 

Finally, as our awareness and knowledge of evolutions over time depend to a large extent 
on  whether  we  measure  them,  for  this  evaluation  we  conclude  also  that  the  experience 
with JASPERS’ development over time is valuable also as regards the realisation of the 
usefulness of a better-tuned monitoring system. During the course of this evaluation, we 
learnt that JASPERS’ processes and performance were gradually improved over time in 
line  with  efforts  to  improve  the  monitoring  system.  Although  we  found  evidence  that 
there is still some scope for further developments and  fine-tuning, we learnt that much 
has already been achieved especially starting with 2018. Data quality for the latest years 
has  improved,  thus  enabling  a  more  informed  and  better  implementation  of  processes, 
and  there  are  also  ongoing  efforts  for  calibrating  better  the  result  orientation  of 
JASPERS’ services. 

External and internal coherence 

With  respect  to  external  coherence,  the  evidence  collected  for  the  evaluation  indicates 
that JASPERS’ activities are fully coherent with EU requirements and standards for EU 
Cohesion  Policy,  and  with  the  EU  level  objectives  for  climate,  environment  and  EU 
competition policy. Its services contribute by guiding project preparation, by supporting 
the  project  assessments,  and  by  disseminating  knowledge  in  line  with  the  EU  policy 
priorities.  

As  regards  internal  coherence  of  JASPERS’s  services,  we  analysed  the  consistency  of 
opinions  issued  by  JASPERS  IQR  and  JASPERS  advisory  on  the  same  ESIF  major 
projects, and found that these opinions were broadly consistent between the two services. 
Nevertheless,  the  evidence  indicates  also  that  there  is  still  some  scope  for  further 
alignment for around 10% of the projects in terms of minimum regulatory requirements 

191 

 
 
 
 
 
 
for project quality, and for a larger number of projects for issues considered non-critical 
from the regulatory perspective.   

EU added value 

The  EU  added  value  of  JASPERS initiative  was  evaluated  based  on  feedback  gathered 
from  the stakeholders  surveyed with  the targeted online consultation.  On this basis, we 
learnt that,  relative to  other advisory services, JASPERS’s added value  is  most evident 
with  respect  to  the  alignment  with  EU  priorities,  the  potential  for  knowledge  transfer 
across Member States and pre-accession countries, and its hands-on technical experience. 
JASPERS’  experts  are  appreciated  especially  for  their  experience  accumulated  while 
working on a variety of projects across sector and Member States, and with the European 
Commission.  In  this  context,  stakeholders  interviewed  further  explained  that,  in  many 
respects, JASPERS’ services are complementary to other advisory services available on 
the  market,  considering  also  that  JASPERS’  EU  wide  experience  and  methodological 
expertise would be rather difficult to replace at local level. 

As  regards  the  need  for  JASPERS  in  the  future,  a  large  majority  of  the  stakeholders 
surveyed  considered  that  the  advisory  assistance  would  still  be  needed  in  the  future,  at 
least  in  some  areas.  Nevertheless,  when  asked  about  their  willingness  to  pay  for  these 
services from own technical assistance, only 19% of the Managing Authorities, and 28% 
of the projects beneficiaries / promoters, stated that they would be willing to do so.  

In conclusion, we consider that this evaluation of the implementation and performance of 
JASPERS  initiative  indicates  that  the  initiative  contributes  to  a  better  planning  and 
implementation  of  EU  funds,  it  helps  promote  EU  requirements  and  standards  across 
countries,  it  has  a  significant  potential  for  EU  added  value  due  to  its  cumulated 
knowledge  and  experience  across  the  Union,  and  it  is  highly  appreciated  by  its 
beneficiaries.  Further,  we  consider  that  the  initiative  can  work  better  by  calibrating 
further  its  effectiveness,  efficiency,  and  internal  coherence,  while  acknowledging  also 
that clear efforts in this regard are currently under way.  

192 

 
 
 
 
 
 
 
 
 
REFERENCES 

Legislative texts 

laying  down  common  provisions  on 

Regulation  (EU)  No  1303/2013  of  the  European  Parliament  and  of  the  Council  of  17 
the  European  Regional 
December  2013 
Development  Fund,  the  European  Social  Fund,  the  Cohesion  Fund,  the  European 
Agricultural for Rural Development and the European Maritime and Fisheries Fund and 
laying  down  general  provisions  on  the  the  European  Regional  Development  Fund,  the 
European  Social  Fund,  the  Cohesion  Fund  and  the  European  Maritime  and  Fisheries 
Fund  and  repealing  Council  Regulation  (EC)  No  1083/2006  (Common  Provisions 
Regulation) 

Regulation (EU) 2015/2017 of the European Parliament and of the Council of 25 June 
2015 on the European Fund for Strategic Investments, the European Investment Advisory 
Hub and the European Project Portal and amending Regulation (EU) No 1291/2013 and 
(EU) No. 1316/2013 (EFSI Regulation) 

Regulation (EU, Euratom) 2018/1046 of the European Parliament and of the Council of 
18  July  2018  on  the  financial  rules  applicable  to  the  general  budget  of  the  Union, 
amending  Regulations  (EU)  No  1296/2013,  (EU)  No  1301/2013,  (EU)  No  1303/2013, 
(EU) No 1304/2013, (EU) No 1309/2013, (EU) No 1316/2013, (EU) No 223/2014, (EU) 
No  283/2014,  and  Decision  No  541/2014/EU  and  repealing  Regulation  (EU,  Euratom9 
No. 966/2012 (Omnibus Regulation) 

Commission Delegated Regulation (EU) No 280/2014 of 3 March 2014 supplementing 
Regulation  (EU)  No.1303/2013  of  the  European  Parliament  and  of  the  Council  laying 
down  common  provisions  on  the  European  Regional  Development  Fund,  the  European 
Social Fund, the Cohesion Fund, the European Agricultural for Rural Development and 
the European Maritime and Fisheries Fund and laying down general provisions on the the 
European  Regional  Development  Fund,  the  European  Social  Fund,  the  Cohesion  Fund 
and the European Maritime and Fisheries Fund 

Commission  Implementing  Regulation  (EU)  No  1011/2014  of  22  September  2014 
laying  down  detailed  rules  for  implementing  Regulation  (EU)  No1303/2013  of  the 
European Parliament and of the Council as regards the models for submission of certain 
information  to  the  Commission  and  the  detailed  rules  concerning  the  exchanges  of 
information between beneficiaries and managing authorities, certifying authorities, audit 
authorities and intermediate bodies 

Commission  Implementing  Regulation  (EU)  2015/207  of  20  January  2015  laying 
down  detailed  rules  implementing  Regulation  (EU)  No  1303/2013  of  the  European 

193 

 
 
 
 
 
 
 
 
 
Parliament and the Council as regards the models for the progress report, submission of 
the information on a major project, the joing action plan, the implementation reports for 
the Investments for growth and jobs goal, the management declaration, the audit strategy, 
the  audit  opinion  and  the  annual  control  report  and  methodology  for  carrying  out  the 
cost-benefit  analysis  and  pursuant  to  Regulation  (EU)  No  1299/2013  of  the  European 
Parliament and of the Council as regards the model for the implementation reports for the 
European territorial cooperation goal  

Directive 2011/92/EU of the European Parliament  and of the Council of 13 December 
2011  on  the  assessment  of  the  effects  of  certain  public  and  private  projects  on  the 
environment (Text with EEA relevance)  

Council  Directive  92/43/EEC  of  21  May  1992  on  the  conservation  of  natural  habitats 
and of wild fauna and flora  

Other references cited in text 

European  Investment  Bank  (2010)  –  Evaluation  of  the  EIB  Role  in  the  JASPERS 
initiative, December 2010.  
(https://www.eib.org/en/publications/evaluation-of-the-eib-role-in-the-jaspers-initiative) 

DG  Regional  and  Urban  Policy  (2012)  –  JASPERS  Evaluation.  Final  Report  13 
December  2012,  evaluation  carried  out  by  AECOM  and  Economics  for  the  European 
Commission. 
(https://ec.europa.eu/regional_policy/en/information/publications/evaluations/2012/jasper
s-evaluation) 

Long,  Scott  J.  and  Freese,  Jeremy  (2013)  –  Regression  Models  for  Categorical 
Dependent Variables Using STATA, Stata Press Publication  

DG Regional and Urban Policy (2014)  – Guide to Cost-Benefit Analysis of Investment 
Projects.  Economic  appraisal 
tool  for  Cohesion  Policy  2014-2020,  European 
Commission, December 2014 
(https://ec.europa.eu/regional_policy/sources/docgener/studies/pdf/cba_guide.pdf ) 

Republic of Croatia State Audit Office (2017) – Impact of the JASPERS programme on 
submitted project proposals by the Republic of Croatia for co-financing from EU funds. 
Report on performance audit, Zagreb, May 2017 
(https://op.europa.eu/webpub/eca/synthesis-report-jaspers/en/) 

194 

 
 
 
 
 
 
 
 
 
 
 
European  Court  of  Auditors  (2018)  –  Joint  Assistance  to  Support  Projects  in  the 
European  Regions  (JASPERS)  –  time  for  better  targeting,  Special  Report  01,  2018 
(https://www.eca.europa.eu/Lists/ECADocuments/SR18_01/SR_JASPERS_EN.pdf) 

JASPERS (2018) -  Quality Manual, version 1.2, 2018 

European  Court  of  Auditors  (2020)  –  The  European  Investment  Advisory  Hub  – 
Launched  to  boost  investment  in  the  EU,  the  Hub’s  impact  remains  limited,  Special 
Report 12, 2020.  
(https://www.eca.europa.eu/en/Pages/DocItem.aspx?did=53644) 

195 

 
 
 
 
 
ANNEX 1: REPLIES TO THE ONLINE TARGETED CONSULTATION  

Figure A1.1 

Note : 210 respondents 
Source : Stakeholders’ consultation for JASPERS evaluation, June – September 2019 

Table A1.1 Respondents to stakeholders’ consultation by country 
Number 
respondents 

Country/ institution 

% in 
total 

European 
Commission 
Austria 
Bulgaria 
Croatia 
Czechia 
France 
Germany 
Greece 
Hungary 
Italy 
Latvia 
Lithuania 
Malta 
Montenegro 
Poland 
North Macedonia 

9 
2 
19 
14 
11 
1 
1 
3 
6 
1 
5 
5 
5 
5 
45 
9 

4% 
1% 
9% 
7% 
5% 
0.5% 
0.5% 
1% 
3% 
0.5% 
2% 
2% 
2% 
2% 
21% 
4% 

196 

 
 
 
 
 
Romania 
Serbia 
Slovakia 
Slovenia 
Spain 
Turkey 
United Kingdom 
Multi country 
Total 
Note : 210 respondents 
Source : Stakeholders’ consultation for JASPERS evaluation, June – September 2019 

17% 
2% 
8% 
1% 
2% 
2% 
0.5% 
0.5% 
100% 

35 
4 
17 
2 
5 
4 
1 
1 
210 

Figure A1.2 

Note : 210 respondents 
Source : Stakeholders’ consultation for JASPERS evaluation, June – September 2019 

Role of respondent 

Table A1.2  Respondents’ experience with JASPERS services, by role and service type 
Capacity 
building 
19 
15 
3 
13 
50 
42 

Managing Authority 
Project promoter/ beneficiary 
Consultancy providing TA 
Other 
Total 
of which, declaring ESIF 

Horizontal 
and strategic 
24 
23 
5 
9 
61 
52 

Non-major 
project 
29 
33 
6 
16 
84 
65 

Major 
project 
56 
80 
19 
27 
182 
149 

197 

 
 
 
 
 
experience 
Note : 210 respondents overall ; multiple answers possible 
Source : Stakeholders’ consultation for JASPERS evaluation, June – September 2019 

Table A1.3 Respondents’ experience with JASPERS services, by role of respondent and 
service 

Note : 210 respondents overall ; multiple answers possible 
Source : Stakeholders’ consultation for JASPERS evaluation, June – September 2019 

Table A1.4 Respondents’ experience with JASPERS services by role and topic of 
support 

Note : 210 respondents overall ; multiple answers possible 
Source : Stakeholders’ consultation for JASPERS evaluation, June – September 2019 

Table A1.5 : Respondents’ experience with frequency of interaction with JASPERS, by 
role  

Note : 210 respondents overall; multiple answers possible 
Source : Stakeholders’ consultation for JASPERS evaluation, June – September 2019 

198 

Role of respondentProject screeningProject preparationProject implementationCapacity buildingIQRPSAHorizontalManaging Authority2252181825921Project promoter/ beneficiary45842419282424Consultancy providing TA91936243Other1225101413610Total881805557684358of which, declaring ESIF experience741494648613951Role of respondentCBAPublic procurementState AidTechnical designRisk and sensitivity analysisEnvironmental assessmentCompliance checksReview applicationManaging Authority4712211931362533Project promoter/ beneficiary7931314862664254Consultancy providing TA1568417867Other244111515241419Total16553718612513487113of which, declaring ESIF experience1394169671091097595Role of respondentFrequent (more than 5 times)Occasional (up to 5 times)Only onceTotalManaging Authority36161062Project promoter/ beneficiary6820896Consultancy providing TA913224Other203528Total1335225210of which, declaring ESIF experience1133624173 
 
 
 
 
 
 
 
 
 
 
 
RELEVANCE 

Table A1.6 Reasons for asking for JASPERS TA’s support by type of respondent 

Note: 210 respondents overall, multiple answers possible 
Question: “Which were the motivations for asking / suggesting JASPERS’ support?” 
Source: Stakeholders’ consultation for JASPERS evaluation, June – September 2019 

199 

Role of respondentLack of skills project prepLack of skills application processStrengthen administrative capacityHigher chance for approvalFaster approvalUsing Jaspers is common practiceNot responsible for Jaspers decisionManaging Authority17132344391610Project promoter/ beneficiary30273562563326Consultancy providing TA33410868Other1010141514108Total6053761311176552of which, declaring ESIF experience4743661061005539 
 
 
 
Table A1.7  Respondents stating that JASPERS is relevant/ very relevant for needs by topic 

Note : 210 respondents 
Question : “Which of the following needs that may be addressed by JASPERS are relevant in your case ? “ Options for reply : Very relevant, Relevant, 
Not very relevant, Not relevant, No opinion. 
Source : Stakeholders’ consultation for JASPERS evaluation, June – September 2019. 

Table A1.8 Respondents’ opinions on JASPERS ability to respond to needs by topic 

Note : 210 respondents 
Question : “Based on your experience, to what extent do you think JASPERS was able to respond to the following needs.” Reply options: Very large 
extent, Large extent, Limited extent, Very limited extent, No opinion. 
Source : Stakeholders’ consultation for JASPERS evaluation, June – September 2019 

200 

Role of respondentSkills on CBACapacity prepare projectsCapacity review applications and CBACompliance EU State AidCompliance EU Public ProcurementCompliance EU environmental directivesCompliance project documentationSpeed up approval processIncrease quality of projectsStandards project documentationStrategies aligned with EU objectivesManaging Authority4339423623474148585034Project promoter/ beneficiary7978755952856674916969Consultancy providing TA2013191813151616221718Other2422221412221521251820All replies "relevant/very relevant"166152158127100169138159196154141% opinions stated88%80%85%71%57%90%76%86%97%80%78%No opinion rate10%10%11%14%16%10%13%12%3%8%14%Role of respondentDevelop skills for CBACapacity prepare projectsCapacity to implement projectsCapacity review applications and CBACompliance EU State AidCompliance EU Public ProcurementCompliance EU environmental directivesCompliance project documentationSpeed up approval processIncrease quality of projectsStandards project documentationStrategies aligned with EU objectivesManaging Authority403826393726464238585431Project promoter/ beneficiary797446726147817071836866Consultancy providing TA191512141916141414222116Other19191017137221621241917All replies "large/very large extent"1571469414213096163142144187162130%  "large/very large extent" in opinions stated89%85%63%85%80%62%88%80%80%94%86%80%% no opinion16%18%29%20%22%27%11%15%15%5%10%23% 
 
 
 
 
 
Table A1.9 Respondents’ satisfaction with JASPERS’ services by topic 

Note : 210 respondents  
Question : “Based on your experience, to what extent are you satisfied with the following aspects?” Reply options: Very large extent, Large extent, 
Limited extent, Very limited extent, No opinion. 
Source : Stakeholders’ consultation for JASPERS evaluation, June – September 2019 

201 

Role of respondentAchievement objectivesTimelinessQuality of advice and expertiseQuality deliverablesCoherence Jaspers support with ECEfficiency of JaspersManaging Authority514856524841Project promoter/ beneficiary877790858485Consultancy providing TA211921191819Other232123222220All replies "large/very large extent"182165190178172165%  "large/very large extent" in opinions stated92%84%93%93%94%88%% no opinion6%7%3%9%13%10% 
 
 
 
 
 
 
 
 
 
EFFECTIVENESS 

Table A1.10 Project related effects of JASPERS, by role of respondent and topic 

Note : 210 respondents  
Question : “Based on your experience, what do you think was the effect of JASPERS on the following aspects?” Reply options: Decisive, Significant, 
Limited, Neutral, Negative, No opinion. 
Source : Stakeholders’ consultation for JASPERS evaluation, June – September 2019 

202 

Respondent roleFaster project preparationFaster project approvalTechnically improved projectsCost savings in projectsImproved project documentationImproved project implementationFaster project implementationAlignment with EU and national policiesManaging Authority3437442250282218Project promoter/ beneficiary6766743975463638Consultancy providing TA151421122011817Other18192113199774All replies "decisive/ signficant"134136160861649473147% "decisive/ significant" in all opinions stated71%72%83%52%85%59%46%78%% "limited or neutral" in all opinions stated25%26%17%45%15%39%52%21%% no opinion10%10%8%21%8%25%24%10% 
 
 
 
Table A1.11 Strategic and horizontal effects of JASPERS, by role of respondent and topic 

Note : 210 respondents  
Question : “Based on your experience, what do you think was the effect of JASPERS on the following aspects?” Reply options: Decisive, Significant, Limited, 
Neutral, Negative, No opinion. 
Source : Stakeholders’ consultation for JASPERS evaluation, June – September 2019 

EU ADDED VALUE 
Table A1.12 Familiarity with and use of other advisory services 

Other advisory services 

Used the service 

Familiar but not used the service 

Total replies 

Structural Reform Support Service 
EIB advisory services (other than 
JASPERS) 
The World Bank advisory services 

11 

64 
51 

10 

41 
40 

203 

207 

207 
207 

Respondent roleIncreased administrative capacityFaster strategy development / planningImproved development of sector strategy / planningManaging Authority281820Project promoter/ beneficiary554548Consultancy providing TA131215Other161013All replies "decisive/ signficant"1128596% "decisive/ significant" in all opinions stated67%56%61%% "limited or neutral" in all opinions stated32%44%39%% no opinion20%28%25% 
 
 
 
 
EBRD advisory services 
Private consultacy services 
Note: Overall, 172 respondents used at least one of these services. Only 21 of the respondents are familiar with at least one of these services but used none.  
Question:  “Please  indicate  your  familiarity  with  alternative  support  schemes  (EIB  advisory,  World  Bank  advisory,  EBRD  advisory,  Private  consultancy, 
Structural  Reform  Support  Service).”  Reply  options:  I  am  familiar  with  the  following  alternative  scheme  (Yes/No),  I  have  benefitted  from  support  from  the 
following alternative scheme (Yes/No). 
Source : Stakeholders’ consultation for JASPERS evaluation, June – September 2019 

207 
207 

34 
151 

41 
25 

Table A1.13 Added value of JASPERS’ services in comparison with other advisory services 

Note : 207 respondents 
Question: “For the mandates and activities you are familiar with, what do you think is the added value of the following JASPERS services as compared to 
possible alternatives (e.g. private consultancy, in-house advisory, other)? Reply options: higher, same, lower, no opinion.   
Source : Stakeholders’ consultation for JASPERS evaluation, June – September 2019 

204 

Role respondentTechnical expertiseHands-on experienceKnowledge transfer from other MSKnowledge context of MSExpertise on administration and proceduresFlexibility in responding to needsAlignment with EU requirementsSynergies with other EU initiativesCostManaging Authority444043303028442721Project promoter/ beneficiary767776696756817242Consultancy providing TA17211617138171310Other21212216161421188All replies "higher added value"15815915713212610616313081% "higher added value" in all opinions stated82%83%84%73%68%55%86%79%59%% "lower added value" in all opinions stated2%2%2%3%5%11%0%1%19%% no opinion7%7%10%13%11%8%8%21%34% 
 
 
 
Table A1.14  Opinions on the need for JASPERS services in the future and willingness to 
pay 

Note :  Respondents  include  62  Managing  Authorities  and  96  beneficiaries/  project 
promoters.  Numbers  reported  in  cells:  proportion  of  replies  “agree  to  a  large  of  very 
large extent”, and number of all opinions in parantheses.   
Question: “To what extent do you agree with the following statements?” Reply options: 
very large extent, large extent, limited extent, very limited extent. 
Source : Stakeholders’ consultation for JASPERS evaluation, June – September 2019 

205 

StatementManaging Authority% agree (number all replies)Beneficiary / Project promoter% agree (number all replies)We will need JASPERS support less because we have developed own internal capacity24% (51)23% (86)Despite improvements in internal capacity, there are still areas where JASPERS support is needed86% (50)92% (84)Assistance by JASPERS will be less relevant with respect to technical assistance and more relevant for review and horizontal support34% (41)33% (80)In case an approval by the European Commission would no longer be needed for major or strategic projects, JASPERS support would no longer be needed30% (53)23% (83)JASPERS will be relevant for contributing to ensure the alignment of national strategies to EU sectoral policy priorities70% (43)89% (79)In the future we would be prepared to pay for JASPERS support using own resources for technical assistance19% (31)28% (60) 
 
 
 
 
ANNEX 2: DATA AND REGRESSION MODELS 

A2.1 Data collected for the evaluation 

Table A2.1.1 :  Summary of data for relevance 

206 

RELEVANCEMandate and serviceTopicNumber projects/ assignments% in relevant populationSourcePeriod coveredRelevanceAll mandates, all servicesDistribution assignments by type, sector, country1091 assignments for programming period 2014-2020100%JASPERS' JADE databaseuntil end Dec 2018RelevanceESIF advisory major projectsMajor projects submitted to European Commission 320 projects submitted (new and phased)100%European Commission SFCuntil end Oct 2019RelevanceESIF advisory major projectsUse of JASPERS advisory assistance for preparation of major projects by procedure, by sector, by country184 projects submitted and assisted (new and phased)100%European Commission SFCuntil end Oct 2019RelevanceESIF advisory major projectsSize of new major projects submitted, with comparison between assisted and non-assisted178 new major projects approved (does not include phased projects)100%European Commission SFCuntil end Oct 2019RelevanceESIF advisory major projectsFrequent topic of JASPERS' advice for preparation of new major projects81 ACNs of assisted new major projects submitted for 102.288%JASPERS' ACNs for completed assignments available in SFCuntil end Dec 2018RelevanceESIF advisory non-major projectsSize of non-major projects assisted by JASPERS. Frequent topic of JASPERS' advice for preparation of non-major projects.49 ACNs for assisted non-major projects (in HR, PL, RO and SK)74%ACNs for completed assignments provided by JASPERSuntil end Dec 2018RelevanceCEF advisoryUse of JASPERS assistance relative to initially planned in SGA for CEF158 CEF projects planned, and 29 projects assisted100%SGA for CEF1 and JASPERS' JADEuntil end Dec 2018RelevanceCEF advisoryUse of JASPERS advisory assistance for preparation of CEF projects by type, sector, country. Size of CEF projects approved, with comparison between assisted and non-assisted.265 CEF projects approved for EU financing, of which 29 assisted by JASPERS100%DG MOVE database for all CEF projects approved; final implementation report for CEF1 mandateuntil Aug 2019RelevanceCEF advisoryFrequent topics of JASPERS' advice for preparation of CEF  projects.9 sampled assignments29%PDRs for the sampled assignmentsuntil December 2018RelevanceIPA advisoryUse of JASPERS advisory assistance for preparation of IPA projects by type, sector, country. Size of IPA projects approved, with comparison between assisted and non-assisted.253 IPA projects approved, of which 17 assisted by JASPERS100%DG NEAR database for all IPA projects approveduntil August 2019 RelevanceESIF review functionDistribution of new major projects submitted by procedure, country and sector210 new major projects submitted for Art 102.1 and Art 102.2, of which 140 assisted by JASPERS100%European Commission SFCuntil end Oct 2019RelevanceESIF review functionLikelihood a new major project is submitted for Art 102.2 (relative to Art 102.1)201 new major projects with complete data for the model96%European Commission SFCuntil end Oct 2019RelevanceESIF capacity buildingDemand for country specific capacity building activities4 sampled countries (HR, PL, RO, SK)78%JASPERS' JADE database and country analysesuntil end Dec 2018RelevanceESIF capacity buildingMain topics for multi country and country specific activities for capacity building38 completed multi counttry and 9 country specific activities for capacity building100%JASPERS' JADE database and the oline Networking Platformuntil end Dec 2018RelevanceESIF capacity buildingPublications on JASPERS' online Networking Platform60 publications100%JASPERS' online Networking Platformaccessed September 2020RelevanceESIF horizontal and strategic supportMain topics for country specific activities for horizontal and strategic support125 completed and on-going HSS assignments100%JASPERS' JADE database and the oline Networking Platformuntil end Dec 2018RelevanceAll mandates, all servicesStakeholders' perceptions of relevance of JASPERS' services and customer satisfaction with these services210 respondents (MAs, project beneficiaries, EC staff, consultancy companies, other)100%Online stakeholders consultation for the evaluationSummer 2019 
 
 
 
 
 
 
Table A2.1.2 :  Summary of data for effectiveness 

207 

EFFCTIVENESSMandate and serviceTopicNumber projects/ assignments% in relevant populationSourcePeriod coveredEffectivenessESIF advisory major projectsAssignments resulting in major projects adopted by the Commission.226 completed assignments, of which 158 resulted in approved projects100%JASPERS' JADE database and European Commission's SFCuntil October 2019EffectivenessESIF advisory major projectsCumulated EU funding for approved major projects, assisted and non-assisted281 approved projects (new and phased), of which 158 assisted by JASPERS100%JASPERS' JADE database and European Commission's SFCuntil October 2019EffectivenessESIF advisory major projectsDurations assignments completed178 assignments completed by December 2018100%JASPERS' JADE databaseuntil December 2018EffectivenessESIF advisory major projectsDuration of projects' lifecycle and project core preparation 210 new major projects submitted to the EC (assisted and non-assisted; phased projects not included).  100%European Commission SFC until October 2019EffectivenessESIF advisory major projectsStage of JASPERS' involvement in core project preparation120 assignments for new major projects assisted by JASPERS and submitted to the Commission85%JASPERS' JADE database and European Commission's SFCuntil October 2019EffectivenessESIF advisory major projectsCorrelation between stage of involvement and duration of core preparation relative to project lifecycle166 new major projects submitted the Commission79%JASPERS' JADE database and European Commission's SFCuntil October 2019EffectivenessESIF advisory major projectsAnalysis rates of interruptions for major projects150 new major projects submitted for Art 102.2100%JASPERS' JADE database and European Commission's SFCuntil October 2019EffectivenessESIF advisory major projectsTopics of observations in EC interruption letters, number observations raised, duration of interruptions69 new major projects interrupted, of which 44 assisted by JASPERS100%European Commission SFC until October 2019EffectivenessESIF advisory major projectsIssues raised by JASPERS advisory for new major projects subsequently interrupted37 new major projects assisted by JASPERS advisory and interrupted by the Commission84%JASPERS' ACNs, European Commission SFCuntil October 2019EffectivenessESIF advisory major projectsAnalysis of likelihood that new major projects submitted for Art 102.2 are not interrupted142 new major projects submitted to the Commission for Art 102.295%JASPERS' JADE database and European Commission's SFCuntil October 2019EffectivenessESIF advisory major projectsAnalysis implementation of new major projects166 new major projects submitted to the Commission and approved/ adopted. 93%JASPERS' JADE database and European Commission's SFCuntil October 2019EffectivenessESIF advisory non - major projectsDuration of assignments completed66 assignments completed100%JASPERS' JADE databaseuntil December 2018EffectivenessESIF advisory non - major projectsStage of JASPERS' involvement in core project preparation6 assignments completed9%ACNs for completed assignments provided by JASPERSuntil December 2018EffectivenessESIF advisory non - major projectsIssues raised by JASPERS for non- projects in ACNs49 ACNs for assisted non-major projects (in HR, PL, RO and SK)74%ACNs for completed assignments provided by JASPERSuntil end Dec 2018EffectivenessCEF advisoryAssignments resulting in CEF projects approved by the Commission.29 CEF projects assisted .100%JASPERS' JADE database and DG MOVE database for approved CEF projectsuntil August 2019EffectivenessCEF advisoryDuration of completed assignments21 assignments completed68%JASPERS' JADE databaseuntil December 2018EffectivenessCEF advisoryEffect of advisory on project technical specifications21 assignments completed and 9 sampled assignments for in-depth analysis68%JASPERS' JADE databaseuntil December 2018EffectivenessIPA advisoryAssignments resulting in IPA projects approved73 advisory assignments . 100%JASPERS' JADE database and DG NEAR database of approved IPA projectsuntil August 2019EffectivenessIPA advisoryDuration completed assignments28 advisory assignments100%JASPERS' JADE databaseuntil December 2018EffectivenessIPA advisoryStage of JASPERS' involvement in project preparation6 sampled assignments21%ACNs and interviews for sampled assignmentsuntil December 2018EffectivenessESIF review functionDuration of completed assignments for approved/ adopted new major projects175 assignments for IQR and PSA98%JASPERS' JADE database and European Commission SFC until October 2019EffectivenessESIF review functionAnalysis of PSA reports for interrupted projects66 major projects submitted for Art 102.2 and interrupted (first PSA report)96%JASPERS' PSA reports, European Commission SFCuntil October 2019EffectivenessESIF capacity buildingExamples of innovative approaches3 examples from 10 sampled assignmentsillustrativeEffectivenessESIF horizontal and strategic supportEffectiveness at assignment level3 examples from 10 sampled assignmentsillustrativeEffectivenessIPA horizontal and strategic supportEffectiveness at assignment level2 examples from 10 Sampled assignmentsillustrativeEffectivenessAll mandates, all servicesStakeholders' perceptions of JASPERS' effects210 respondents (MAs, project beneficiaries, EC staff, consultancy companies, other)100%Online stakeholders consultation for the evaluationSummer - Autumn 2019JASPERS' ACNs and interviews for sampled assignmentsuntil December 2018 
 
 
 
Table A2.1.3 :  Summary of data for efficiency, coherence, and EU added value 

A2.2 Models  
M1. Model for the use of CPR Art 102.2 for submission of major projects  
(Section 5.1 Relevance, ESIF mandate, Review function) 

Table A2.2.1: Variables used in the model: 

Variable 

Values 

Share of 
"yes" 

71% 
67% 

17% 
31% 
28% 
24% 

1=yes, 0=no 
1=yes, 0=no 

1=yes, 0=no 
1=yes, 0=no 
1=yes, 0=no 
1=yes, 0=no 

Projects submitted for 102.2 (dependent 
variable) 
Jaspers' TA assistance  
Year submission major project 
Year submission 2015-2016 
Year submission 2017 
Year submission 2018 
Year submission 2019 
Total cost major project 
Total cost: 50 million euro 
Total cost: 50-100 million euro 
Total cost: 100-200 million euro 
Total cost: 200-400 million euro 
Total cost: > 400 million euro 
EU amount major project 
Sector major project 
Energy and solid waste 
Rail, air, maritime 
Roads 
Smart development and ICT 
Water and wastewater 
Note: 201 new major projects submitted and with no missing data for all variables. 

1=yes, 0=no 
1=yes, 0=no 
1=yes, 0=no 
1=yes, 0=no 
1=yes, 0=no 

1=yes, 0=no 
1=yes, 0=no 
1=yes, 0=no 
1=yes, 0=no 
1=yes, 0=no 

14% 
35% 
26% 
18% 
7% 

7% 
32% 
37% 
12% 
12% 

208 

EFFICIENCYMandate and serviceTopicNumber projects/ assignments% in relevant populationSourcePeriod coveredEfficiencyAll mandates all services, except for review functionAnalysis country action plans522 assignments in JADE and country plans for sampled countries (HR, PL, RO)52%JASPERS' JADE database and country plansuntil December 2018EfficiencyAll mandates all servicesAnalysis administratively closed assignments99 assignments administratively closed100%JASPERS' JADE database and country plansuntil December 2018EfficiencyAll mandates all servicesAnalysis completed assignments562 completed assignments (includes also 2021-2027)100%JASPERS' JADE databaseuntil December 2018EfficiencyAll mandates all servicesAnalysis work intensity190 assignments completed or closed during 2017-201998%JASPERSJanuary 2017 - February 2019EfficiencyAll mandates all servicesHuman and financial resources for JASPERSAnnual data for the period 2007-2018100%JASPERS' annual reports2007-2018COHERENCEMandate and serviceTopicNumber projects/ assignments% in relevant populationSourcePeriod coveredExternal coherenceESIF mandateAlignment with EU requirements and standards for cohesion policy, climate, environment, and state aidapprox. 120 major and non-major projects and templates for ACNs, PSA and IQR reportsn.a.JASPERS' documentation, European Commission SFCuntil October 2019Internal coherenceESIF advisory for major projectsAlignment of advisory asssessments with PSA assessmentsACNs and PSA reports for 77 major projects submitted for Art. 102.284%JASPERS' ACNs, European Commission for PSA reportsuntil October 2019EU Added ValueMandate and serviceTopicNumber projects/ assignments% in relevant populationSourcePeriod coveredEU Added ValueAll mandates, all servicesStakeholders' perceptions of JASPERS' added value relative to other alternative schemes210 respondents (MAs, project beneficiaries, EC staff, consultancy companies, other)100%Online stakeholders consultation for the evaluationSummer - Autumn 2019 
 
 
 
  
  
  
  
  
  
  
  
P(z) 
0.041 

-2.27 
-2.05 
-2.05 
-1 

-2.18 
-1.82 
-1.89 
-0.94 

Coef.  z-value 
-2.04 
-0.96 

Table A2.2.2:  Ordered logistic 
regression 
Use of Art 102.2 by new major 
projects 
Jaspers' TA assistance (1=yes, 0=no) 
Total cost (relative to > 400 million 
euro) 
Total cost: 50 million euro 
Total cost: 50-100 million euro 
Total cost: 100-200 million euro 
Total cost: 200-400 million euro 
Year submission (relative to 2019) 
Year submission 2015-2016 
Year submission 2017 
Year submission 2018 
Sector (relative to Rail, air, and 
maritime) 
Energy and solid waste 
Roads 
Smart development and ICT 
Water and wastewater 
Number projects: 201; LR chi2(12)=47.64; Prob > chi2=0, Pseudo 
R2=0.196. 
Statistical significance of 5% if P(z) <=0.05. 

1.30 
0.90 
1.13 
-0.10 

1.75 
1.81 
1.9 
-0.18 

2.38 
1.95 
1.16 

3.3 
3.96 
2.53 

0.023 
0.041 
0.040 
0.318 

0.001 
0.000 
0.011 

0.081 
0.070 
0.057 
0.860 

M2-3 : Models for the ratio of duration of core preparation in total project duration 
(Section 5.2 Effectiveness, ESIF mandate, Advisory major projects) 

Table A2.2.3 : Description of variables in the model 
Dependent variable 

Ratio preparation in total project duration 

Values 
0% -100% 

Independent variables 

JASPERS' involvement in core preparation 

JASPERS' from start of core preparation 

JASPERS' involvement after start of core 
preparation 

JASPERS' involvement after core preparation 

Values 

1=yes, 
0=no 
1=yes, 
0=no 
1=yes, 
0=no 
1=yes, 
0=no 

Mean 
65% 
Share of 
"yes" 

60% 

12% 

48% 

5% 

209 

 
 
 
 
 
 
 
 
 
  
 
 
  
 
 
  
 
 
Total costs major project (million euro) 

Total cost: 50 million euro 

Total cost: 50-100 million euro 

Total cost: 100-200 million euro 

Total cost: 200-400 million euro 

Total cost: > 400 million euro 
Sector major project 

Energy and solid waste 

Rail, air, maritime 

Roads 

Smart development and ICT 

Water and wastewater 

1=yes, 
0=no 
1=yes, 
0=no 
1=yes, 
0=no 
1=yes, 
0=no 
1=yes, 
0=no 

1=yes, 
0=no 
1=yes, 
0=no 
1=yes, 
0=no 
1=yes, 
0=no 
1=yes, 
0=no 

14% 

36% 

27% 

16% 

8% 

8% 

33% 

40% 

8% 

11% 

Note : 166 new major projects submitted and with no missing data for all variables. 
Source : Based on data from JASPERS’ JADE database and European Commission 
SFC2014. 

Table A2.2.4: Linear regression models with robust standard errors  

Ratio core preparation in total project 
duration 

Jaspers' involvement in core preparation 
Jaspers' involvement from start of core 
preparation 
Jaspers' involvement after start of core 
preparation 
Project total costs (relative to > 400 million 
euro) 

Total cost: 50 million euro 
Total cost: 50-100 million euro 
Total cost: 100-200 million euro 
Total cost: 200-400 million euro 
Sector (relative to Roads) 
Energy and solid waste 

Model 1 

Model 2 

Coef. 

0.06 

t-value 
(P(t)) 
1.50 (0.136) 

Coef. 

t-value (P(t)) 

-0.13 

-2.07 (0.040) 

0.07 

1.81 (0.073) 

-0.14 

0.13 
0.11 
0.08 

-1.85 
(0.006) 

-0.12 

-1.76 (0.081) 

2.00 (0.048)  0.14 
1.62 (0.107)  0.11 
1.19 (0.235)  0.08 

2.40 (0.018) 
1.89 (0.061) 
1.25 (0.214) 

-0.39 

-7.25 

-0.38 

-7.21 (0.000) 

210 

 
  
  
  
  
 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
Rail, air and maritime 

Smart development and ICT 

Water and wastewater 

Country fixed effects 

Model fit tests 

-0.29 

-0.30 

-0.29 

(0.000) 
-6.85 
(0.000) 
-3.67 
(0.000) 
-4.24 
(0.000) 

-0.28 

-6.82 (0.000) 

-0.24 

-3.26 (0.001) 

-0.22 

-3.55 (0.001) 

YES 

R-squared = 0.5781;  
Root MSE= 0.1649 

R-squared = 0.6253;  
Root MSE= 0.15596 

Note : 166 new major projects submitted. Statistical significance of 5% if P(z) <=0.05. 

M4-M5 : Models for the probability that a project is not interrupted during 
approval/ adoption procedures  
(Section 5.2 Effectiveness, ESIF mandate, Advisory major projects) 

Table A2.2.5: Description of variables used in the model 

Variable 

Values 

Share of 
"yes" 

32% 

24% 

43% 

1=yes, 0=no 

1=yes, 0=no 

Projects not interrupted (dependent 
variable) 
JASPERS' early involvement in core 
preparation 
JASPERS' later involvement in core 
preparation 
Total costs 
Total cost: 50 million euro 
Total cost: 50-100 million euro 
Total cost: 100-200 million euro 
Total cost: 200-400 million euro 
Total cost: > 400 million euro 
Sector major project 
Energy and solid waste 
Rail, air, maritime 
Roads 
Smart development and ICT 
Water and wastewater 
Note: 142 major projects submitted for Art 102.2 with no missing data for all variables. 

1=yes, 0=no 
1=yes, 0=no 
1=yes, 0=no 
1=yes, 0=no 
1=yes, 0=no 

1=yes, 0=no 
1=yes, 0=no 
1=yes, 0=no 

8% 
27% 
44% 
12% 
8% 

12% 
33% 
26% 
20% 

211 

 
 
 
  
  
  
  
  
  
  
  
 
 
 
 
Table A2.2.6 :  Logistic regression  
Probability the project is not 
interrupted 
JASPERS' early involvement (1=yes, 
0=no) 
JASPERS' later involvement (1=yes, 
0=no) 
Total cost (relative to > 400 mil. euro) 
Total cost: 50 million euro 
Total cost: 50-100 million euro 

Total cost: 100-200 million euro 
Total cost: 200-400 million euro 
Sector (relative to Roads) 

Energy and solid waste 

Rail, air and maritime 

Smart development and ICT 

Water and wastewater 

Country fixed effects 

Model fit tests 

Model 1 

Model 2 

Coef. 

z-value (P(z))  Coef. 

z-value (P(z)) 

1.04 

1.775 (0.076) 

2.32 

2.909 (0.004) 

0.64 

1.295 (0.195) 

1.22 

2.022 (0.043) 

0.55 
0.67 

-0.06 

0.24 

-1.53 

-0.98 

-1.79 

-1.41 

0.682 (0.495) 
0.982 (0.326) 
-0.092 
(0.927) 
0.335 (0.738) 

0.09 
0.54 

0.18 

0.79 

0.095 (0.924) 
0.692 (0.489) 

0.231 (0.817) 

0.960 (0.337) 

-2.129 
(0.033) 
-1.810 
(0.070) 
-2.551 
(0.011) 
-1.825 
(0.068) 

-1.80 

-2.198 (0.028) 

-0.44 

-0.686 (0.493) 

-0.84 

-0.860 (0.390) 

-1.87 

-2.067 (0.950) 

NO 
LR chi2(10) = 10.74 
Prob > chi2=0.3779 
Pseudo R2=0.0554 

YES 
LR chi2(12) = 38.71 
Prob > chi2=0.002 
Pseudo R2=0.1995 

Note : 142 new major projects for Art 102.2. Statistical significance of 5% if P(z) 
<=0.05. 

M6-M7: Models for financial progress in implementation of approved major 
projects 
(Section 5.2 Effectiveness, ESIF mandate, Advisory major projects) 

Table A2.2.7: Variables used in the model: 
A) Dependent variable:  
Financial progress (share certified expenditure by 2019/2018) 
1_No progress 0% 
2_Progress up to 10% 
3_Progress in 10% -50% 
4_Progress in 50%-80% 
5_Progress in 80%-100% 
6_Progress more than 100% 

16% 
12% 
28% 
14% 
20% 
10% 

212 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
B) Independent variables : 

Variable 

Values 

Share of 
"yes" 
60% 

19% 

27% 

1=yes, 0=no 

1=yes, 0=no 

1=yes, 0=no 

50% 
25% 
25% 

1=yes, 0=no 
1=yes, 0=no 
1=yes, 0=no 

Jaspers' advisory for core preparation 
Jaspers' advisory early for core 
preparation 
Jaspers' advisory early for core 
preparation of young projects 
(start>=2017) 
EU amount (ERDF and CF) 
EU amount: 50 million euro 
EU amount: 50-100 million euro 
EU amount: 100-800 million euro 
Sector major project 
Energy and solid waste 
Rail, air, maritime 
Roads 
Smart development and ICT 
Water and wastewater 
Year start project implementation 
Year <= 2014 
Year 2015 
Year 2016 
Year 2017 
Year >=2018 
Note : 166 new major projects adopted with no missing data for all variables. 
Source : Based on data from JASPERS’ JADE database and European Commission 
SFC2014. 

1=yes, 0=no 
1=yes, 0=no 
1=yes, 0=no 
1=yes, 0=no 
1=yes, 0=no 

1=yes, 0=no 
1=yes, 0=no 
1=yes, 0=no 
1=yes, 0=no 
1=yes, 0=no 

21% 
20% 
19% 
22% 
19% 

8% 
33% 
40% 
8% 
11% 

213 

 
  
  
  
 
Table A2.2.8 :  Ordered logistic regressions  

Note : 166 new major projects adopted. Statistical significance of 5% if P(z) <=0.05. 

214 

Coef.z-value (P(z))Coef.z-value (P(z))Coef.z-value (P(z))Jaspers' advisory for core preparation0.050.125 (0.900)Jaspers' advisory for early preparation0.631.334 (0.180)Jaspers' advisory for early preparation of young projects1.282.071 (0.038)EU amount (relative to > 100 mill euro)EU amount: 50 mill euro1.272.840 (0.010)1.292.894 (0.000)1.292.892 (0.004)EU amount: 50-100 mill euro1.493.093 (0.000)1.503.121 (0.000)1.402.914 (0.004)Sector (relative to Roads)Energy and solid waste-3.11-4.223 (0.000)-3.19-4.487 (0.000)-3.24-4.543 (0.000)Rail, air and maritime-2.33-4.516 (0.000)-2.55-4.992 (0.000)-2.53-5.162 (0.000)Smart development and ICT-0.93-1.095 (0.270)-1.32-1.503 (0.130)-1.62-1.826 (0.068)Water and wastewater-3.09-4.278 (0.000)-3.47-4.687 (0.000)-3.35-4.858 (0.000)Start implementation (relative to 2018/2019)Year <= 20144.626.298 (0.000)4.776.376 (0.000)5.206.519 (0.000)Year 20155.067.013 (0.000)5.227.103 (0.000)5.667.177 (0.000)Year 20163.264.862 (0.000)3.495.025 (0.000)4.015.185 (0.000)Year 20171.332.382 (0.020)1.452.562 (0.010)1.592.761 (0.006)Country fixed effectsModel fit testsModel 3YESLR chi2(27) = 208.95Prob > chi2=0.0000Pseudo R2=0.3642Model 1Financial progress in implementationModel 2LR chi2(27) = 206.34Prob > chi2=0.0000Pseudo R2=0.3596 LR chi2(27)=204.56Prob > chi2=0.0000Pseudo R2=0.3565 
 
 
 
 
 
 
ANNEX 3: JASPERS SUPPORT IN 2007-2013 

RELEVANCE 

Table A3.1 JASPERS completed assignments for ESIF, 2007-2013, by sector and type170 

Source : Jaspers Annual Reports 2010-2013.  

Table A3.2 : JASPERS completed assignments for ESIF, 2007-2013, by country and 
type 

Notes : n.a = not applicable. 
Source : Jaspers Annual Reports 2010-2013 and European Commission SFC2007, 
downloaded September 2020. 

170  The  classification  of  assignments  by  sector  differs  slightly  from  the  current  period  since  the  codes  used  have 

changed over time.  

215 

Sector \ Type assignmentMajor projectsNon-major projectsHorizontalCapacity buildingTotal% in totalKnowledge economy.energy and waste1194447321327%Rail, air and maritime125215115219%Roads99124211715%Urban infrastructure and services276110446%Water and wastewater1505045024531%Multi sector00180182%Other100010.1%Total5211331306790100%% in total66%17%16%1%100%AssignmentsProjects adopted by EC (% of assignments)Bulgaria3447%241817710%Croatia450%03071%Cyprus3100%530111%Czechia6365%530719%Estonia944%240152%Greece3100%00030.4%Hungary6362%222699%Latvia1060%141162%Lithuania956%291213%Malta1250%930243%Poland15881%2624020826%Romania10182%3448018323%Slovakia3871%530466%Slovenia1493%1820344%Multi country0n.a.04041%Networking Platform0n.a.00110.1%Total52174%1331306790100%% in Total66%17%16%1%100%Total% in TotalCountryMajor projectsNon-major projectsHorizontal and strategic supportCapacity building 
 
 
 
 
 
 
 
                                                 
Table A3.3 : Major projects adopted for the period 2007-2013, by Member State 

Country 

(1) 
Bulgaria 
Croatia 
Cyprus 
Czechia 
Estonia 
France 
Germany 
Greece 
Hungary 
Ireland 
Italy 
Lithuania 
Latvia 
Malta 
Poland 
Portugal 
Romania 
Slovenia 
Slovakia 
Spain 
United 
Kingdom 

Total 

Number of 
major projects 
adopted by 
EC 
(2) 
17 
11 
3 
45 
8 
33 
37 
57 
39 
3 
51 
5 
6 
6 
215 
26 
93 
14 
27 
60 

11 
767 

Jaspers TA 
support  
(% projects) 

Framework contract  
for appraisal (% 
projects) 

(3) 
94% 
18% 
100% 
91% 
50% 
0% 
0% 
5% 
100% 
0% 
14% 
100% 
100% 
100% 
60% 
0% 
89% 
93% 
100% 
0% 

0% 
50% 

(4) 
6% 
9% 
33% 
20% 
25% 
27% 
27% 
53% 
23% 
33% 
90% 
100% 
33% 
33% 
42% 
58% 
6% 
14% 
19% 
75% 

73% 
39% 

Source: European Commission SFC2007, downloaded September 2020.  

Table A3.4 Major projects adopted for the period 2007-2013, by sector 

Source: European Commission SFC2007, downloaded September 2020.  

216 

SectorNumber of major projects adopted by ECJaspers TA support (% projects)EC Framework contract for appraisal (% projects)Energy and solid waste7672%29%Rail, air, maritime15651%40%Roads19646%41%Smart development and ICT20335%49%Water and wastewater13663%26%Total76750%39% 
 
 
 
 
 
Table A3.5 :  Median values for project financing by sector and JASPERS advisory 

Note : Data covers all 767 major projects approved by EC for the period 2007-2013. 
Source: European Commission SFC2007, downloaded September 2020.  

217 

With JASPERS TANot assistedWith JASPERS TANot assistedEnergy and solid waste74723426Rail, air, maritime1791419152Roads1381189364Smart development and ICT78885623Water and wastewater73604830SectorTotal cost (mill euro)EU Amount (mill euro) 
 
 
 
 
Table A3.6 : Topics of JASPERS technical advice for projects supported, 2007-2011 

Topic Jaspers' contribution 

Projects assisted by 
Jaspers on the topic 
122 
58 
48 
40 

% in all projects 
assisted 
74% 
35% 
29% 
24% 

Cost benefit analysis 
Funding and financing plan 
Environmental issues 
Project concept and programming 
Project implementation and 
structures 
Demand analysis 
Project design 
Risk and sensitivity analysis 
Assistance with interruptions 
Project cost estimation 
Procurement 
State aid 
Note : Shares inferred by authors based on the findings reported. Data cover 165 
projects.  
Source : European Commission, DG REGIO evaluation of JASPERS 2012. 

24% 
24% 
21% 
19% 
11% 
10% 
10% 
8% 

40 
40 
35 
31 
18 
16 
16 
13 

Table A3.7 JASPERS’s assignments for ESIF non-major projects by sector, 2007-2013 

Sector 

Number 
assignments 

% in 
total 

Knowledge economy, energy and 
waste 
Roads 
Rail, air and maritime 
Urban infrastructure and services 
Water and wastewater 
Total 
Source : Jaspers Annual Reports 2010-2013 and European Commission SFC2007, 
downloaded September 2020. 

33% 
9% 
16% 
5% 
38% 
100% 

44 
12 
21 
6 
50 
133 

218 

 
 
 
 
 
Table A3.8 JASPERS’s assignments for ESIF non-major projects by country, 2007-2013  

Country   

Number 
assignments 
24 
5 
5 
2 
2 
1 
2 
9 
26 
34 
5 
18 
133 

Bulgaria 
Cyprus 
Czechia 
Estonia 
Hungary 
Latvia 
Lithuania 
Malta 
Poland 
Romania 
Slovakia 
Slovenia 
Total 
Source : Jaspers Annual Reports 2010-2013 and European Commission SFC2007, 
downloaded September 2020. 

% in 
total 
18% 
4% 
4% 
2% 
2% 
1% 
2% 
7% 
20% 
26% 
4% 
14% 
100% 

219 

 
 
 
 
EFFECTIVENESS 

Table A3.9 Interruption topics for major projects, 2007-2011 

Topic 

Proportion projects 
interrupted 
43% 
40% 
34% 
32% 
24% 

Environmental issues 
Financial plan 
Cost benefit analysis 
Project design 
Project concept 
Implementation and 
structures 
Risk and sensitivity analysis 
Project costs 
Feasibility study 
Procurement 
State aid 
Note : Data covers 146 major projects.  
Source : European Commission, DG REGIO evaluation of JASPERS 2012. 

24% 
23% 
14% 
13% 
8% 
4% 

220 

 
 
 
 
 
 
ANNEX 4: JASPERS SUPPORT 2014-2020 

RELEVANCE 

A4.1 ESIF Mandate 

Table A4.1 JASPERS’ assignments ESIF mandate for project and horizontal support  

Sources :  JASPERS’s  JADE  database,  December  2018 ;  European  Commission 
SFC2014, October 2019. 

Table A4.2 JASPERS’ portfolio for review assignments  

Sources : JASPERS’s JADE database, December 2018. 

221 

 
 
 
 
 
 
 
 
 
 
 
Table A4.3  New major projects approved/ adopted by EC until October 2019171  

Country 

(1) 
Austria 
Bulgaria 
Czechia 
Denmark 
Spain 
France 
Greece 
Croatia 
Hungary 
Italy 
Latvia 
Lithuania 
Malta 
Poland 
Portugal 
Romania 
Slovenia 
Slovakia 
Total 

Projects 

Art. 102.1 (IQR) 
% with 
JASPERS TA 
(3) 

(2) 

1 
3 
2 

2 

11 

2 
3 
3 
1 
20 

5 

7 
60 

100% 
67% 
0% 

100% 

91% 

100% 
100% 
100% 
100% 
60% 

100% 

100% 
80% 

Projects 

Art 102.2 (PSA) 
% with 
JASPERS TA 
(5) 
100% 
100% 
100% 

(4) 
1 
9 
1 

4 
3 
8 

11 
5 

81 
6 
14 
4 
3 
150 

75% 
67% 
75% 

100% 
20% 

44% 
33% 
93% 
100% 
100% 
61% 

Total 
projects 

(6)=(2)+(4) 
1 
10 
4 
2 
4 
5 
8 
11 
11 
7 
3 
3 
1 
101 
6 
19 
4 
10 
210 

% with 
JASPERS'  
TA in total 
(7) 
100% 
100% 
75% 
0% 
75% 
80% 
75% 
91% 
100% 
43% 
100% 
100% 
100% 
48% 
33% 
95% 
100% 
100% 
67% 

Source : European Commission SFC2014,  October 2019. 

Table A4.4 JASPERS’ assignments for non-major projects 

Country  On-going 

Bulgaria 
Croatia 
Cyprus 
Czechia 
France 
Greece 
Hungary 
Latvia 
Malta 
Poland 
Portugal 

3 
23 
2 
1 
0 
1 
0 
0 
1 
8 
1 

Completed 
and closed 
15 
23 
0 
2 
1 
3 
2 
2 
2 
5 
1 

Total  % in total 

18 
46 
2 
3 
1 
4 
2 
2 
3 
13 
2 

13% 
33% 
1% 
2% 
1% 
3% 
1% 
1% 
2% 
9% 
1% 

171  SFC2014 dataset includes additional projects for which the review/ assessment was carried out in 2019 (therefore, 

after the reference end date for the JADE dataset).  

222 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
                                                 
Romania 
Slovakia 
Slovenia 
Total 
% in total 
Sources : JASPERS’s JADE database, December 2018. 

23 
1 
0 
64 
46% 

17 
1 
2 
76 
54% 

29% 
1% 
1% 
100% 

40 
2 
2 
140 

Table A4.5 Horizontal and strategic support 

% in 
total 
Country 
8% 
Bulgaria 
11% 
Croatia 
1% 
Cyprus 
2% 
Czechia 
2% 
Greece 
4% 
Hungary 
1% 
Ireland 
3% 
Latvua 
4% 
Lithuania 
3% 
Malta 
16% 
Poland 
31% 
Romania 
9% 
Slovakia 
4% 
Slovenia 
1% 
Spain 
100% 
Total 
Note : Data covers assignments completed and on-going by 2018. 
Source : JASPERS’ JADE database, December 2018. 

Assignments 
10 
14 
1 
3 
2 
5 
1 
4 
5 
4 
20 
39 
11 
5 
1 
125 

223 

 
  
  
 
 
 
 
 
Table A4.6 Interruption rates by country for major projects submitted for Art. 102.2 

Assisted by JASPERS TA 

Not assisted by JASPERS 
TA 

% interrupted 

Country 

% interrupted 

Number 
projects 
1 
9 
1 
2 
6 
11 
1 
36 
2 
13 
4 
3 
3 
92 

Austria 
Bulgaria 
Czechia 
France 
Greece 
Hungary 
Italy 
Poland 
Portugal 
Romania 
Slovenia 
Slovakia 
Spain 
Total 
Note :  n.a.= not applicable. 
Source : European Commission SFC2014, October 2019. 

0% 
67% 
0% 
50% 
67% 
73% 
100% 
25% 
0% 
54% 
100% 
33% 
100% 
48% 

Number 
projects 
0 
0 
0 
1 
2 
0 
4 
45 
4 
1 
0 
0 
1 
58 

n.a. 
n.a. 
n.a. 
100% 
50% 
n.a. 
100% 
29% 
100% 
100% 
n.a. 
n.a. 
100% 
43% 

Overall rate of 
interruption 
(%) 

0% 
67% 
0% 
67% 
63% 
73% 
100% 
27% 
67% 
57% 
100% 
33% 
100% 
46% 

224 

 
 
 
 
A4.2 CEF1.1 Mandate 
Table A4.7 JASPERS’s assignments and projects approved for CEF financing, by 
country 

Country 

CEF 
projects 
cohesion 
envelope 

CEF 
projects 
blending 

Projects 
in Annex 
I, SGA 

Projects 
supported 
by 
JASPERS 

JASPERS 
TA 
assignments 

% JASPERS 
supported in 
all CEF 

43% 
11% 

8 

2 
8 

3 
2 

3 
2 

6 
25 

2 
19 

3 
11 

3 
2 
1 
1 

1 
2 
1 
4 
8 
7 

6 
17 
2 
39 

Austria 
Belgium 
Bulgaria 
Croatia 
Cyprus 
Czechia 
Denmark 
Estonia 
Finland 
France 
Germany 
Greece 
Hungary 
Ireland 
Italy 
Lithuania 
Malta 
Netherlands 
Poland 
Portugal 
Romania 
Slovakia 
Slovenia 
Spain 
Sweden 
Multi 
country 
Total 
Note :  Data  covers  only  projects  approved  for  CEF  financing,  and  considers  only  the 
CEF1.1 mandate for JASPERS. 
Sources :  DG  MOVE  dataset  Autumn  2019,  SGA  DG  MOVE-JASPERS  for  CEF1.1, 
Final report for CEF1.1 mandate.  

28 
9 
13 
17 
6 

17 
5 
1 

21 
193 

12 
72 

2 
29 

2 
31 

7 
4 
1 

1 
3 
4 

5 
8 
1 

7 
8 
1 

2 
2 

58 

8 

32% 

38% 
47% 
14% 

6% 

11% 

225 

 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
 
 
 
A4.3 IPA Mandate 

Table A4.8 : JASPERS’ IPA portfolio by country and type of assignment 

Capacity 
building and 
horizontal 
1 
9 

Country 

Project 
preparation 

4 

Kosovo 
Montenegro 
North 
25 
Macedonia 
28 
Serbia 
16 
Turkey 
73 
Total 
% total 
72% 
Source : JASPERS JADE database, December 2018. 

22 
22% 

5 
7 

Implementation  Total  % total 

4 

1 
2 

7 
7% 

1 
17 

1% 
17% 

30% 
36% 
16% 
100% 

31 
37 
16 
102 
100% 

Country 

Table A4.9  Approved IPA projects 
Approved 
projects 
32 

Albania 
Bosnia 
Herzegovina 
Croatia 
Kosovo 
North Macedonia 
Montenegro 
Serbia 
Multi country 
Total 
Source : DG NEAR dataset, August 2019. 

45 
13 
27 
25 
33 
47 
31 
253 

Assisted by 
JASPERS 

5 
2 
10 

17 

226 

 
 
  
  
  
  
  
 
  
  
  
  
  
 
 
 
EFFECTIVENESS 

A4.4 CEF MANDATE 

Table A4.10 Sampled assignments for JASPERS’ advisory for CEF1.1 mandate 

Source: JASPERS’ project development reports issued for the assignments. 

227 

CountryProject descriptionTotal cost (mill euro)CEF financingPeriod JASPERS advisory for CEF1Objective CEF1 advisoryBGModernisation of railway line (43 km)168Submitted in 2016. Not adopted18/02/2016 - 19/04/2016(2.03 months)Review project application and supporting documentationBGDeployment ERTMS (210 km)74Submitted in 2016. Not adopted18/02/2016 - 19/04/2016(2 months)Review project application and supporting documentationHRConstruction of railway track (43 km)371Submitted Feb 2016. Approved in 201609/02/2016 - 06/04/2016(1.9 months)Review project application and supporting documentationHUModernisation of railway line (56 km)377Submitted Feb 2016. Approved Jul 201601/12/2015 - 19/04/2016(4.7 months)Review project application and supporting documentationHUUpgrading of motorway (10 km)49Submitted Feb 2016. Approved in 201607/12/2015 - 19/04/2016(4.5 months)Review project application and supporting documentationHUUpgrade railway bridge (0.5 km)114Submitted Feb 2016. Approved in Jul 201604/04/2016 - 19/04/2016(0.5 months)Review project application and supporting documentationROUpdate feasibility study for railways4Submitted Feb 2016. Approved in Jul 201602/03/2017- 13/06/2017(3.4 months)Support preparation of application for the study and of the ToR for the procurement procedure for the study. SKConstruction of bridge over Danube River (600 m)118Submitted Feb 2015, Approved Jul 201516/12/2014 - 22/12/2014(0.25 months)Review project application and supporting documentationSKModernisation of motorway (6 km)74Sumbitted Feb 2016. Approved Aug 201628/04/2008 - 30/03/2016( 96 months)Assistance with CBA, feasibility study, project applicationSKModernisation of railway line (32 km)473Submitted in 2016. Not adopted18/02/2016 - 06/04/2016(1.5 months)Review project application and supporting documentation 
 
 
 
 
 
 
EFFICIENCY 

A4.5 Analysis of country plans for assignments 

Table A4.11 Summary results from the analysis of country plans for sampled countries 

Planning coordinates 

Croatia 

Poland 

Romania 

Assignments matched in JADE and country 
plans 
Assignments in country plans but not JADE 
Assignments with consistent data in plans and 
JADE 
Assignments with expected completion date 
Assignments on track or completed on time 
(relative to initial planning) 
Median initial delay (for delayed assignments) 
Prolongued assignments (updated expected 
completion date) 
Median duration of prolongation 

86% 
7% 

79% 
87% 

23% 

87% 
7% 

86% 
98% 

43% 

74% 
17% 

86% 
92% 

45% 

19 months  12 months  20 months 

78% 

90% 

86% 

31 months  24 months  32 months 

Table A4.12 Administratively closed assignments across countries 

Country 

All 
assignments 

Administratively 
closed  

Austria 
Bulgaria 
Croatia 
Cyprus 
Czechia 
France 
Greece 
Hungary 
Ireland 
Italy 
Kosovo 
Latvia 
Lithuania 
Malta 
Montenegro 
North 
Macedonia 
Poland 
Portugal 
Romania 
Serbia 

1 
69 
142 
5 
38 
12 
25 
47 
2 
23 
1 
13 
11 
13 
17 

31 
128 
5 
191 
37 

0 
3 
30 
0 
7 
0 
2 
4 
0 
1 
0 
3 
1 
2 
4 

1 
10 
0 
13 
7 

228 

% admin closed in 
country's assignments 
0% 
4% 
21% 
0% 
18% 
0% 
8% 
9% 
0% 
4% 
e0% 
23% 
9% 
15% 
24% 

3% 
8% 
0% 
7% 
19% 

% admin 
closed in 
total 
0% 
3% 
31% 
0% 
7% 
0% 
2% 
4% 
0% 
1% 
0% 
3% 
1% 
2% 
4% 

1% 
10% 
0% 
14% 
7% 

 
 
 
 
6 
2 
0 
0 

57 
16 
9 
16 

11% 
13% 
0% 
0% 

Slovakia 
Slovenia 
Spain 
Turkey 
United 
0% 
Kingdom 
0% 
Multi country 
100% 
Total 
Note : Includes assignments for advisory, capacity building, and horizontal and strategic 
support for all mandates.  
Source : JASPERS’ JADE database, December 2018. 

6% 
2% 
0% 
0% 

0% 
0% 
10% 

2 
38 
949 

0 
0 
96 

COHERENCE 

Table A4.13: Comparison of PSA reports and ACNs for projects with non-critical issues 
in PSA 

Topic with non-critical 
issue(s) in PSA report 

Number 
projects 

Significant 
issue 

Assessed, no 
issues raised 

Conclusions in ACN: 

Assessed, 
with 
comment  
19 
27 

% 
significant 
issues and 
comments 
43% 
68% 

1 
0 

26 
12 

46 
40 

Cost benefit analysis 
Feasibility study 
Risk and sensitivity 
analysis 
Project description 
Project costs 
Capacity of beneficiary 
Financial plan 
Project timetable 
Note: Based on analysis of 77 major projects submitted for Art. 102.2. Data in table 
reports on the projects which do not have critical issues for the topic, but do have non-
critical issues identified in the PSA report. 
Source: PSA reports and ACNs. 

23 
17 
14 
29 
25 
32 

21 
0 
5 
4 
11 
11 

45 
19 
21 
38 
35 
54 

1 
1 
1 
4 
4 
2 

49% 
5% 
29% 
21% 
43% 
24% 

229