COMMISSION STAFF WORKING DOCUMENT EVALUATION Mid-term evaluation of the Structural Reform Support Programme
Date: 2020-10-01
Institutions: Directorate-General for Structural Reform Support, European Commission
Subjects: EU Member State, EU aid, EU financing, EU programme, administrative reform, economic convergence, economic reform, institutional reform, structural adjustment, sustainable development

EN 

  EN 

   EUROPEAN COMMISSION  Brussels, 1.10.2020  SWD(2020) 219 final  COMMISSION STAFF WORKING DOCUMENT  EVALUATION  Mid-term  evaluation of the Structural Reform Support Programme {SWD(2020) 220 final}  
 
 
Table of contents 

LIST OF ABBREVIATIONS .......................................................................................................... 2 

1. 

INTRODUCTION ................................................................................................................... 3 

2.  BACKGROUND ON THE INTERVENTION LOGIC OF THE SRSP ................................. 5 

3. 

IMPLEMENTATION / STATE OF PLAY ............................................................................ 9 

4.  METHODOLOGY ................................................................................................................ 16 

5.  ANALYSIS AND ANSWERS TO THE EVALUATION QUESTIONS............................. 20 

5.1.  Relevance ........................................................................................................................ 20 

5.2.  Effectiveness ................................................................................................................... 25 

5.3.  Efficiency ........................................................................................................................ 32 

5.4.  Coherence ....................................................................................................................... 42 

5.5.  EU added value ............................................................................................................... 50 

6. CONCLUSIONS ........................................................................................................................ 55 

ANNEX 1: PROCEDURAL INFORMATION ............................................................................. 59 

ANNEX 2: STAKEHOLDER CONSULTATION ........................................................................ 61 

ANNEX 3: EVALUATION FRAMEWORK ................................................................................ 66 

1 

 
 
 
 
 
LIST OF ABBREVIATIONS 

Term or acronym 

Meaning or definition 

CSP 

CSR  

DG 

Cooperation and support plan 

Country-specific recommendation 

Directorate-General 

DG REFORM 

Directorate-General for Structural Reform Support 

ESIF 

EU 

EUR 

ISSG 

PA 

SRSP 

SRSS 

SWD 

European Structural and Investment Funds 

European Union 

Euro 

Interservice Steering Group 

Preparatory Action 

Structural Reform Support Programme 

Structural Reform Support Service 

Staff working document 

TAIEX 

Technical Assistance and Information Exchange instrument 

2 

 
 
 
1.  INTRODUCTION 

Well-defined  reforms  are  crucial  for  modernising  economies,  encouraging  investment, 
creating jobs and raising living standards. To facilitate these reforms across the European 
Union, Regulation (EU) 2017/8251 on the establishment of the Structural Reform Support 
Programme  (‘the  SRSP  Regulation’)  was  adopted  in  May  2017.  The  objective  of  the 
Structural  Reform  Support  Programme’s  (the  ‘programme’)  is  to  contribute  to 
institutional,  administrative  and  growth-sustaining  reforms  in  Member  States  by 
providing technical support to national authorities. The SRSP has a budget of EUR 222.8 
million for the period 2017-2020.  

In  accordance  with  Article  16  of  the  SRSP  Regulation,  the  programme  is  subject  to  a 
mid-term  evaluation.  The  overall  goal  of  this  evaluation  is  to  assess  the  programme’s  
relevance,  effectiveness,  efficiency,  coherence  and  EU  added  value,  in  line  with  the 
Better  Regulation  guidelines,  and  to  draw  lessons  that  can  serve  possible  future 
improvements2.  The  mid-term  evaluation  also  ensures 
transparency  and 
accountability of the Commission’s activities.  

the 

To  support  the  mid-term  evaluation  exercise,  an  external  contractor  was  engaged  to 
perform  an  evaluation  report.  This  report  was  carried  out  over  a  period  of  12  months, 
starting on 20 November 2018. It covers the technical support provided under the SRSP’s 
2017  and  2018  budgetary  cycles,  as  well  as  projects  funded  through  the  voluntary 
transfers of additional resources to the SRSP budget by Greece and Bulgaria in 20183. It 
also covers technical support projects funded under the Preparatory Action of 20164. In 
total,  303  technical  support  projects  in  24  Member  States  fall  within  the  scope  of  this 
evaluation.  

In this staff working document (SWD), the Commission’s staff presents and reflects on 
the  main  outcomes  and  findings  of  the  mid-term  evaluation  and  provides  evidence  and 
data  which  could  serve  as  a  basis  for  further  improving  the  functioning  of  the  SRSP 
where  relevant.  The  SWD  also  draws  on  other  sources  of  information  from  the 

1 Regulation (EU) 2017/825 of the European Parliament and of the Council, OJ L 129, 19.5.2017, p. 1–16, 
as amended by Regulation (EU) 2018/1671, OJ L 284, 12.11.2018, p. 3–5 (the SRSP Regulation).  
2  Article  16  requires  the  mid-term  evaluation  report  to  ‘include  information  on  the  achievement  of  the 
Programme's objectives, the efficiency of the use of resources and the Programme's European added value. 
It  shall  also  address  the  continued  relevance  of  all  objectives  and  actions.’  All  this  is  covered  under  the 
Commission’s Better Regulation criteria. 
3 Article 11 (of the SRSP) refers to the voluntary contributions that a Member State can transfer from the 
structural funds to the SRSP for the use of that particular Member State.  
4 The Preparatory Action was introduced in 2016, before the SRSP was adopted, to help certain Member 
States. Six Member States benefitted from this support (Bulgaria, Czechia, Croatia, Romania, Slovenia and 
Slovakia) for a value of EUR 3 million.  

3 

 
 
 
 
                                                           
Directorate-General for Structural Reform Support,  including its internal IT system and 
the  annual  monitoring  reports,  annual  activity  reports,  and  ex  ante  evaluation  of  the 
SRSP. It should be underlined that the evaluation was performed when the programme’s 
implementation  was still at  an early stage.  This  SWD  outlines exploratory  conclusions, 
proportionate to the scope of the evaluation.   

The SWD has the following structure. Firstly, it describes the background of the SRSP 
and summarises the programme’s state of play. Subsequently, the SWD provides a brief 
overview of the methodology used for the evaluation, including associated limitations to 
the robustness of the findings. The main results of the evaluation report are presented in 
the next section and structured according to the five Better Regulation criteria. The last 
section of the SWD provides the overall conclusion and lessons learned.   

4 

 
 
 
 
2.  BACKGROUND ON THE INTERVENTION LOGIC OF THE SRSP 

The  effective  and  efficient  design  and  implementation  of  administrative  and 
institutional  reforms  is  essential  for  flexible  economies  that  are  able  to  adapt  to 
changing  circumstances.  Fruitful  and  timely  implemented  reforms  contribute  to  a 
favourable 
reinforces 
competitiveness, supports job creation, and enhances investment and sustainable growth.  

institutional  environment.  This, 

regulatory  and 

turn, 

in 

In the aftermath of the economic crisis, many EU Member States have been facing 
low  potential  growth,  significant  macroeconomic  imbalances  and  obstacles  to 
investment. Reforms were needed to address these challenges and enable governments to 
continue adapting to changing circumstances. It is an EU political priority to advance the 
implementation  of  growth-enhancing  reforms  in  order  to  strengthen  the  capacity  of 
Member  States  to  adjust  their  economies.  Not  only  does  this  help  to  boost  economic 
growth,  job  creation  and  social  welfare,  but  it  also  contributes  to  the  process  of 
convergence across the EU. 

The  process  of  designing  and  implementing  institutional  and  growth  enhancing 
reforms is complex and requires a high degree of knowledge and skills. Member States 
have different levels of in-house expertise and capacity to prepare and carry out  much-
needed  reforms.  Furthermore,  they  have  to  constantly  keep  up  with  changes,  whether 
political or from the world around us. Often, the Member States most in need of reform 
are at the same time the ones least equipped. Some EU countries do not have sufficient 
capacity to initiate and implement reforms  in response to the reform challenges that lie 
ahead. Others have the capacity, but their administrations are still struggling to adapt to 
constantly  changing  circumstances  arising  from  globalisation  and  technological  and 
international developments.  

Experience  in  economic  policy  coordination  through  the  European  Semester 
framework  shows  that  the  implementation  of  some  reforms  is  lagging  behind.  
Although in the multiannual assessment Member States achieved at least ‘some progress’ 
with  the  implementation  of  around  two  thirds  of  the  country-specific  recommendations 
(CSRs), the implementation of the annual CSRs has a slower pace overall. Furthermore, 
Member States sometimes struggle with the adequate and timely implementation of EU 
legislation.  Given  that  the  economies  of  EU  Member  States  are  strongly  intertwined, 
especially  those  that  take  part  in  the  Economic  and  Monetary  Union,  inadequate 
implementation of reforms can have negative effects beyond the borders of the Member 
State concerned.  

To  address  these  challenges,  the  Commission  decided  to  engage  in  closer 
cooperation  with  Member  States  to  help  them  implement  comprehensive  reforms 
that  foster  common  goals.  The  Commission  already  had  positive  experiences  with 
supporting reforms in Greece and Cyprus, in light of the euro crisis that started in 2009. 

5 

 
 
 
At  the  request  of  the  national  authorities,  the  Commission  created  the  Task  Force  for 
Greece (2011) and the Support Group for Cyprus (2013). The objective was to help these 
countries  make  a  series  of  reforms  in  a  wide  spectrum  of  public  policy  areas.  These 
reforms  formed  part  of  their  financial  assistance  packages  from  the  EU.  Given  the 
success  of these task forces, and seeing the benefits  that reforms  could  have in  making 
economies more flexible and competitive, the Commission decided to broaden the scope 
of the support provided. In 2015, the Structural Reform Support Service (SRSS) was set 
up to roll out support for reforms to all EU countries.  

In  2017,  a  dedicated  financing  instrument  was  adopted:  the  Structural  Reform 
Support Programme (SRSP). The SRSS, and subsequently, the Directorate-General for 
Structural Reform Support (DG REFORM), have been managing the programme since5. 
As laid down in the SRSP Regulation, the SRSP’s general objective is to contribute to 
institutional,  administrative  and  growth-sustaining  reforms  in  the  Member  States  by 
providing support to national authorities.6 The support may also help Member States who 
are preparing for participation in the euro area7. In order to achieve this general objective, 
the SRSP has set the following specific objectives8: 

a)  support the initiatives of national authorities to design their reforms according to 
their  priorities,  taking  into  account  initial  conditions  and  expected  socio-
economic impacts; 

b)  support the national authorities in enhancing their capacity to formulate, develop 
and  implement  reform  policies  and  strategies  and  in  pursuing  an  integrated 
approach ensuring consistency between goals and means across sectors; 

c)  support  the  efforts  of  national  authorities  to  define  and  implement  appropriate 
processes and methodologies by taking into account good practices of and lessons 
learned by other countries in addressing similar situations; 

d)  assist  the  national  authorities  in  enhancing  the  efficiency  and  effectiveness  of 
human-resource management, inter alia, by strengthening professional knowledge 
and skills and setting out clear responsibilities.  

5  During  the  time  period  evaluated,  the  Structural  Reform  Support  Service  (SRSS)  was  responsible  for 
managing the SRSP. On 1 January 2020, DG REFORM succeeded the SRSS. This SWD refers only to DG 
REFORM,  even  though  during  the  implementation  period  the  mid-term  evaluation  the  SRSS  was  the 
service responsible.  

6

 Although regional and local administrations can benefit from technical support under the SRSP, national 
administrations were the beneficiary authorities for the vast majority of the projects.   
7 Article 4 of the SRSP Regulation.  
8 Article 5(1) of the SRSP Regulation.  

6 

 
 
 
                                                           
The  programme’s  general  and  specific  objectives  are  also  described  in  the  intervention 
matrix  (Figure  1),  visualising  the  relations  between  the  different  components  of  the 
intervention.  

The SRSP is the main funding programme for projects providing technical support 
for reforms to EU countries. Support under the SRSP is available to any Member State 
facing  challenges  in  designing  and  implementing  reforms.  This  includes  reforms  to 
address  challenges  identified  in  the  European  Semester  process  of  economic  policy 
coordination (e.g. the CSRs), reforms to address EU priorities or reforms undertaken at 
the  Member  State’s  own  initiative.  Furthermore,  technical  support  can  be  provided  to 
help  Member  States  fully  implement  EU  law.  EU  countries  can  also  request  technical 
support  for  economic  adjustment  programmes.  Moreover,  since  the  amendment  to  the 
SRSP  Regulation  in  2018,  the  SRSP  can  also  finance  reform-related  actions  and 
activities to help Members States prepare for their entry to the euro area9. 

The technical support can be provided to help design and implement reforms that 
relate  to  a  broad  range  of  policy  domains,  including  the  following  public  policy 
areas: 

revenue administration and public financial management; 

 
  governance and public administration; 
  growth and business environment (including climate, energy and environment);  
 
 

labour market, education, health and social policy; 
financial services and access to finance. 

The support provided through the programme is demand-driven and tailor-made to 
suit the specific situation of the country concerned. Actions eligible for funding include 
the  formulation  of  policy  strategies  or  roadmaps,  the  provision  of  long-  and  short-term 
experts, and capacity-building activities, such as workshops, training and working visits. 
More examples of eligible actions are presented in the logic of intervention (Figure 1). 

The  technical  support  to  Member  States  is  intended  to  help  successfully  design  and/or 
implement the relevant reforms by tackling obstacles in the reform process. The results 
expected  in  the  short  term  include  the  provision  of  strategies,  reform  roadmaps  or 
drafts  of  new  legislation.  Other  short-term  results  expected  are  new  or  improved 
procedures  and  methodologies,  as  well  as  organisational  change  and  enhanced  human 
resource  management10.  The  programme’s  long-term  impacts  are  expected  to  be:  (a) 
strengthened institutions, governance and public administration; (b) enhanced sustainable 

9 New Article 5a of the SRSP Regulation. 
10 The expected results are also commonly referred to as outcomes.  

7 

 
 
 
                                                           
growth,  job  creation  and  social  inclusion;  and  (c)  enhanced  cohesion,  competitiveness, 
productivity and investment (see Figure 1).  

Figure 1: Logic of intervention (source: on the basis of the SRSP Regulation, EY 2020) 

Technical support projects can cover any part of the reform cycle – from identifying 
the  needs  to  monitoring  and  evaluating  the  outcomes.  At  the  same  time,  the  effective 
follow-up of the actions, in  terms  of actual implementation  of the reforms,  remains the 
prerogative  of  the  recipient  Member  State.  Ownership  of  the  reforms  by  the  Member 
State is essential for their successful implementation. 

As  the  SRSP  is  a  newly  established  funding  programme,  the  current  mid-term 
evaluation is  the first  major  evaluation exercise for the  programme  as a whole.  In 
order to ensure that the technical support to Member States is effective and cost-efficient, 
the Commission monitors the programme’s performance, based on the indicators that are 
set out in Annex 1 of the SRSP Regulation.  In addition to the mid-term evaluation, the 
Commission also uses other mechanisms to monitor the programme’s progress, including 
annual monitoring reports.  

8 

To contribute to institutional, administrative and growth-sustaining structural reforms in the Member StatesStrengthened institutions, governance and public administrationEnhanced cohesion, competitiveness, productivity and investmentEnhanced sustainable growth, job creation and social inclusionCONTEXTUAL FACTORSPolitical and institutional stability, as well as ownership of the Member States’ authoritiesWillingness of Member States to request technical supportEffective engagement of Member StatesAdequate monitoring systemCoherence with other EU/national interventions implemented by the respective Member StatesNEEDSGENERALOBJECTIVEELIGIBLE ACTIONSEXPECTED OUTPUTSSPECIFICOBJECTIVESEXPECTED IMPACTEXPECTED RESULTSAdoption of a strategyImproved internal working procedures, methodologies and processes / organisationAdoption of (new) procedures and actions to enhance the implementation of reformsOrganisational change, change management, improved human -resource managementAdoption of a draft of a new law, act or modification of an existing oneHYPOTHESISProvision of expertise related to policyOrganisation of local operational support in areas such as asylum, migration and border controlCollection of data and statisticsIT capacity buildingInstitutional, administrative or sectoral capacity building and related supporting actions at all governance levels, also contributing to the empowerment of civil society: seminars, conferences and workshops, working visits, training activityAction Plans, Roadmaps, Guidelines, Recommendations, StrategiesTraining materialsReports, including working visit reportsLocal operational support organised in areas such as asylum, migration and border controlNew or revised legislative proposalsData and statistics collectedMinutes / reports / agendas of seminars, meetings and workshopsLimited administrative and institutional capacityLimited application and implementation of Union lawChallenging design and implementation of growth-sustaining structural reforms in line with the Union’s economic and social goalsTo support the initiatives of national authorities to design their reforms according to their priorities, talking into account initial conditions and expected socio-economic impactsTo assist the national authorities in enhancing the efficiency and effectiveness of human-resource management, inter alia, by strengthening professional knowledge and skills and setting out clear responsibilitiesTo support the national authorities in enhancing their capacity to formulate, develop and implement reform policies and strategies and in pursuing an integrated approach ensuring consistency between goals and means across sectorsTo support the efforts of national authorities to define and implement appropriate processes and methodologies by taking into account good practices and lessons learned by other countries in addressing similar situationsProvision of short-term and long-term experts related to operational activitiesStudies, research, evaluationsCommunication projectsCompilation / dissemination of materialsAny other relevant activity to support objectives set out in Art. 4 and 5Operational activities realized with the support of short-term or long-term expertsCommon methodologies, indicators or benchmarks developedDevelopment, maintenance, operations and quality control of IT infrastructure and applications needed to implement the relevant reformsActions taken regarding programmes geared towards the digitalisation of public servicesStudies, research, analyses, surveys, evaluations, impact assessments, guides developed or published, educational materialCommunication projects; awareness-raising and information campaigns; media campaigns and eventsAny other relevant output associated to activities in support of general and specific objectives set out in Articles 4 and 5Materials to disseminate information compiled and published 
 
 
 
 
 
 
3.  IMPLEMENTATION / STATE OF PLAY 

3.1. How the SRSP is implemented 

The  submission,  analysis  and  selection  of  technical  support  requests  under  the 
SRSP  is  organised  in  annual  cycles,  in  accordance  with  the  provisions  of  the  SRSP 
Regulation.  The Member States  have until  the end of October to  submit their technical 
support requests for the budgetary cycle of the upcoming year11. Once they have done so, 
the Commission assesses these requests, based on the criteria set out in Article 7 of the 
SRSP  Regulation12.  To  avoid  double  funding  and  ensure  complementarity,  the 
Commission verifies whether requests overlap with actions implemented under other EU 
instruments  or  programmes.  The  Commission  then  selects  the  requests  that  are  best 
suited  for  funding.  Subsequent  to  this,  the  Commission  agrees  with  the  Member  State 
concerned  on  the  priority  areas,  the  objectives,  an  indicative  timeline,  the  scope  of  the 
support  measures  and  the  estimated  financial  contribution.  This  agreement  between  the 
Commission and the individual  Member State is the  cooperation and  support plan. The 
cooperation  and  support  plans  are  multiannual  documents,  signed  and    amended  each 
year  (if  need  be)  by  the  Commission  and  Member  States  requesting  support  from  the 
SRSP. The technical support projects financed through the programme do not require co-
financing by the Member State concerned. 

The  SRSP  is  implemented  on  the  basis  of  annual  work  programmes.  The  annual 
work  programmes  are  adopted  every  year  and  serve  as  a  Financing  Decision.  They  set 
out the support measures to be implemented for the year concerned, the dedicated annual 
budget,  and  the  expected  results.  The  annual  work  programmes  include  the  priority 
actions  for  the  programme,  as  agreed  between  the  Commission  and  the  individual 
Member States in the respective cooperation and support plans.  

To  facilitate  efficient  communication  with  the  Commission  and  the  coordinated 
submission  of  technical  support  requests,  each  Member  State  has  designated  a  national 
coordinating  authority  within  its  government  structures.  This  coordinating  authority 
functions as the Commission’s interlocutor for implementing the cooperation and support 
plans,  and  the  SRSP  in  general.  It  is  responsible  for  collecting  all  requests  for  the 
Member State in question, and subsequently submitting the requests to the Commission. 
The  coordinating  authority  can  also  be  responsible  for  prioritising  the  requests  before 
submitting them to the Commission. The national beneficiary authorities are the actual 

11  The  provision  of  support  under  Article  11  (support  to  be  funded  through  additional  voluntary 
contributions) follows a different timeline and is linked to the transfer of the additional contributions from 
the Member State concerned. 
12  These  criteria  are:  urgency,  breadth  and  depth  of  the  problems  identified,  the  support  needs  in  the 
relevant  policy  area,  the  analysis  of  socioeconomic  indicators,  and  the  administrative  capacity  of  the 
beneficiary authority concerned. 

9 

 
 
 
                                                           
recipients  of  the  technical  support,  for  example  the  thematic  ministries  that  have 
requested  assistance.  The  SRSP  never  gives  money  to  the  authorities,  only  technical 
support,  such  as  strategic  and  legal  advice,  studies,  training  and  expert  visits  on  the 
ground. The coordinating authority is expected to coordinate the ensuing implementation 
of the support measures in the country and is also encouraged to take appropriate action 
if challenges arise during the technical support’s implementation.  

3.2. Implementation of SRSP 2017 and SRSP 2018 

The budget for SRSP 2017 and SRSP 2018 was EUR 53 million (Figure 2)13. Under 
SRSP 2017, 159 requests from 16 Member States were selected to receive funding, out of 
the 271 requests submitted for a total of EUR 83 million – an amount almost four times 
the  EUR 22.5  million  budget  available  for  2017.  In  2018,  the  SRSP’s  second  year  of 
implementation,  there  was  an  increase  in  demand  from  Member  States.  In  total,  146 
requests from 24 Member States were selected for funding under SRSP 2018, out of the 
444 support requests submitted.  The submitted requests amounted  to  an estimated total 
value  of  EUR 152  million.  This  amount  was  five  times  the  budget  available  for  SRSP 
2018 (EUR 30.5 million). 

Funding source 

SRSP 2017 
SRSP 2018 
Total 

Number requests 
submitted  
271 
444 
715 

Number of requests 
selected 
159 
146 
305 

Available budget 

EUR 22.5 million 
EUR 30.5 million 
EUR 53 million 

Figure 2: Number of requests submitted and selected under SRSP 2017 and SRSP 2018 (source: Annual 
Monitoring Report 2017 and 2018) 

13 Under Article 11 of the SRSP Regulation, Member States can make an additional voluntary contribution 
to the programme, transferring their resources dedicated to technical assistance to the SRSP. A contribution 
from  a  Member  State  is  used  exclusively  to  support  requests  for  technical  support  in  that  particular 
Member State. In 2018, Bulgaria and Greece made use of this possibility, which amounted to an additional 
EUR 21.5 million. Furthermore, EUR 3 million were allocated to the Preparatory Action (2016). 

10 

 
 
 
 
 
 
 
 
 
                                                           
Figure  3  shows  the  number  of  requests  submitted  per  Member  State  under  SRSP  2017 
and SRSP 2018. 

Figure 3: Number of requests submitted for funding under SRSP 2017 and 2018 per 
Member State (source: SRSP Annual Monitoring Report, 2018) 

 ) 

The high demand from Member States in both 2017 and 2018 indicates a significant 
interest in the instrument. In line with the principle of sound financial management, the 
Commission  prioritised  requests  allowing  for  quick  delivery  of  support  and  rapid 
implementation  of  reforms  on  the  ground,  and  requests  addressing  defined  objectives, 
with strong results expected on the ground. 

Figure 4 shows the number of requests selected per Member State under SRSP 2017 and 
SRSP 2018.  

Figure 4: Number of requests selected for funding under under SRSP 2017 and SRSP 2018 per Member 
State (source: SRSP Annual Monitoring Report, 2018) 

11 

 
 
 
 
 
 
 
 
Figure  5  shows  the  circumstances  of  the  requests  that  have  been  selected  under  SRSP 
2017  and  SRSP  2018.  Over  half  of  the  selected  requests  were  related  to  implementing 
challenges  identified  in  the  context  of  economic  governance  processes,  in  particular  to 
CSRs. 

Circumstances of the selected requests 
Economic governance process, including CSRs  51% 
27% 
EU priorities 
7% 
EU law 
11% 
Member States’ own initiative 
4% 
Economic adjustment programmes 

SRSP 2017 

SRSP 2018 
55% 
29% 
6% 
7% 
3% 

Figure 5: Circumstances of the selected requests under SRSP 2017 and SRSP 2018 (source: SRSP 
Annual Monitoring Report 2017 and 2018) 

When  the  Commission  decides  to  fund  a  request  for  technical  support,  the  request 
becomes a project14.  In the selection process,  the  Commission  can decide  to  implement 
one large request through multiple projects. Similarly, the Commission can also decide to 
implement multiple related requests through one single project. Therefore, the number of 
selected requests can differ from the number of projects (i.e. support measures) that are 
implemented.  The  number  of  projects  approved  under  SRSP  2017  and  SRSP  2018  is 
provided in Figure 6.  

Funding source 
SRSP 2017 
SRSP 2018 
Total 

Number of approved projects 
144 
143 
287 

Figure 6: Number of projects approved under SRSP 2017 and SRSP 2018 (source: Commission’s 
internal IT system, EY 2020) 

In  total,  246  technical  support  projects  funded  under  SRSP  2017  and  SRSP  2018  were 
evaluated  as  part  of  the  mid-term  evaluation,  covering  90%  of  the  allocated  financial 
resources  under  the  two  budgetary  cycles.  More  than  50%  of  the  evaluated  projects 
under SRSP 2017 and SRSP 2018 were implemented in 7 Member States (Romania, 
Croatia,  Malta,  Cyprus,  Lithuania,  Slovenia,  and  Poland).  A  detailed  overview  of  the 
distribution of evaluated projects among EU countries is provided in Figure 715.  

14 Technically a project is a ‘support measure’ under the SRSP Regulation terminology. 
15 This figure excludes two multi-country projects. 

12 

 
 
 
 
 
                                                           
Figure 7: Number of projects funded under SRSP 2017 and 2018 that were evaluated 
per Member State (source: Commission’s internal IT system, 30 April 2019, EY 2020) 

Technical support under SRSP 2017 and SRSP 2018 covered all five thematic areas. 
As demonstrated in Figure 8, out of the evaluated projects funded under SRSP 2017 and 
SRSP 2018 the largest share of the approved budget was allocated to projects in area of 
growth, business environment and sectoral issues (EUR 13.5 million), followed by labour 
market,  education,  health  and  social  services  (EUR 11.4  million)  and  revenue 
administration and public financial management (EUR 9.1 million).  

Figure 8: Approved budget and number of projects implemented by thematic area under 
SRSP  2017  and  SRSP  2018 (source:  Commission’s  internal  IT  system,  30  April  2019, 
EY 2020) 

In line with the SRSP Regulation, the measures of the programme are implemented either 
directly  by  the  Commission  or  indirectly  in  accordance  with  the  Financial  Regulation. 
Under  direct  management  this  includes  public  procurement  contracts,  grants  and 
reimbursement  of  costs  incurred  by  external  experts.  The  underlying  procedures  for 
concluding contracts/agreements with technical support providers under the SRSP are in 

13 

 
 
 
 
 
 
 
 
 
line with the Financial Regulation and thereby in accordance with the principle of sound 
financial management.  

In line with the Financial Regulation, some activities are implemented under the indirect 
management  mode  in  view  of  the  type  of  the  implementing  entity.  Entities,  such  as 
international  organisations,  which  have  undergone  an  ex  ante  (/pillar-)  assessment  of 
their  procedures  and  systems,  guaranteeing  a  level  of  protection  of  the  EU  financial 
interests equivalent to that guaranteed by the Commission in direct management, can act 
under indirect management for the tasks they undertake to implement the actions.  

The annual work programme contains an overview of the actions and the related budget 
that  are  implemented  under  direct  (public  procurement  and  grants)  and  indirect 
management (agreements with  implementing organisations). For SRSP 2017 and SRSP 
2018,  around  98%  of  the  evaluated  budget  was  implemented  through  grants, 
procurement, indirect management and TAIEX16 (Figure 9). 

Figure  9:  Most  frequently  used  means  of  delivery  for  evaluated  projects  funded 
under SRSP 2017 and SRSP 2018 (source:  Commission’s internal IT system, 30 
April 2019, EY 2020) 

For  the  projects  evaluated  in  the  mid-term  evaluation,  the  types  of  providers  most 
involved in implementing projects under SRSP 2017 and SRSP 2018 were: 

international organisations (37%) 

 
  private companies (22%) 
  government bodies (8%) 
  public law bodies (7%) 

16

 TAIEX provides  short-term  expertise exchange by public  sector experts from Member States. It is  not 

contracted, but utilised through a Service Level Agreement between DG REFORM and DG NEAR. 

14 

 
 
 
 
 
 
 
                                                           
  bodies governed by private law with a public service mission (4%). 

These providers accounted for 78% of the evaluated budget allocated under SRSP 2017 
and  201817.  Looking  at  the  specific  measures  financed,  the  largest  proportion  of  the 
support measures implemented concern the following sectors/actions: 

revenue administration (11%) 
capital markets union (9%) 

SRSP 2017 
 
 
  healthcare system (9%) 
 
 
  better regulation (5%)18 

energy, including climate change (6%) 
education and training (5%) 

SRSP 2018 
 
tax policy (13%) 
 
education and training (9%) 
  digital public administration (8%) 
  governance (8%) 
  healthcare system (7%) 
  Capital Markets Union (5%)19 

Given that the evaluation took place in the middle of the implementation period of SRSP 
2017 and SRSP 2018, the majority of the technical support projects were still in progress. 
At the time of the mid-term evaluation, 22% of the projects had been completed20. 

17

 The  remaining share can be  attributed to other types of providers or  missing  information. For projects 
funded  under  Article  11,  the  providers  most  involved  were  international  organisations  and  government 
bodies, delivering approximately 81% of the projects evaluated. 
18 SRSP Annual Monitoring Report, 2017. 
19 SRSP Annual Monitoring Report, 2018. 
20 Date of consultation: 30 April 2019. SRSP 2017: 36.7% (44 out of 120). SRSP 2018: 7.9% (10 out of 
126). A completed project refers to a project whose final deliverables have been approved.  

15 

 
 
 
 
 
 
                                                           
4.  METHODOLOGY 

4.1. Fact-finding 

The mid-term evaluation was carried out by an external contractor and coordinated by the 
Directorate-General for Structural Reform Support (DG REFORM), with the support of 
an  inter-service  steering  group  of  other  Commission  departments.  The  contractor 
delivered an evaluation report, which served as a basis for this staff working document21. 

The mid-term evaluation assessed projects funded under the SRSP’s budgetary cycles of 
2017 and 2018, projects funded through the voluntary transfers of additional resources to 
the  SRSP  budget  by  Greece  and  Bulgaria  in  2018,  and  projects  funded  under  the 
Preparatory Action of 2016. Figure 10 provides an overview of the number of evaluated 
projects  per  funding  source.  The  mid-term  evaluation  covered  89.4%  of  the  allocated 
budget  for  2016-2018,  representing  303  implemented  projects  and  409  means  of 
delivery22.  The  staff  working  document  is  focused  primarily  on  the  two  budgetary 
cycles of the SRSP and refers to the Preparatory Action and the Article 11 only where 
relevant. The findings for evaluated projects funded under Article 11 and the Preparatory 
Action are similar to evaluated projects funded under SRSP 2017 and SRSP 2018. 

Funding source 

PA (2016) 
SRSP out of which: 
  SRSP (2017) 
  SRSP (2018) 

Article 11 (2018) 
Total 

Number of 
projects 
evaluated 
23 
246 
120 
126 
34 
303 

Approved financial 
resources (EUR) 

2 733 223 
47 709 027 
19 608 927 
28 100 100 
18 830 000 
69 272 250 

% out of the 
allocated financial 
resources 
91.1% 
90.0% 
87.2% 
92.1% 
87.6% 
89.4% 

Figure 10: Approved financial resources for the evaluated projects (source: Commission’s internal IT 
system and targeted interviews, EY 2020) 

The mid-term evaluation was divided into three main phases: (1) the inception phase, 
in which the structure of the evaluation was defined, based on a conceptual framework; 
(2) the data collection phase, in which the data was collected and structured, based on the 
data collection strategy  defined in  the first  phase;  and (3) the finalisation and feedback 

21 Ernst & Young, ‘Mid-term Evaluation of the Structural Reform Support Programme (SRSP) 2017-2020’ 
(2020),  available  on  https://data.europa.eu/doi/10.2887/656262.  Executive  summaries  are  available  on 
https://data.europa.eu/doi/10.2887/584399  (English),  https://data.europa.eu/doi/10.2887/244815  (French), 
and https://data.europa.eu/doi/10.2887/530192 (German). 
22

 Some of the projects (means of delivery) were excluded from the sample, as EY was a provider for some 

projects and some would be evaluated under Fiscalis (subject to a separate evaluation). 

16 

 
 
 
 
                                                           
phase,  in  which  the  intermediary  and  final  evaluation  results  were  reported  to 
stakeholders.  The  evaluation  was  carried  out  in  line  with  the  principles  commonly 
applied  for  the  evaluation  of  EU  initiatives,  as  laid  down  in  the  Better  Regulation 
guidelines.  

The  evaluation  comprised  different  fact-finding  tools  and  techniques,  such  as  desk 
research,  scoping  interviews,  targeted  interviews,  online  consultation  and  case  studies. 
Overall,  the  evaluation  relied  to  a  large  extent  on  stakeholder  consultation.  The  fact-
finding  exercises  built  on  the  evaluation  framework  that  was  designed  during  the 
inception  phase  of  the  report.  The  evaluation  framework,  including  the  evaluation 
questions  and the corresponding sources  of information,  is  provided in  Annex 3 of this 
staff working document.  

The  desk  research  involved  the  review  and  analysis  of  relevant  documentation.  These 
can  be  grouped  into  six  main  types:  (1)  EU  treaties  and  legal  acts,  i.e.  EU  legislation 
underpinning  the  SRSP,  the  Preparatory  Action  and  Article  11;  (2)  Commission 
preparatory documents, i.e. staff working documents, cooperation and support plans; (3) 
other  Commission  resources,  i.e.  internal  DG  REFORM  documentation;  (4)  online 
resources, including web pages of informative nature from official sources; and (5) other 
resources, including literature and studies.  

A  major  source  of  information  for  the  mid-term  evaluation  was  the  Commission  (DG 
REFORM) internal IT system for monitoring technical support projects.  Developed in 
2018,  this  system  provided  a  broad  range  of  quantitative  and  qualitative  information 
about  the  requests  and  projects  under  the  SRSP.  Data  extracts  from  this  system  were 
provided to the contractor for the analyses.  

In  addition  to  the  desk  research  and  information  from  the  Commission’s  internal  IT 
system, 16 semi-structured scoping interviews with DG REFORM officials were carried 
out  (heads  of  unit/cluster,  advisers  and  country  coordinators).  The  aim  of  these 
interviews  was  to  get  a  deeper  understanding  of  the  relevant  issues  for  the  different 
stakeholders.  The  intention  was  also  to  set  up  the  detailed  methodology  and  data 
collection tools. The scoping interviews were not project-specific.  

As a  follow-up to the scoping interviews, 31 semi-structured targeted interviews were 
carried out. This involved interviewing a sample of 25 DG REFORM project managers 
and  a  sample  of  staff  members  from  six  other  Commission  departments  involved  in 
SRSP  assessment/implementation  or 
in  managing  complementary  EU  funding 
programmes  or  policies.  To  ensure  that  the  samples  were  representative,  different 
elements  were  taken  into  consideration  when  selecting  the  interviewees,  including  the 
thematic areas of the technical support projects and their geographical coverage.  

17 

 
 
 
the  evaluation 

included  an  online  consultation 

targeting  185 
Furthermore, 
stakeholders23.  Feedback  was  collected from  9  coordinating  authorities, 107  beneficiary 
authorities  and  69  technical  support  providers24.  The  questions  in  the  questionnaires 
covered all the five Better Regulation evaluation criteria. In order to optimise the results 
of this consultation, the questions were adapted to the types of stakeholders, according to 
the  level  and  nature  of  engagement  in  the  technical  support  funded.  Both  the  targeted 
interviews and the online consultation were project-specific.  

Additionally,  the  Commission  carried  out  a  public  consultation  between  March  and 
June 2019 on its  official website.  The goal  was  to  collect  information  from  the general 
public, businesses, trade  associations  and interest  groups. However,  given  the very low 
number of respondents (11), the extent to which these results can be used is limited25. The 
number of respondents was low due to the fact that the SRSP specifically targets public 
administrations  and  bodies  –  not  individuals  –  and  that  targeted  interviews,  targeted 
consultations and scoping interviews had already been carried out. Public administrations 
and bodies were largely covered by the external contractor’s evaluation report. Moreover, 
it  is  presumed  that  the  programme  was  not  widely  known  among  the  general  public, 
given  that  it  had  only  recently  been  established.  Because  the  limited  response  to  the 
public consultation cannot serve as solid evidence, the staff working document refers to 
the public consultation results only for illustrative and transparency purposes.  

Lastly,  the  evaluation  included  case  studies  at  Member  State  level.  The  case  studies 
provided  an  analysis  of  how  the  SRSP  projects  were  carried  out  on  the  ground  in  four 
countries  (Croatia,  Greece,  Italy  and  Romania).  Later  in  the  process,  the  geographical 
coverage  of  the  interviews  was  extended  to  five  additional  Member  States  (Ireland, 
Latvia, Malta, Portugal  and Slovakia) so that the findings would be more relevant. The 
relevant Member States were selected on the basis of the number of approved projects, 
the allocated budget and the geographical coverage. The main purpose of the case studies 
was  to  assess  in  more  detail  the  SRSP’s  relevance,  effectiveness,  efficiency,  coherence 
and  EU  added  value  in  the  particular  Member  States.  The  case  studies  included 
interviews  with  coordinating  authorities  (8),  beneficiary  authorities  (22)  and  technical 
support  providers  (21).  These  interviews  related  to  28  evaluated  projects,  covering  the 
five thematic areas of the SRSP. The case studies functioned as a method for gathering 
additional information in order to have more robust answers to the evaluation questions 
(i.e. triangulation). 

23  The  mid-term  evaluation  targeted  only  beneficiary  authorities  that  received  technical  support  – 
authorities of which the technical support requests were not selected have not been consulted.  
24 In total 429 stakeholders were contacted: 230 beneficiary authorities (response rate: 46%), 173 technical 
support providers (response rate: 39%) and 26 coordinating authorities (response rate: 34%). The avarage 
response rate of the targeted online consultation was 43%.  
25 An overview of the results of the public consultation is provided in the annex of this document. 

18 

 
 
 
                                                           
4.2. Limitations and methodological issues  

As  required,  the  external  evaluator  contracted  for  this  assignment  carried  out  all  tasks 
under  the  scrutiny  of  an  inter-service  steering  group  and  the  steer  of  DG  REFORM. 
Overall,  the  quality  and  representativeness  of  the  collected  evidence  (data, 
documentation,  interviews,  online  consultation  and  case  studies)  are  considered  to  be 
satisfactory.  The  contractor  was  able  to  collect  extensive  and  meaningful  data  that 
allowed  conclusions  to  be  drawn.  Nevertheless,  there  were  some  limitations  and 
methodological issues. These were mitigated to the extent possible, with any limitations 
given due consideration.  

Firstly, the most important constraint to the robustness of the findings was the fact that 
the  SRSP  had  only  been  very  recently  established  when  the  mid-term  evaluation 
took  place.  The  Regulation  was  adopted  in  May,  and  the  first  call  was  issued 
immediately  after  that.  The  fact  that  the  mid-term  evaluation  was  performed  at  a  very 
early  stage  in  the  programme’s  implementation  limits  the  extent  to  which  fixed 
conclusions on the SRSP’s success can be drawn. This staff working document provides 
first conclusions and lessons learned, which might evolve during the SRSP’s continuing 
implementation.  

Secondly,  in  particular,  the  Commission’s  (DG  REFORM)  new  internal  IT  system 
was  not  in  place  yet  for  SRSP  2017  but  was  implemented  only  afterwards.  The 
system was evolving significantly, and many questions and categories of data were added 
only  at  later  stages  and  not  necessarily  retroactively  completed.  Therefore,  information 
was  to  some  extent  incomplete.  This  shortcoming  was  overcome  by  triangulating  the 
information  based  on  other  data  sources,  including  the  interviews,  online  consultation 
and case studies. Furthermore, the response rate to the online consultation was sufficient 
and ensured a balanced representation of stakeholders.  

Thirdly, the breadth and diversity of the programme’s activities posed an important 
methodological challenge, in that it would not  have been possible within the available 
resources  and  timeframe  to  cover  all  activities  in  the  detail  needed  to  draw  robust 
conclusions.  Moreover,  much  of  the  programme’s  support  plays  a  contributing  role 
alongside other factors, such as the action of national administrations. This contributing 
role is difficult to assess without in-depth qualitative research.  

Lastly, given the nature of a mid-term evaluation, many of the projects evaluated were 
still  ongoing  at  the  time  of  the  exercise.  Therefore,  results  and  impact  can  only  be 
measured to  a limited extent. A certain  share of the projects’ expected results were not 
yet delivered. In those cases, the likelihood of achieving the results was assessed only to 
the extent that information was available. The case studies confirmed that it is too early 
to expect considerable long-term impacts at this stage of the programme.  

19 

 
 
 
5.  ANALYSIS AND ANSWERS TO THE EVALUATION QUESTIONS 

The  mid-term  evaluation  assessed  the  performance  of  the  SRSP  against  the  following 
five  criteria:  relevance,  effectiveness,  efficiency,  coherence  and  EU  added  value.  The 
external  study  consisted  of  22  specific  evaluation  questions,  which  were  subsequently 
grouped according to  their topic in  order to  provide a coherent  framework. The overall 
analysis  is  based  on  evidence  from  both  the  external  evaluation  report  and  the 
Commission’s own sources. This chapter presents the analysis and provides the answers 
to the general evaluation questions. 

5.1. Relevance 

To  what  extent  are  the  SRSP  actions  appropriate  for  supporting  Member  States  in 
strengthening their administrative and institutional capacity, supporting the design of 
growth-sustaining structural reforms and the application/implementation of EU law? 

The overall assessment of the programme’s relevance focused on two aspects:  

 

 

the  relationship  between  the  needs  of  the  Member  States  at  the  time  of  their 
request  for  technical  support  and  the  objectives  and  actions  of  the  technical 
support provided;  
the continued suitability of the programme objectives. 

5.1.1.  Member State needs and the objectives of technical support 

The  representation  of  the  programme’s  logic  of  intervention  (Figure  1)  illustrates  the 
alignment of the SRSP design and specific objectives with the needs of the Member 
States at the time of their request for technical support.  

The programme’s logic of intervention identifies three challenges facing Member States: 

1)  Limited administrative and institutional capacity; 
2)  Challenges in designing and implementing growth-sustaining reforms in line with 

the EU’s economic and social goals; 

3)  Limited application and implementation of EU law.  

Both the beneficiary authorities and providers26 of the technical support consulted during 
the mid-term evaluation confirmed these as the main challenges that Member States were 

26 112 beneficiary authorities and 77 providers were consulted. Among the providers, 67.5% indicated ‘a) 
administrative  and  institutional  capacity’  and  49.5%  indicated  ‘b)  challenges  in  the  design  and 
implementation  of  structural  reforms  in  line  with  the  Union’s  economic  and  social  goals’  as  main 
challenges faced by Member States requesting technical support. 

20 

 
 
 
 
                                                           
facing  at  the  time  of  the  request  for  technical  support.  Figure  11  shows  that  ‘limited 
administrative/institutional capacity’ and ‘challenges in the design and implementation of 
structural  reforms’  were  the  challenges  most  frequently  mentioned  by  beneficiary 
authorities consulted.  

Figure 11: 'Which of the following challenges was your institution facing at the moment of requesting technical 
support under the SRSP?' 112 beneficiary authority respondents (source: targeted online consultations, EY 2020). 

With the case studies it was possible to specify these broad challenges. The case studies 
analysis  shows  that  often  the  institutions  responsible  for  designing  and  implementing 
structuralreforms  in  the  relevant  Member  States  had  a  limited  capacity  to  implement 
reforms due to less efficient and effective human resource management. This in turn 
weakened  the  organisation,  affecting  also  its  capacity  to  implement  enforcement 
measures.  Moreover,  these  institutions  face  bottlenecks  due  to  an  institutional  setting 
where competencies of sectoral and non-sectoral ministries overlap and responsibilities 
are  unclear.  Finally,  they  lack  a  systematic,  analytical,  and  integrated  approach  to 
implementing reforms that would ensure a regular assessment of priorities, contextual 
conditions,  expected  socio-economic  impacts  and  costs  and  benefits  of  different  policy 
options27.  

Analysis of the actions and activities28 funded under the SRSP revealed that the Member 
States’  needs,  as  requested  under  the  SRSP,  have  all  been  addressed  to  a  large 
extent  by  the  projects,  and  that  on  average,  each  project  covered  almost  two  of  the 
needs mentioned above. The distribution of the projects according to the Member States’ 

27

 Based on the interviews with stakeholders of the relevant case studies.  

28  The  analysis  measured  the  percentage  of  projects  whose  actions  and  activities  were  addressing  the 
Member States’ identified needs. 16.7% (41 out of 246) of the projects are related to all three of the needs 
mentioned above; 47.6% (117 out of 246) are related to two of the needs mentioned above; 35.0% (86 out 
of 246) addressed only one of the needs mentioned above. 

21 

 
 
 
 
 
                                                           
needs is provided in Figure 12. Respondents to the public consultation were also positive 
about the extent to which the SRSP responds to the needs of Member States29. 

Project funding source 

# of projects related to 
strengthening 
administrative and 
institutional capacity 

# of projects related to 
supporting the design 
and implementation of 
growth-sustaining 
structural reforms 

# of projects related to 
application / 
implementation of EU 
law 

SRSP 2017 

SRSP 2018 

82 

83 

88 

102 

37 

51 

Total 

% (out of 246) 

165 

67.1% 

190 

77.2% 

88 

35.8% 

Figure 12: Needs addressed by evaluated projects funded under SRSP 2017 and SRSP 2018 (source: 
Commission’s internal IT system and targeted interviews, EY 2020). 

By  pursuing  its  specific  objectives30,  the  programme  provided  technical  support  to 
national authorities to develop the administrative and institutional capacity they were 
lacking and that is necessary for them to design, formulate, develop and implement  the 
structural  reforms  they  need,  based  on  their  priorities,  conditions  and  expected 
socioeconomic  impacts.  The  support  also  aimed  at  helping  Member  States  to  adopt  an 
integrated  approach  to  reforms,  which  ensures  consistency  across  sectors,  as  well  as 
coherence  with  EU  priorities,  programmes  and  legislation.  Furthermore,  the  funded 
actions  aimed  at  supporting  national  authorities  to  put  in  place  new,  more  robust 
institutional processes and methodologies to enhance the efficiency and effectiveness of 
human-resource  management,  strengthening  professional  knowledge  and  skills  and 
setting out clear responsibilities. 

5.1.2.  Continued suitability of the programme’s objectives 

The SRSP’s objectives and actions were and continue to be relevant for Member States’ 
needs.  The  analysis  showed  that  the  SRSP  is  perceived  as  a  suitable  and  appropriate 
instrument  for  supporting  Member  States  in  strengthening  their  administrative  and 
institutional capacity. Almost all (97.3%) of the stakeholders31 consulted agreed that 
the  SRSP  is  a  suitable  instrument  for  providing  technical  support  to  design  and 
implement growth-sustaining structural reforms (Figure 13). 

29

  Four  to  five  respondents  (question  subdivided  per  policy  area)  expressed  the  view  that  the  SRSP 

 See logic of intervention (Figure 1).  

responds ‘to a high extent’ or ‘to the fullest extent’ to the EU Member State’s needs.  
30
31  97.3%  (180/185)  of  the  stakeholders  consulted  –  99%  of  beneficiaries  (106/107),  94.2%  of  providers 
(65/69),  and  all  coordinating  authorities  (9/9)  -  strongly  agreed  or  agreed  that  the  SRSP  is  a  suitable 
instrument to provide technical support.  

22 

 
 
 
 
 
                                                           
Figure 13: ‘The Structural Support Reform programme is a suitable instrument to provide technical 
support' (source: targeted online consultations, EY 2020) 

Moreover, the the majority of  the beneficiary  authorities consulted  (65%) consider that 
technical support is still needed32. The findings from the case studies33 suggest that this 
is  due  to  the  long-term  and  broad  nature  of  the  Member  States’  needs,  their 
interconnectedness  and  the  willingness  of  institutions  to  expand  the  application  of  the 
newly  developed  skills  and  capacity  to  implement  reforms  in  other  policy  areas.  The 
implementation  of  institutional,  administrative  and  growth-sustaining  reforms  in  the 
Member States is  a long-term  process  linked to  long-term needs. While  the SRSP  has 
supported the launch of this process, reforms require sustained effort over several years 
for economies to become more modern, flexible and competitive. From the case studies34 
it emerged that, due to the SRSP’s flexibility, the simple contracting and implementing 
procedures,  and  the  absence  of  co-financing  requirements,  Member  States  have  a 
tendency  to  consider  SRSP  as  a  relevant  instrument  to  help  design  and  implement 
national  reform  agendas.  Moreover,  the  implementation  of  institutional,  administrative 
and growth-sustaining reforms in the Member States is linked to wider needs. Therefore, 
the beneficiaries still need support for designing reforms both within the same thematic 
areas  and  in  different  ones.  Furthermore,  the  incremental  value  of  the  skills  and 
capacity developed by the beneficiary of technical support should also be considered. 
In  other  words,  once  the  beneficiary  gains  experience  with  the  instrument,  they  can 
identify other needs and potential areas of action, and thus plan to use the instrument to 
enlarge the scope and breadth of the reforms to implement.  

32 When asked about the extent to which their institution still needed technical support, 65% (i.e. 69 out of 
107) of the beneficiaries declared they need it to a large or to a very large extent. 
33
 Based on interviews with 12 beneficiary authorities of the case studies.  
34 Based on interviews with 12 beneficiary authorities of the case studies.  

23 

 
 
 
 
 
                                                           
In  conclusion,  the  challenges  faced  by  Member  States  as  defined  in  the  programme’s 
logic  of  intervention  are  aligned  with  the  main  challenges  expressed  by  the  consulted 
beneficiary  authorities  and  providers.  Almost  all  stakeholders  consulted  (beneficiary 
authorities, coordinating authorities and providers) indicated that the SRSP is a suitable 
instrument for providing technical support. The programme is perceived as a significant 
tool  for  improving  the  Member  States’  capacity  to  define  more  efficient  processes  and 
methodologies  and/or  progress  towards  more  effective  human  resource  management. 
Since its second year of implementation, the SRSP has seen a regular increase in demand 
for  the  programme  from  Member  States35.  This  is  the  first  indication  that  the  support 
provided by the Commission continues to be needed and is still relevant. 

35 Within the scope of this evaluation, data from SRSP2017 and SRSP2018 show a considerable increase. 
24 Member States submitted 444 requests for support (the SRSP 2017 round saw 271 requests for support 
from 16 Member States). 

24 

 
 
 
  
 
 
 
                                                           
5.2. Effectiveness 

How successful is the SRSP in achieving or progressing towards its objectives? 

For the criterion  of  effectiveness,  the  mid-term  evaluation  examined to  what  extent the 
objectives  set  were  achieved  and  how  this  was  linked  to  the  EU  intervention.  The 
effectiveness  of  an  intervention  is  measured  in  terms  of  both  results  and  impacts. 
However, the mid-term evaluation of the SRSP was carried out while the programme’s 
implementation was still at a very early stage. Considering that impacts are expected only 
in the long-term, sometimes years after completion of the intervention, a solid assessment 
of the SRSP’s impacts was not possible at this stage. Therefore, the evaluation focused 
mainly on the SRSP’s effectiveness in terms of results36.  

The overall assessment of the programme’s effectiveness was composed of the following 
three sub-analyses: 

  an examination of the extent to which the technical support projects were well-

designed; 

  an  examination  of  the  extent  to  which  the  planned  activities  and  actions  were 

feasible and expected results had been achieved; 

  a preliminary exploration of the programme’s achieved impact on the ground. 

In addition to these three specific areas of analysis, the evaluation also aimed to identify 
contextual  factors  that  either  encouraged  or  hindered  progress  towards  the  results  and 
impact.  

5.2.1.  Design of the technical support 

The  evidence  collected  during  the  evaluation  shows  that  the  objectives  defined  at 
project level corresponded with the specific objectives of the SRSP. The majority of 
projects evaluated (around 60%) had links with more than one specific objective37. These 

36  According  to  Article  16(3)  of  the  SRSP  Regulation,  the  evaluation  of  the  long-term  impact  of  the 
programme is subject to an ex post evaluation. 
37 More than 40% of the projects evaluated  had as objective support to the design of reforms. More than 
50% were linked either to supporting the development and implementation of reform policies or supporting 
the  definition  and  implementation  of  processes  and  methodologies.  Additionally,  more  than  10%  of  the 
projects evaluated had support to more efficient and effective human resource management as one of their 
objectives. Around 60% covered more than one specific objective. The evidence collected covers projects 
under  SRSP  2017  and  SRSP  2018  and  was  based  on  the  Commission’s  internal  IT  system,  updated  in 
accordance with targeted interviews with project managers.  

25 

 
 
 
 
 
 
 
 
 
 
 
                                                           
 
 
outcomes demonstrate that the technical support projects are designed in such a way as to 
contribute to the objectives set out in the SRSP Regulation.  

Furthermore, the evaluation shows that in most cases (85%), initial challenges faced by 
the  beneficiary  authorities  when  preparing  the  support  requests  were  taken  into 
account when the projects were designed and implemented38. At the same time, the 
findings show that when the SRSP actions were designed, the relevant socio-economic 
impacts  were  largely  taken  into  account  as  well  (72%  of  the  cases  evaluated)39.  The 
socio-economic 
inclusion,  sustainable  growth, 
competitiveness, productivity, investment, job creation and enhanced cohesion.  

impacts  mostly  related 

to  social 

In  addition,  the  evaluation  suggests  that  SRSP  actions  are  often  designed  in  a 
comprehensive  manner,  by  paying  attention  to  accompanying  measures  and/or  the 
appropriate sequencing of reform measures – i.e. multiple projects that aim to address the 
same  reform  challenge,  for  instance  from  different  angles  or  at  different  points  in  the 
reform  design  or  implementation  timeline.  In  Member  States  such  as  Cyprus,  Czechia, 
Croatia,  Greece,  Romania  and  Slovenia,  between  40%  and  60%  of  the  projects  were 
implemented in connected reform areas, encompassing in-built synergies. This indicates 
that  different  SRSP-funded  projects  are  linked  to  each  other  in  order  to  achieve  a 
comprehensive  approach  towards  the  reform  challenge.  Generally,  in  Member  States 
where  the  total  number  of  projects  is  higher,  the  degree  of  interconnection  between 
projects also tends to be higher40.  

Lastly, the evidence collected shows that in the majority of the SRSP projects evaluated, 
the successful experiences and lessons learned from other EU countries are included 
in the design of the activities. Almost 60% of the projects evaluated included activities 
allowing  for  the  exchange  of  best  practices,  such  as  short-term  expert  missions,  study 
visits,  workshops,  seminars  and  conferences41.  In  addition  to  the  exchange  of  EU 

38 Targeted  online consultation of 113 replies  from beneficiary authorities to  this question: 85% confirm 
(completely taken into account (41%) or to a large extent taken into account (44%)); 11% to a small extent 
or not at all.  
39 Online consultation of replies from beneficiary authorities: 72% (81 out of 113) said that socio-economic 
impacts had been taken into account  when defining the content of the technical support requests. In cases 
in which socio-economic impacts had not been taken into account, respondents indicated in most cases (5 
out of 7) that the project did not have socio-economic indicators.  
40 The degree of comprehensiveness/complementarity was measured on the basis of the number of projects 
addressing  a  topic  that  is  already  covered  by  at  least  one  other  project,  compared  to  the  total  number  of 
projects implemented in that Member State.  
41 58.5% of the projects evaluated (144 out of 246); 56.4% of the total value of all projects evaluated under 
SRSP 2017 and SRSP 2018. This conclusion is based on the assumption that ‘short-term expert missions’, 
‘study visits’ and ‘workshops/conferences/seminars’ are eligible activities that imply the exchange of good 
practices  and  lessons  learned  from  other  EU  countries.  Evidence  based  on  the  Commission’s  internal  IT 
system, updated after interviews with  project managers and the targeted online consultation  were  used to 
confirm the results.  

26 

 
 
 
 
 
 
                                                           
Member State expertise, workshops and other training activities were also  organised by 
experts from international organisations or other experts.  

Overall,  these  findings  suggest  that  the  technical  support  projects  are  designed  with 
satisfactory consideration of the specific situation of the individual beneficiary authority 
and  the  related  socio-economic  impacts.  Technical  support  is  generally  provided  in  a 
comprehensive manner, while often incorporating in the project’s design the exchange of 
best practices between EU countries. All of these aspects can contribute to effective and 
targeted technical support activities. 

5.2.2.  Feasibility and achieving the expected results 

To examine the feasibility of the programme’s activities, the evaluation focused on the 
extent  to  which  the  implementation  of  technical  support  projects  is  in  accordance  with 
the planned schedule. The collected evidence shows that the greatest share of the projects 
under SRSP 2017 and SRSP 2018 are on track (84.1%), i.e. the project’s implementation 
is  aligned  with  the  planned  start  and  end  date42.  This  indicates  that  the  needs  of  the 
Member State and subsequent activities are very often translated into realistic time 
schedules. Some projects (around 5%) were faced with delays43. The targeted interviews 
with  DG REFORM  project  managers suggested  that these delays  were often caused by 
political  changes  or  changes  in  the  beneficiary  authority’s  needs.  Delays  also  occurred 
due to challenges in identifying the most suitable technical support provider.  

The  evaluation  also  assessed  to  what  extent  the  projects  have  achieved  their 
results44. In those cases where projects had not yet achieved their expected results, which 
was common given that many projects were still in progress at the time of evaluation, the 
likelihood  of  achieving  the  results  was  measured.45  As  outlined  in  Figure  14,  the 
evaluation  findings  show  that  most  of  the  projects  completed  under  the  SRSP  have 

42 84.1% of the evaluated projects under the SRSP 2017 and SRSP 2018 (207 out of 246). 
43

 4.9% (16 out of 246) of the evaluated projects under SRSP 2017 and SRSP 2018 were marked as having 

delays. 
44  The  achievement  of  expected  results  refers  to  expected  results  that  were  defined  in  the  Commission’s 
internal  IT  system  or  specified  in  the  targeted  interviews.  The  assessment  of  the  extent  to  which  the 
expected results of the evaluated projects had been achieved was based on an ad hoc scoring system. For 
each project for which information was available, a  score  was assigned on the basis of the percentage of 
expected results that  were produced, based on the  information available in the  Commission’s internal IT 
system and following the targeted interviews with project managers. 
45

 For projects where the results had not been achieved, a second assessment was carried out regarding the 
likelihood that the results would still be achieved, assessing the extent to which factors existed that could 
hinder or contribute to the achievement of expected results.  

27 

 
 
 
 
 
 
 
                                                           
achieved their expected results  (70.4%).46 The majority of the completed  projects that 
had  not  yet  achieved  their  results  had  a  high  likelihood  of  still  doing  so  (63.6%)47.  As 
anticipated,  only  a  small  number  of  ongoing  projects  has  already  achieved  their  results 
(4.2%).  The  evidence  shows,  too,  that  most  of  the  ongoing  projects  have  a  high 
likelihood of achieving their results (86.4%)48. The likelihood of achieving the results 
is low for 13.2% of the completed and ongoing projects.  

0%

20%

40%

60%

80%

100%

Completed

38 

7 

4 

5 

Ongoing

8 

95 

12 

74 

Projects for which results have been already achieved
Projects for which the likelihood to achieve the results is high
Projects for which the likelihood to achieve the results is low
Projects for which no information is available

Figure 14: Effectiveness of SRSP projects (source: Commission’s internal IT system and targeted 
interviews, EY 2020). 

The  evaluation  also  assessed  the  contextual  factors  influencing  the  delivery  of  project 
outputs. These contextual factors can either be positive or negative (see Figure 15).  

46

 The achievement of results is not entirely dependent on the status of the implementation of the projects. 
It  is  possible  to  encounter  completed  projects  that  have  not  entirely  achieved  their  results,  as  well  as 
ongoing projects that have achieved their results. 
47 These completed projects have gone beyond their planned end date, but activities are still ongoing. 
48 95 projects out of 110.  

28 

 
 
 
 
 
 
 
 
                                                           
Figure 15: Factors influencing the delivery of outputs (source: targeted online consultation, EY 2020) 

The most common positively influencing factors are: the level of commitment from all 
stakeholders  (45.6%);  the  provider’s  level  of  expertise  (15.2%);  and  the  political 
circumstances (15.2%). The most common negatively influencing factors are: the level 
of  collaboration  among  all  actors  involved  (29.7%);  resource  constraints  (29.7%);  and 
political circumstances (21.7%)49.  

In  conclusion,  the  evidence  from  the  mid-term  evaluation  shows  that  the  SRSP  is 
progressing  towards  achieving  its  results,  taking  into  account  the  evaluation’s 
limitations  as  many  activities  are  still  in  progress.  The  Member  States’s  needs  are  to  a 
good extent translated into feasible support actions under the SRSP. The majority of the 
projects  adhere  to  the  planned  schedules,  which  implies  that  the  designed  activities  are 
realistic. Meanwhile, the number of projects with delays is low. Furthermore, most of the 
projects  are likely to  achieve their results, or have already done so. A small number of 
completed  projects  did  not  achieve  their  results.  A  lack  of  collaboration  among 
stakeholders is  a challenging factor  for delivering the projects’ outputs. Changes in the 
political structures of the beneficiary authority also pose an evident risk for the effective 
implementation of technical support. These and other negative factors should be avoided 
and  mitigated  to  the  extent  possible.  Optimising  the  exchange  of  good  project 
management  practices  among  DG  REFORM  project  managers,  for  instance,  through 
dedicated  training  or  knowledge-sharing  platforms,  could  help  to  further  increase  the 
number of projects  that achieve their results.  At the same time, it should be underlined 
that technical support for reforms is often most needed in institutions that are embedded 
in  challenging  political  and  governmental  structures.  Therefore,  some  degree  of  risk  is 
inevitable.  Reinforcing  the  commitment  from  all  stakeholders  in  combination  with 
effective  collaboration  and  adequate  technical  expertise  can  increase  the  likelihood  of 
achieving the projects’ results.  

49 Evidence from the targeted online consultation.  

29 

 
 
 
 
 
 
                                                           
5.2.3.  Achievement of impacts 

As stated earlier, it is too early to measure the long-term impacts of the SRSP. The 
same  goes  for  examining 
the  programme’s  unintended  or  unexpected  effects. 
Nevertheless,  the  stakeholders’  perception  at  this  early  stage  was  used  to  assess  the 
extent  to  which  technical  support  projects  helped  the  Member  States  to  implement  the 
relevant  reforms,  as  planned  in  the  cooperation  and  support  plans.  The  evaluation  also 
explored the contextual factors under which the results of SRSP actions would produce 
impacts on the ground. 

Figure 16: The extent to which changes were observed in the beneficiary institutions between 2016 and 
2018 and SRSP impacts (source: targeted online consultation, EY 2020). 

The evidence collected from the targeted online consultation shows that stakeholders see 
signs of improvement with respect to the needs that the beneficiary authorities are facing 
(Figure 16). The highest perception of improvement is with ‘increased administrative and 
institutional  capacity’  (73.7%).  Perceived  improvements  are  the  lowest  for  ‘improved 
application  and  implementation  of  EU  law’  (42.1%).  The  majority  of  the  stakeholders 
consider  that  these  improvements  are  related  to  the  technical  support  received  through 
the programme50. The  respondents to the public consultation were  also positive in their 

50 The number of answers and respondents per category is 38.  

30 

 
 
 
 
 
 
 
 
 
                                                           
view on the extent to which technical support has helped to achieve the SRSP’s specific 
objectives51. 

The evaluation  also  looked closely into the specific  case studies. Concrete examples of 
impact on the ground include: 

  Support to the spending review process in Slovakia: the spending review resulted 
in the adoption of legislation, and Slovakia achieved EUR 83 million in savings 
in 2017 (0.09% of GDP); 

  Education  (curricular)  reform  in  Croatia:  the  pilot  phase  of  curricular  reform 
was implemented in 74 schools in the 2018/2019 school year, and 400 tutors were 
trained under a follow-up project; 

  A  strategy  for  internal  auditing  and  training  internal  auditors  at  local 
government level in Romania: the strategy for internal auditing was produced and 
adopted,  and  524  city  halls,  prefectures  and  county  councils  were  subject  to  an 
internal audit.  

The  evidence  collected  through  the  consultation  of  stakeholders  shows  that  the  most 
commonly  mentioned  factor  positively  influencing  the  achievement  of  impacts  is  the 
beneficiary authority’s commitment52. A lack of political support and stability is the most 
commonly mentioned negative factor for achieving the results53.  

In  conclusion,  taking  into  account  the  limitations  of  stakeholders’  perceptions  and 
individual case study assessment, the achieved impacts are considered proportionate 
to the level of completion of the programme. However, it would be premature to draw 
vast  conclusions  on  the  SRSP’s  impact  when  most  of  the  projects  are  still  in  progress. 
Even  achieving  the  project’s  expected  results  does  not  guarantee  that  there  will  be  an 
impact  on  the  ground.  Thus,  a  comprehensive  evaluation  at  a  later  stage  of  the 
programme is essential to provide better substantiated evidence for measuring the extent 
to which an impact has been achieved in the long term.  

51  Three  to  six  respondents  (question  subdivided  per  objective)  indicated  that  the  technical  support 

contributed ‘to a high extent’ or ‘to the fullest extent’ to achieving the SRSP’s objectives.  

52  56.0%  of  respondents  named  this  as  a  positively  influencing  factor  (14  mentions;  25  respondents); 
respondents were given the possibility of providing more than one answer.  
53  Lack  of  political  support:  55.6%  (5  mentions);  lack  of  political  stability:  33.3%  (3  mentions;  9 
respondents); respondents were given the possibility of providing more than one answer. 

31 

 
 
 
 
 
 
 
 
 
                                                           
5.3. Efficiency  

To what extent have the SRSP programme cycle and actions been time-efficient, cost-
effective, and aligned to the needs of the Member States?  

Efficiency  considers  the  relationship  between  the  resources  used  and  the  changes 
generated  by  the  programme.  Differences  in  the  way  the  support  is  designed, 
implemented  and  monitored  can  have  a  significant  influence  on  the  effects.  Therefore, 
assessing  the  programme’s  efficiency  is  also  related  to  verifying  whether  other  choices 
could have maximised the use of resources for the greater benefit of the beneficiaries.  

The overall assessment of the SRSP’s efficiency focused on three aspects:  

 

 

 

the  operational  efficiency  of  the  SRSP  governance,  in  terms  of  the  time-
effectiveness  of  the  SRSP  cycle,  including  its  monitoring  tools,  as  well  as  the 
quality of the requests and the cooperation with other Commission departments; 
the cost-effectiveness  of the support provided, in  terms  of the correspondence 
between the results achieved and the budget allocated;  
the time-efficiency of the budget execution. 

In parallel, the analysis looked at aspects of simplification, pinpointing areas where there 
is potential to reduce inefficiencies and simplify the intervention. 

5.3.1.  Operational efficiency of the SRSP governance 

The  assessment  of  the  efficiency  of  the  SRSP  governance54  focused  on  the  time-
effectiveness  of  the  programme  cycle,  including  the  monitoring  mechanisms  in  place, 
cooperation with other Commission departments and the improved quality of the requests 
for technical support.  

Time-effectiveness was analysed by looking at the duration of the different phases of the 
programme and verifying whether their duration was aligned with the timing of the needs 
of  the  target  group,  i.e.  the  beneficiary  authorities  in  the  Member  States.  An  important 
methodological  limitation  must  be  underlined.  The  analysis  considered  only  technical 
support projects funded under SRSP 2018 (126 in total); due to the timing of the SRSP 
Regulation’s  adoption,  SRSP  2017  followed  a  considerably  shorter  cycle.  Therefore, 

54  For  this  mid-term  evaluation,  SRSP  governance  refers  to  the  formal  and  informal  arrangements  that 
determined how decisions were made and how SRSP projects were implemented between 2016 and 2018. 
These arrangements include processes, roles and responsibilities that have been established in order to help 
achieve the programme’s objectives. 

32 

 
 
 
 
                                                           
2018 was the first year in which it was possible to adopt the planned timetable. Figure 17 
shows the SRSP annual cycle.  

Figure 17: The SRSP annual cycle55 

Based on the analysis of the Commission’s  (DG REFORM) internal IT system and the 
results  of  the  targeted  interviews,  it  emerged  that  for  the  SRSP  2018  cycle,  it  took  on 
average  5  months  for  the  selection  process  to  be  completed:  from  the  October  2017 
deadline for submitting the technical support requests to the approval in March 2018 of 
the Financing Decision and related SRSP annual work programme containing the full list 
of  projects  selected  for  funding.  This  period  comprised  the  phases  of  submission, 
analysis  and  selection  of  the  technical  support  requests,  as  well  as  the  adoption  of  the 
annual  work  programme  and  the  Financing  Decision  and  the  conclusion  of  the 
cooperation and support plans. Once the Financing Decision was adopted, for most of the 
projects (81%)56 it took at least 6 months before the planned technical support started to 
be delivered57. Figure 18 shows the distribution of evaluated projects under SRSP 2018 
by starting date.  

55  The  cooperation  and  support  plans  are  being  prepared  before  the  adoption  of  the  Financing  Decision. 
After the Financing Decision is adopted, the cooperation and support plans are signed.  
56 102 out of 126 of the SRSP 2018 projects.  
57  The  start  of  the  project  is  considered  to  be  the  launch  of  the  procedures  for  concluding 
contracts/agreements with technical support providers. 

33 

 
 
 
 
                                                           
Figure 18: Distribution of projects under SRSP 2018 by starting date (source: Commission’s internal IT 
system and targeted interviews, EY 2020) 

The average planned duration of the projects under SRSP 2018, from the procedures for 
concluding  contracts/agreements  with  technical  support  providers  to  the  completion  of 
the project58, was 13 months. As demonstrated in Figure 19, the planned duration of most 
projects  (71%)59  was  between  6  and  18  months.  Therefore,  on  average,  the  whole 
process  could take  24  months,  from the submission of the  request  by the Member 
States to the completion of the technical support project60.  

Figure 19: Planned duration of contracting and implementation phases under SRSP 2018 (source: 
Commission’s internal IT system and targeted interviews, EY 2020) 

The  targeted  online  consultations  of  both  beneficiaries  and  providers  confirmed  the 
alignment  of  the  SRSP  timeline  described  above  with  the  timing  of  the  target 
group’s  needs:  74.1%  of  the  beneficiaries  consulted  and  66.6%  of  the  national 
coordinating  authorities  consulted  agreed  entirely  or  to  a  large  extent  that  the  period 

58 Project completed includes the project’s evaluation, after the activities have been carried out. 
59 89 out of 126 responses to this question. 
60 The duration of each phase was calculated based on official documents or by computing averages based 
on data from the Commission’s internal IT system. 

34 

511662311104%8.7%52.4%18.3%8.7%7.9%0%10%20%30%40%50%60%020406080100120Not available< 6 months6 - 12 months13 - 18months19 - 24months> 24 monthsNumber of projectsShare of projects out of total 
 
 
 
 
                                                           
between the submission of the technical support requests and the start of the delivery of 
the technical support was timely and efficient. Moreover, 84.6% of the beneficiaries, 
all  coordinating  authorities,  and  86.2%  of  the  providers  agreed  entirely  or  to  a  large 
extent that the implementation and delivery of the technical support was delivered on 
time  and  in  accordance  with  the  needs  of  the  beneficiaries61.  Respondents  of  the 
public consultation also positively assessed the programme’s efficiency62. Finally, 82.6% 
of  the  providers  consulted  agreed  and  strongly  agreed  that  the  duration  of  the  the 
procedures  for  concluding  contracts/agreements  was  aligned  to  the  needs  of  the 
beneficiaries.  

Although in general, stakeholders agreed that the current programme cycle is in line with 
the  beneficiaries’  time  needs,  with  the  targeted  interviews  and  case  studies  it  was 
possible to  identify  some areas  where  there is  room for improvement. For example, 
some of the providers63 considered that the duration of the discussions  on the details of 
the  contracts/agreements  was  not  always  proportionate  to  the  duration  of  the 
contracts/agreements.  Some  beneficiaries  signalled  that  they  would  have  wanted  DG 
REFORM  to  involve  them  more  in  the  process  of  selecting  the  providers,  even  though 
most  beneficiaries64  were  pleased  with  the  interaction  with  the  provider.  One  in  four 
considered  that  the  alternatives  offered  were  limited  and  did  not  represent  the  most 
adequate choice for their needs. 

A  specific  area  where  it  emerged  that  there  could  be  room  for  improvement  was  the 
monitoring of the SRSP. Being a legal requirement, the systematic and regular review 
of progress and quality was continuously performed throughout the annual SRSP cycle, 
thus  helping  to  ensure  the  programme’s  efficiency.  The  majority  of  stakeholders  had  a  
largely  positive  perception  of  the  procedures  and  tools  used  by  the  Commission  to 
monitor the technical support’s implementation and saw them as effective. This was true 
for  79.8%  of  the  beneficiaries,  66.6%  of  the  coordinating  authorities  and  83%  of  the 
providers who took part in the targeted online consultation65. However, a more in-depth 
analysis of both the Commission’s practical monitoring arrangements (based mainly on 
the  data  stored  in  the  Commission’s  internal  IT  system)  and  Member  State  practices 

61  In  numbers  this  corresponds  to  88  of  the  104  beneficiary  authorities,  9  of  the  9  national  coordinating 
authorities, and 57 of the 69 providers who answered this question of the targeted online consultation.  
62 The majority of the respondents view the SRSP as being to some extent, to a high extent or to the fullest 
extent efficient (rapid, timely, cost-effective) compared to other EU, national or local interventions.  
63 2 of the 21 providers interviewed.  
64 20 of the 21 beneficiary authorities stated that the interaction with the providers was good. 
65 In response to the targeted online consultation question ‘Do you agree with the following statement: The 
procedures and tools used to monitor the implementation of the technical support are effective?’, 79.8% of 
the  beneficiaries  (83  out  of  104)  answered  either  ’agree’  or  ‘strongly  agree’  while  6.7%  (7  out  of  104) 
answered either ‘disagree’ or ‘strongly disagree’; 83% of the providers (54 out of 65) answered ‘strongly 
agree’ and ‘agree’; 66.6% of the coordinating authorities (6 out 9) answered ‘agree’ and ’strongly agree’ 
while 22.2% answered ‘disagree’ (2 out of 9). 

35 

 
 
 
                                                           
revealed  that  all  the  coordinating  authorities  consulted  wanted  additional  support  from 
DG  REFORM  in  order  to  understand  their  monitoring  prerogatives  and  to  adapt  their 
review  capacity. The existence of different  national  approaches  to  monitoring  activities 
showed  the  need  for  coordination  and  a  procedural  framework  that  could  help 
coordinating  authorities  and  beneficiaries  to  have  a  clear,  strategic  overview  of  the 
overall  support  received  as  well  as  progress  and  results  achieved.  This,  in  turn,  would 
help  to  maximise  the  effects  of  the  technical  support,  allowing  national  authorities  to 
draw lessons and better define future needs.  

It is important to underline that since the SRSP 2018 cycle, the Commission design of the 
monitoring mechanism has evolved significantly and continues to do so. Analysis of the 
information available in April 2019 on the Commission’s internal IT system revealed that 
improvements were needed, in particular to have more complete information on expected 
activities, expected results, state of play of activities and results. Hence, as a result of the 
internal feedback processes in DG REFORM, the Commission’s working methods have 
been  continuously  developed,  reviewed  and  improved.  Furthermore,  additional  tools 
have been used to monitor the progress of the projects. Progress reports have been used 
by the providers as a formal mechanism to inform the beneficiaries and DG REFORM of 
the  progress  of  projects.  At  the  same  time,  meetings  and  events,  as  well  as  calls  and 
emails have served as informal tools to ensure that all stakeholders were updated on the 
progress  of  projects.  From  the  targeted  consultations,  it  also  emerged  that  the  good 
collaboration  with  and  continuous  support  from  the  DG  REFORM  staff  were  strong 
determinants of the quality of the monitoring system.  

Similarly,  the  stakeholders  had  a  positive  perception  of  the  coordination  between  DG 
REFORM  and  the  rest  of  the  Commission.  All  the  other  Commission  departments 
interviewed saw their cooperation with DG REFORM as efficient66. 

The  coordination  mechanism  was  structured  in  such  a  way  that  potential  overlaps  and 
double  financing  were  identified  and  avoided  both  at  the  technical  level,  through  the 
feedback provided by the Commission departments on technical support requests, and at 
strategic level,  through the High-Level  Steering  Committee meetings of  representatives 
of different Commission departments. Dialogue between DG REFORM’s policy officers 
and  their  counterparts  in  the  relevant  Commission  departments  has  also  been 
strengthened.  This  system  is  a  determining  factor  in  making  the  SRSP  action  efficient 
and coherent with other EU programmes (for more information on the coherence of the 
SRSP, see Section 5.4). While the coordination mechanism was effective (i.e. no double 
funding), it was also pointed out that sharing information on the actual implementation of 

66 The relevant question in the scoping interviews was: ‘How efficient do you find the cooperation with the 
SRSS  and  the  other  Commission  Services  within  the  coordination  mechanism?’  6  of  the  6  Commission 
departments interviewed said the cooperation was efficient. 

36 

 
 
 
                                                           
the projects, their outcomes and lessons learned could lead to more efficient cooperation 
between DG REFORM and the Commission departments. This, in turn, would result in 
better assessment of overlaps and complementarities and make it easier to assess progress 
with respect to the implementation of country-specific recommendations.  

One of the most prominent factors that mitigates such considerations to a large extent is 
the  use  and  continuous  development  of  the  Commission’s  internal  IT  system,  which 
fostered the exchange of information between DG REFORM and the other Commission 
departments,  as  pointed  out  by  most  of  the  Commission  departments  consulted  (4 
Directorates-General  out  of  6).  The  IT  system  facilitated  the  cooperation  mechanisms 
because  it  enabled  easier  monitoring  (i.e.  a  centralised  approach,  extraction  of  reports) 
and  communication  (i.e.  direct  contact  at  technical  level  between  DG  REFORM  and 
other Commission departments).  

The efficiency of the governance system, the SRSP cycle and cooperation with the other 
Commission departments can be reflected in the the growing number and increasingly 
good  quality  of  the  technical  support  requests  and  in  the  increasing  number  of 
Commission departments taking part in the procedures for the analysis and selection 
of technical support requests. Figure 20 shows an increase in the number of requests of 
better quality analysed by DG REFORM under SRSP 2018 compared to SRSP 2017 and 
an increased number of Commission departments. The analysis showed that the quality 
of  technical  support  requests67  has  considerably  increased  from  one  SRSP  annual 
cycle to another. 

SRSP 2017 

SRSP 2018 

Budget 

EUR 22.5 million 

EUR 30.5 million 

Requests received 

271 (worth EUR 80 
million) 

444 (worth EUR 150 
million ) 

Directorate-General 
that commented 

15 Directorates-
General 

25 Directorates-
General 

Average score of 
selected requests 

 2.68 

Average score of 
requests not selected 

 1.87 

 2.91 

 2.08 

67 In the analysis and selection phase, each technical support request received several scores, which have 
been processed for the scope of the evaluation. Each request had a set of scores with the letters ‘A’, ‘B’ and 
‘C’.  The  letters  were  converted  to  numbers  (‘3’  for  ‘A’,  ‘2’  for  ‘B’,  ‘1’  for  ‘C’,’0’  where  there  was  no 
score) in order to compute the average of the scores for each project. 

37 

 
 
 
  
                                                           
Requests selected 

159 

146 

Member States 

16 Member States 

24 Member States 

Average size of project  EUR 140,000 

EUR 207,000 

Figure 20: Comparison between SRSP 2017 and SRSP 2018 cycles 

To  conclude,  the  findings  suggest  that  the  governance  of  SRSP  and  the  cooperation 
between DG REFORM and other Commission departments have been efficient. The 
duration  of  the  design  of  the  technical  support,  meaning  both  requests  and  projects, 
seems  to  have  corresponded  to  the  needs  of  the  target  group.  At  the  same  time,  the 
quality  of  the  technical  support  requests  increased  from  one  SRSP  annual  cycle  to 
another.  Furthermore,  the  programme’s  implementation,  including  its  monitoring,  was 
aligned  with  the  needs  of  the  target  groups.  The  analysis  and  the  stakeholder  feedback 
highlighted  that  the  Commission’s  current  internal  IT  system  is  largely  adequate  for 
monitoring.  However,  there  is  potential  for  improvement  to  make  the  system  more 
complete.  Common  standards  for  national  monitoring  practices  could,  for  example,  be 
considered  in  the  future.  Furthermore,  that  the  governance  system  and  cooperation 
between  DG  REFORM  and  the  other  Commission  departments  were  efficient  was 
reflected  in  the  increased  quality  of  the  technical  support  requests  and  in  the  positive 
perception  of  the  Commission  departments  interviewed.  Overall,  the  governance  of  the 
SRSP  has  been  continuously  developing,  duly  reflecting  the  changing  needs  of  the 
stakeholders involved and the learning curve that goes with the creation and management 
of a new programme. 

5.3.2.  Cost-effectiveness of the support  

The cost-effectiveness of the SRSP projects was assessed by verifying whether the level 
of  achievement  of  the  expected  results  was  commensurate  with  the  budget  already 
allocated. Cost-effectiveness analyses usually look at the relationship between the costs 
of an action and the benefits obtained. While costs are in general more easily measurable 
in monetary terms, the benefits tend to emerge over a longer timeframe and are generally 
harder to measure objectively. Considering that the object of the mid-term evaluation was 
the implementation of a newly developed programme for the first 2 years, and given that 
only the results and not the benefits could be monitored at this point in time, the analysis 
considered project results as a proxy to benefits.  

The  mid-term  evaluation  assessed  the  correlation  between  financial  resources  allocated 
and the project status. From the data available in the Commission’s internal IT system, it 
emerged  that  on  average,  all  projects  were  on  track,  as  several  steps  had  already  been 
taken towards achieving the expected results. Figure 21 shows that the average score for 
the degree of achievement of the expected results for the completed projects was 3.8 out 

38 

 
 
 
of 468. The projects that have not yet achieved their expected results were mainly ongoing 
projects (110 out of 121). These ongoing projects had a high likelihood that they would 
still  achieve  their  expected  results  by  the  end  of  their  expected  implementation  period 
(score 2.8 out of 3)69.  

Funding source / 
Project status 

Degree of 
achievement of the 
expected results 
(score 1 to 4) 

Number of projects 
that have already 
achieved the 
expected results  
( score = 4) 

Number of projects 
that have not yet 
achieved the 
expected results 
(score =1, 2 or 3) 

Total number of 
project subject to 
this analysis  

Completed 

Ongoing 

3.8 

2.7 

38 

8 

11 

110 

49 

118 

Figure 21: Degree of achievement of the expected results of evaluated projects under SRSP 2017 and 
SRSP 2018 (source: Commission’s internal IT system and targeted interviews, EY 2020) 

When  comparing  the  status  of  implementation  and  achievement  of  results  to  the  cost 
associated to the projects, it emerged that the financial resources of those projects having 
already  achieved  their  results  (30%)  amounted  to  EUR 7.4  million,  i.e.  around  16%  of 
the total budget allocated to the entire sample of projects evaluated. The cost associated 
to  the  projects  that  had  not  yet  achieved  the  expected  results  was  EUR 39.5  million, 
representing  around  84%  of  the  budget  allocated  to  the  entire  sample  of  projects 
evaluated.  

All  the  results  have  been  achieved  for  almost  30%  of  the  evaluated  projects  that 
correspond  to  16%  of  the  approved  financial  resources,  most  of  which  are  linked  to 
completed projects. At the same time, for the remaining 70% of the projects, there is  a 
high  likelihood  that  the  results  will  be  achieved,  most  of  which  are  linked  to  ongoing 
projects.  Therefore,  the  achieved  results  are  considered  commensurate  to  the 
resources allocated.  

In  conclusion,  the  SRSP  actions  can  be  considered  cost-effective  so  far,  while 
acknowledging the limited scope of the analysis. The achieved results are commensurate 
with the budget used. All the results have been achieved for almost 30% of the evaluated 
projects, which correspond to 16% of the approved financial resources, most of which are 

68

 The following score was applied to the projects: 1: No steps have been taken towards the results; 2: A 
few steps have been taken towards the results; 3: Several steps have been taken towards the results; 4: The 
results were achieved. In total, 167 out of the total sample of 246 projects (SRSP 2017 and SRSP 2018) 
were subject to this analysis. Projects for which information was not available on both approved financial 
resources and progress achieved with respect to the expected results were excluded.  
69 The following score was applied to the projects: 1: The results will certainly not be achieved; 2: There is 
a  low  likelihood  that  the  results  will  be  achieved;  3:  There  is  a  high  likelihood  that  the  results  will  be 
achieved.  The  score  for  the  likelihood  that  completed  projects  which  did  not  yet  achieve  their  expected 
results (11 out of 49) would still do so was 2.6 out of 3.  

39 

 
 
 
 
 
                                                           
linked  to  completed  projects.  At  the  same  time,  for  the  remaining  70%  of  the  projects 
(i.e.  84%  of  the  amounts),  there  is  a  high  likelihood  that  the  results  will  be  achieved, 
most of which are linked to ongoing projects.  

5.3.3.  Time-efficiency of the budget execution 

The time efficiency of the budget execution refers to the capacity to contract the financial 
resource allocated (i.e. absorption capacity70) and the capacity to make payments on time 
for the contracts/agreements  that are  signed. Given that information  on payments  made 
under the SRSP was not available at the time of the evaluation, the analysis focused on 
the  alignment  of  the  absorption  capacity  to  the  status  of  the  projects,  as  well  as  on  the 
status of the means of delivery and the existence of possible delays. 

The  analysis  of  the  absorption  capacity  determined  what  percentage  of  the  approved 
financial  resources  for  267  projects  under  SRSP  2017  and  SRSP  2018  had  been 
contracted. It emerged that 71% of the overall approved budget for the evaluated projects 
had  already  been  contracted,  i.e.  EUR 37  million  out  of  EUR 47  million.  The  findings 
suggest  that  the  absorption  rate  is  aligned with  the  two-year  duration  of  the  SRSP 
project cycle, as well as with the status of the contracting procedure under each funding 
source.  Procedures  under  SRSP  2017  are  at  a  more  advanced  stage  of  implementation 
(almost  34%  have  been  completed),  hence  the  higher  absorption  rate  (95.8%).  At  the 
same time, most of the contracting procedures under SRSP 2018 are ongoing (more than 
90%), which is correlated with a lower contracting rate, i.e. 67% (Figure 22). 

Funding source 
/ 
Project status 

SRSP 2017 

# of 
contracting  
procedures  
evaluated 
146 
(100%) 

Approved  
amount 
EUR (in 
million) 

20 

Completed 

49 (33.6%) 

5.3 

Ongoing 

97 (66.4%) 

14.8 

SRSP 2018 

158 

27 

Completed 

15 (9.5%) 

1.2 

Ongoing 

143 
(80.5%) 

25.9 

Contracted  
amount 
EUR ((in 
million) 

% of approved 
amount already 
contracted  
amount 

19.2 

5.1 

14.1 

17.9 

1 

17 

95.8% 

96.5% 

95.6% 

66.5% 

84.1% 

65.7% 

70 Ratio between the financial resources contracted and the financial resources approved. 

40 

 
 
 
 
 
                                                           
Total 

303 

47 

37.1 

81.1% 

Figure 22: Financial resources approved and contracted by funding source (Commission’s internal IT 
system and targeted interviews, EY 2020) 

Although  the  absorption  rate  is  aligned  with  the  SRSP  cycle,  there  are  signs  that  some 
delays might occur71, particularly  as the share of  means  of delivery  with  risk  of delays 
increased from 9% under SRSP 2017 to 17% under SRSP 2018. The distribution of the 
number  of  means  of  delivery  with  risks  of  delays  is  presented  in  Figure  23.  Of  these, 
indirect  management,  specific  contracts  under  framework  contracts  and  grant  direct 
awards are the means of delivery with the highest shares of risks of delays. The rise in 
the share of procedures with risks of delays could be a symptom for the lower contracting 
rate registered under SRSP 2018 compared to SRSP 2017. 

Figure 23: Means of delivery with the highest shares of risks of delays (source: Commission’s internal 
IT system and targeted interviews, EY 2020).  

To conclude, although some minor risks of delays  were noted, the  financial resources 
were  contracted  in  a  time-efficient  manner,  as  shown  by  the  alignment  of  the 
absorption rate with the status of the projects.  

71 The population analysed comprises 386 means of delivery in the scope of this evaluation for SRSP 2017, 
SRSP 2018 and Article 11 (2018). 

41 

 
 
 
 
 
 
 
                                                           
5.4. Coherence 

How do the various components of the SRSP operate together to achieve its objectives 
and how does the SRSP interact with other interventions that have similar objectives?  

In  line  with  the  Better  Regulation  guidelines,  the  assessment  of  the  programme’s 
coherence is divided into two parts: internal and external coherence. Internal coherence 
covers the design of an EU intervention. It looks at how the different components of the 
intervention interact, and whether they are logically linked to achieve the objectives set. 
The external coherence is part of an evaluation that examines how an EU intervention 
interacts  (contradicts,  overlaps,  complements)  with  other  programmes,  instruments, 
policies, and actions. 

Specifically, the SRSP evaluation comprised the following sub-analyses: 

  an  explanation  of  the  rationale  of  the  SRSP’s  logic  of  intervention  (internal 

coherence); 

  an  analysis  of  how  the  SRSP  is  integrated  in  the  EU’s  economic  governance 

process (external coherence); 

  an analysis of the extent to which the SRSP interacts efficiently with other EU 
interventions  and  Commission  Directorates-General  (DGs),  particularly  in 
preventing double funding (external coherence); 

  an  analysis  of  the  extent  to  which  beneficiary  authorities  are  aware  of  other 
programmes  and  policies  with  similar  objectives  as  the  SRSP  (external 
coherence).  

5.4.1.  Rationale of the logic of intervention 

The  rationale  of  the  SRSP  intervention  is  visualised  in  the  programme’s  logic  of 
intervention (Figure 1). The causal relations between the different components of the 
programme  can  be  clearly  identified.  All  the  projects  under  the  evaluation  cover  all 
types  of  eligible  actions.  These  eligible  actions  delivered  the  expected  outputs  or  are 
expected  to  still  do  so.  Subsequently,  the  outputs  are  expected  to  deliver  the  results, 
which, in turn, are expected to trigger the defined impacts. For example, the experts in a 
specific policy area (eligible action) are expected to deliver a reform strategy (expected 
output).  The  strategy  is  then  expected  to  be  adopted  by  the  relevant  Member  State 
(expected 
the  correct  adoption  and 
implementation  of  the  strategy,  in  turn,  is  expected  to  trigger  enhanced  sustainable 

result)72.  Subject 

to  contextual 

factors, 

72 Subject to assumptions and contextual factors.  

42 

 
 
 
 
 
                                                           
 
growth,  job  creation  and  social  inclusion,  or  greater  cohesion,  competitiveness, 
productivity and investment (expected impact).  

At the level of individual projects, the expected activities, outputs and results/outcomes 
are defined in  the  description of the  action or the  terms of  reference, depending on the 
means  of  delivery.  These  are  drafted  following  extensive  consultation  with  the 
beneficiary authority about the needs and expected results.  

5.4.2.  EU economic governance process 

Naturally, the SRSP exists in the wider structures of EU policies and interventions. DG 
REFORM is also closely involved in the European Semester process, where it interacts 
with  other  Commission  departments.  The  European  Semester  is  a  framework  for  the 
coordination of economic policies across the EU. Each year, the Commission carries out 
a  detailed  analysis  of  each  Member  State’s  plans  for  budget,  macroeconomic  and 
structural 
It  also  provides  EU  governments  with  country-specific 
recommendations for the following 12-18 months.  

reforms. 

Figure 24: Interaction between the annual cycles of the SRSP and the European Semester73 

73 NRP = national reform programme; CR = country report; ES = European Semester 

43 

 
 
 
 
 
 
 
                                                           
The projects under the SRSP are supporting the European Semester. Both the SRSP 
and the European Semester are organised in annual cycles, which feed into one another 
(Figure  24).  As  visualised  in  the  figure  above,  European  Semester  country  reports 
(including the CSRs) and the national reform programmes in a given year can serve as a 
guiding document for defining Member States’ requests for technical support under the 
SRSP in the following year.  

The  DG  REFORM  country  coordinators  participate  in  the  European  Semester 
country team  meetings  to  ensure  that the knowledge and experiences  gained from  the 
SRSP  feed  into  the  European  Semester.  These  meetings  have  replaced  the  part  of  the 
coordination  mechanism  involving  the  technical  support  country  teams74.  The  DG 
REFORM  country  coordinators inform  the European Semester country teams  about  the 
requests  submitted  and  about  the  state  of  implementation  of  the  projects  linked  to  the 
European  Semester.  In  addition,  SRSP  projects  can  provide  the  European  Semester 
country teams with information from the ground on the state of play of certain reforms in 
Member  States  and  reasons  behind  obstacles  in  specific  policy  areas.  This  input 
optimises  the  quality  of  the  country  analysis.  Moreover,  the  progress  on  the 
implementation of SRSP projects is included in the country report of each Member State.  

The  majority  of  the  projects  selected  under  SRSP  2017  (51%)  and  SRSP  2018 
(55%) were related to the implementation of reforms in the context of the economic 
governance process, in particular to CSRs75. Of the projects evaluated that were linked 
to  European  Semester  recommendations,  21.6%  have  already  achieved  their  expected 
result76. Of the remaining projects that did not achieve their expected results, 82.8% have 
a high likelihood of doing so77. The fact that a significant share of the SRSP projects are 
related  to  the  implementation  of  CSRs,  as  do  the  aligned  governance  structures, 
demonstrates that the SRSP contributes to the economic governance process. A potential 
for  further  improvement  and  even  stronger  synergies  to  optimise  the  integration  in  the 
economic governance process can be explored. 

5.4.3.  Interaction  between  DG  REFORM  and  other  Commission  departments 

and instruments 

As  stated  above,  the  European  Semester  is  a  process  that  allows  for  a  comprehensive 
approach  towards  structural  reforms  and  the  coordination  of  different  actions  and 
policies, in which DG REFORM plays an integral role. DG REFORM has put in place a 

74 See the following section about the horizontal part of the coordination mechanism. 
75 SRSP Annual Monitoring Report 2017; SRSP Annual Monitoring Report 2018. 
76 25 out of 116 projects, based on an ad hoc scoring system.  
77 48 out of 58 projects, based on an ad hoc scoring system.  

44 

 
 
 
 
                                                           
coordination  mechanism  in  order  to  ensure  that  the  technical  support  provided  is 
aligned with the activities and policies of other Commission departments and to prevent 
any financing overlap.  

DG REFORM has the mandate to steer and coordinate the technical support provided by 
the  Commission  to  the  Member  States.  The  coordination  mechanism  has  the  following 
horizontal structure: 

  A  ‘High-Level  Steering  Committee’,  which  meets  at  least  once  a  year  (or  per 
SRSP  funding  round)  at  senior  management  level,  and  makes  recommendations 
and provides guidance on all aspects of the technical support provided to Member 
States,  including  funding  priorities.  The  High-Level  Steering  Committee  also 
endorses the list of requests proposed for funding under a funding round; 

  A  ‘Technical  Support  Working  Group’,  which  meets  quarterly  to  exchange 
information on the provision of technical support, optimise cooperation  between 
Commission departments, and prepare the process of consultation on requests for 
support received under the SRSP. 

As  stated  above,  the  Commission’s  policy  departments  are  consulted  for  the 
assessment of the technical support requests. Given that DG REFORM is itself not a 
policy  department,  the  appropriate  design  and  implementation  of  projects  requires 
interaction with departments that are responsible for defining the Commission’s policies 
and  that  potentially  fund  actions  with  similar  objectives.  The  consultation  with  policy 
departments  allows  these  departments  to  inform  DG  REFORM  of  links  between  the 
request and their policy priorities. Moreover, this gives these departments the opportunity 
to  signal  if  a  request  overlaps  with  projects  funded  under  their  programmes  or 
instruments,  potentially  leading  to  double  funding.  Lastly,  departments  can  identify 
synergies between the requests for technical support and the actions or projects that they 
themselves fund78. Hence, the consultation between departments allows the Commission 
as an organisation to timely identify synergies and overlaps. 

The  findings  from  scoping  interviews  and  targeted  interviews  with  departments 
confirm  that  the  complementarities  between  the  SRSP  and  other  programmes  are 
ensured  to  a  large  extent.  The  interviews  point  out  that  this  is  mainly  due  to  DG 
REFORM’s  efforts  and  to  the  high  level  of  collaboration  and  communication  between 
the Commission’s different departments. These findings are in line with the outcomes of 
the  2018  Annual  Activity  Report,  which  show  that  all  Commission  departments  are 
satisfied  with  the  efficiency  of  the  coordination  mechanism79.  The  interviews  with 
Commission departments also confirm DG REFORM’s efforts to avoid funding overlaps 

78 Template for consultation with Commission departments.  
79 100% satisfaction (15/15) European Commission, ‘Annual Activity Report 2018 – SRSS’.  

45 

 
 
 
                                                           
through the checking system mentioned above. However, respondents also indicated that 
approving the SRSP in the middle of the programming period creates challenges for the 
other EU programmes with similar objectives.  

SRSP  support  is  directly  complementary  to  many  other  Commission  support 
programmes,  most  notably  the  European  Structural  and  Investment  Funds  (ESIF)  and 
Fiscalis  2020.  To  ensure  close  coherence  and  complementarity  between  the  technical 
support and assistance provided under the ESIF funds and that provided under the SRSP, 
different terminology is used (See box below).  

Box: Technical assistance versus technical support  

Technical  support  is  provided  to  Member  States  under  the  Structural  Reform  Support 
Programme to help Member States implement structural reforms.  

Technical  assistance  is  funded  from  the  European  Structural  and  Investment  Funds 
(ESIF) to help Member States implement ESIF-funded programmes and projects.  

The  scope  of  ESIF  techical  assistance  to  policy  reforms  is  geared  to  the  policy  areas 
falling  within  the  remit  of  ESIF.  For  instance,  in  a  number  of  Member  States,  the 
European  Social  Fund  (ESF)  supports  in-depth  reforms  or  elements  of  reforms  in 
employment, education and social policy areas, as well as public administration reforms. 
These reforms are aligned, similarly as in the case of SRSP support, to country-specific 
challenges identified under the European Semester. 

The SRSP is active in a vast array of policy areas also going beyond those covered by the 
ESIF, and the SRSP technical support can take place in any stage or phase of the reform 
process, also using quite pervasive tools of assistance (such as continuous support on the 
ground). There is anyhow a range of various EU instruments is at the disposal of Member 
States to draw upon. 

Until 2019 (but particularly before the SRSP’s adoption in 2017), technical assistance to 
tax  administrations  was  also  funded  through  the  Fiscalis  2020  programme.  The 
programme aims to ensure that the taxation systems in the internal market work better by 
promoting  closer  cooperation  between  participating  countries,  their  tax  authorities  and 
their officials. 

Some overlaps with the SRSP are possible in the areas of public administration, growth 
(ESIF), and labour markets (ESF). Technical assistance under the structural funds is not, 
in principle, provided for support in the area of financial services and tax administration / 
public financial management.  

Technical support under the SRSP helps Member States to design projects to implement 
structural  reforms  linked  to  the  EU  governance  processes  and  priorities,  including 

46 

 
 
 
national priorities in line with EU priorities; this technical support can be provided in any 
stage  or  phase  of  the  reform  process,  using  quite  pervasive  tools  of  assistance  as  well 
(such as continuous support on the ground). This is why the scope of technical support is 
much broader and deeper than the scope of ESIF technical assistance. 

At  the  time  of  writing  this  staff  working  document,  there  were  a  number  of  checks  in 
place  to  ensure  that  synergies  and  funding  overlaps  were  being  timely  identified  at 
Member  State  level.  By  submitting  their  technical  support  requests  under  the  SRSP, 
Member States are obliged to inform the Commission immediately of other related 
ongoing actions financed out of the EU budget80. Subsequently, if a request is selected 
for  funding,  the  relevant  Member  State  is  required  to  confirm  to  the  Commission  that 
there  is  no  overlap  between  the  selected  request  under  the  SRSP  and  concrete  actions 
funded under other EU instruments, and that there is no double funding. The coordinating 
authority  of  a  Member  State  is  also  obliged  to  send  a  signed  letter  to  the  Commission, 
confirming  the  absence  of  double  funding  for  all  approved  technical  support  requests 
under the SRSP of that specific annual cycle, after having consulted the relevant national 
managing authorities of ESIF funds. The optimisation of synergies with other actions and 
projects and the declaration that there is no double funding and no duplication of efforts 
are  integral  parts  of  the  guiding  principles  of  the  cooperation  and  support  plans.  By 
signing the plan, the Member State commits to comply with these principles.  

At the time of the mid-term evaluation, the verification systems mentioned above were 
gradually being put in place. The case studies indicate that at the time of evaluation, the 
verification systems for double funding were relatively informal. Given this context, the 
general perception of the stakeholders consulted is that the process could be improved by 
further formalising the procedures for checking for double funding before the submission 
of  requests  and  after  their  selection.  The  evaluation  also  suggests  that  Member  States 
would  need  to  increase  their  capacity  in  order  to  perform  more  in-depth  checks.  As 
described above, many improvements have already been made since then.  

5.4.4.  Awareness of other policies and programmes 

Lastly, the evaluation assessed the extent to which the beneficiary authorities were aware 
of  other  policies  and  programmes  whose  objectives  were  similar  to  the  SRSP’s.  As 
indicated  in  Figure  25,  25.2%  of  the  responding  beneficiary  authorities  stated  that 
they  were  indeed  aware  of  these  programmes  and  policies81.  The  EU  programme 
similar  to  the  SRSP  that  is  most  commonly  known  among  beneficiary  authorities  is, 

80  Disclaimer  in  the  SRSP  request  template;  see  Article  12  of  the  Regulation  (EU)  2017/825  (SRSP 
Regulation).  
81 27 out of 107 beneficiary authority respondents.  

47 

 
 
 
 
                                                           
according  to  the  evidence  from  the  targeted  online  consultation,  the  European  Social 
Fund82.  Two  other  EU  programmes  identified  in  the  public  consultation  were  the 
European Regional Development Fund and the Cohesion fund.   

One  of  the  reasons  for  requesting  support  under  the  SRSP  could  be  the  relatively  low 
administrative burden, as DG REFORM  carries  out  the contractual procedures83. The 
findings suggest, however, that beneficiary authorities sometimes lack an overview of all 
funding  alternatives.  From  the  targeted  online  consultation  and  case  studies  it  also 
emerged that for some respondents the SRSP was appreciated because the entirety of the 
support  is  covered  by  the  SRSP  and  that  there  is  no  need for  co-financing  at  national 
level84. 

Figure 25: 'Are you aware of other policies or programmes implemented with European/national funds 
that have similar objectives as the SRSP? (source: targeted online consultation of beneficiary 
authorities, EY 2020). 

In conclusion, the SRSP can so far be considered internally and externally coherent to a 
high degree, taking into account the limited scope of the mid-term evaluation. Firstly, the 
different  components  of  the  logic  of  intervention  have  clear  causal  links.  Furthermore, 
the  SRSP  is  considered  to  be  supportive  of  the  EU  economic  governance  process.  The 
SRSP  and  the  European  Semester  are  adequately  feeding  into  each  other,  providing  in 
both  directions  input  to  enhance  the  coordinated  approach  towards  achieving  the 
objectives.  The  fact  that  a  significant  share  of  the  SRSP  projects  are  related  to  the 
implementation  of  CSRs  demonstrates  that  the  SRSP  contributes  to  the  economic 
governance  process.  In  addition,  the  evaluation  evidence  shows  that  complementarities 
between  the  SRSP  and  other  programmes  are  ensured  to  a  large  extent.  DG  REFORM 
has continued to optimise its coordination mechanism, which allows for the coordination 

82  10  out  of  17  beneficiary  authority  respondents  mentioned  the  European  Social  Fund.  2  mentioned 
Fiscalis. 2 mentioned Erasmus+. 3 mentioned other EU programmes.  
83 8 out of 22 respondents.  
84 This was indicated by 5 out of 22 beneficiary authorities (23%) and 1  out of 5 respondents in the case 

study of Greece.  

48 

 
 
 
 
 
                                                           
of technical  support with other Commission  departments. Synergies and  the absence of 
double  funding  are  best  ensured 
through  consultation  with  other  Commission 
departments  during  the  assessment  of  the  requests  submitted  under  the  SRSP,  and 
through the declaration of absence of double funding for the selected request for support. 
At the level  of the Member State, double funding checks are carried out in  a relatively 
informal  manner.  These  procedures  have  been  further  developed  and  formalised  in 
subsequent  years,  in  line  with  the  recommendations  of  the  SRSP  mid-term  evaluation. 
Lastly,  the  findings  from  the  targeted  online  consultation  suggest  that  beneficiary 
authorities are sometimes aware of other programmes and policies with objectives similar 
to the SRSP’s but lack an all-encompassing overview of these funding alternatives. 

49 

 
 
 
 
5.5. EU added value 

What is the additional value resulting from the intervention at EU level, compared to what 
could be achieved by Member States at national and/or regional levels?  

In accordance with the principle of subsidiarity in areas of non-exclusive competence, the 
EU should only intervene when the objectives can be better achieved at EU level than at 
the level  of the Member State.  In this  light,  the evaluation  assessed  what  resulted from 
the  SRSP-funded  activities  in  value  that  was  above  what  could  reasonably  have  been 
expected from Member States acting at national or regional levels.  

Given that the programme was only set up recently, there are limitations as to how much 
EU added value can be measured in terms of impact. Taking this into consideration, the 
overall assessment of the programme’s EU added value was composed of the following 
two sub-analyses: 

  an explanatory assessment of why SRSP support is managed at EU level and to 

what extent this support is requested by Member States; 

  an  examination  of  good  practices  exchange,  cooperation  among  institutions 
from different Member States and cross-border technical support facilitated 
by the programme. 

5.5.1.  Rationale of the SRSP and Member States’ demand 

All Member States should benefit from a resilient and flexible European economy. 
A  strong  European  economy  relies  on  Member  State  economies  that  function  well  and 
are  adaptable.  However,  Member  States  sometimes  lack  the  required  expertise  or 
administrative  and  institutional  capacity  to  carry  out  reforms  that  the  European  Union 
recommends  to  them85.  The  same  can  be  observed  for  the  application  of  EU  law.  This 
inadequacy to reform could jeopardise sustainable growth across the EU. By providing 
EU-level  technical  support  for  reforms,  it  is  possible  to  tackle  knowledge  and 
capacity constraints in Member States. Highly needed reforms could otherwise remain 
unaddressed  at  national  level.  In  other  words,  in  order  to  achieve  the  objective  of 
economic growth shared by all Member States, technical support provided at Union level 
is  needed  to  help  those  Member  States  and  public  institutions  that  cannot  address  the 
reform challenges with their own national means.  

85 ‘Reform design and implementation in Member States, as a response to the CSRs has not yet reached the 
desired level and outcome’ (SRSP ex ante evaluation, p.5).  

50 

 
 
 
 
 
 
 
 
                                                           
 
 
 
In  addition,  the  relatively  low  bureaucratic  and  administrative  burden  of  the  SRSP  for 
beneficiary authorities, together with the absence of co-financing, contributes to Member 
States making use of the available support. The evidence from the evaluation shows that 
this  is  particularly  relevant  for  beneficiary  authorities  coping  with  limited  financial 
resources or administrative capacity86. Often, these institutions are the ones most in need 
of reforms.  

The  evidence  on  the  submitted  requests  suggests  a  high  demand  for  technical 
support  managed  at  EU  level.  As  for  the  period  evaluated,  every  year,  an  increasing 
number of Member States request technical support under the SRSP87. Moreover, 80% of 
the  Member  States  benefiting  from  support  under  SRSP  2017  and  SRSP  2018  had 
increased  their  number  of  submitted  requests  from  one  year  to  the  next88.  In  total,  25 
Member  States  received  technical  support  under  the  Preparatory  Action  (2016),  SRSP 
2017 and SRSP 2018.  

5.5.2.  Exchange of good practices and cross-border cooperation 

The  SRSP’s  EU  added  value  also  lies  in  its  ability  to  facilitate  the  exchange  of  good 
practices  among different  Member States.  The evidence  collected  shows that  52.5% of 
the projects evaluated made use of good practices and lessons learned89. As outlined 
in  Figure  26,  the  main  types  of  outputs  that  include  good  practice  sharing  are:  (1) 
recommendations,  (2)  analyses  and  reports  and  (3)  workshops,  training  and  training 
materials90.  

86 36% of the beneficiary authorities (8 out of 22) mentioned in the interviews the low level of bureaucracy 
and the speedy technical support. 34% of the beneficiary authorities (9 out of 26) mentioned the simplified 
contracting  and  implementing  procedures.  23%  (5  out  of  22)  mentioned  no  need  for  co-financing  as  a 
reason for requesting support under SRSP instead of other policies or programmes.  
87  As  part  of  the  Preparatory  Action  (2016),  six  Member  States  received  support:  Bulgaria,  Czechia, 
Croatia,  Romania,  Slovenia  and  Slovakia.  In  the  subsequent  year  (SRSP  2017),  10  new  Member  States 
submitted  requests  for  the  first  time:  Cyprus,  Estonia,  Greece,  France,  Italy,  Lithuania,  Latvia,  Malta, 
Poland and Portugal. In the year after that (SRSP 2018), 9 new Member States requested support: Austria, 
Belgium, Germany, Spain, Finland, Hungary, Ireland, Luxembourg and Sweden. 
88 13 out of 16 Member States.  
89 159 out of 303 projects; 181 out of 409 means of delivery (44.3%).  
90 Based on the assessed means of delivery that includes good practice sharing.  

51 

 
 
 
 
                                                           
Recommendations

Analysis, report

Workshops (including material), training,
training material

Action plan, Roadmap

Guidelines, Handbooks

101 

90 

69 

59 

57 

Study visit reports

34 

Communication (including material) and
other events

Legislative proposal

8 

8 

Terms of reference

3 

0

20

40

60

80

100

120

Figure 26: Main types of output developed based on activities that include sharing good practices 
(source: Commission’s internal IT system and targeted interviews, EY 2020). 

Furthermore, from the evaluated means of delivery that used good practices and lessons 
learned,  28.2%  was  provided  by  a  public  sector  entity,  such  as  government  bodies  or 
public  servant  experts91.  Naturally,  support  provided  from  one  public  entity  to  another 
enables the sharing of good practices among EU countries. Evidence from targeted online 
consultation  shows  that  most  beneficiary  authorities  (89%)  perceive  an  increase  of 
knowledge  sharing  between  their  institution  and  institutions  from  other  Member 
States  (Figure  27).  Likewise,  91%  of  the  technical  support  providers  perceived  an 
increase  in  knowledge  exchange92.  The  respondents  of  the  public  consultation  were 
positive, too, about the programme’s EU added value in terms of sharing good practices 
and promoting cooperation.93  

91 51 out of 181.  
92 Targeted online consultation: 31 out of 34.  
93 6 out of 9 respondents indicated that the SRSP adds value to a high extent or to the fullest extent in terms 
of sharing good practices; 7 out of 9 respondents indicated that the SRSP adds value to some extent or 
to a high extent in terms of promoting cooperation.  

52 

 
 
 
 
 
 
                                                           
1 
3% 

2 
5% 

1 
3% 

13 
34% 

21 
55% 

Strongly agree

Agree

Disagree

Strongly disagree

I don't know

Figure 27: Beneficiary authorities’ degree of (dis)agreement with the statement: 'The technical support 
increased the sharing of knowledge and good practices between my institution and institutions from 
other countries' (source: targeted online consultation, EY 2020). 

Technical  support  under  the  SRSP  can  also  address  reform  challenges,  which  are 
common to several Member States. This can take two forms. In the first type, several 
Member  States  may  make  similar  requests  for  support  to  address  a  common  reform 
challenge, if they realise that their reform should be best tackled across national borders. 
All  the  interested  Member  States  still  need  to  submit  their  SRSP  requests  individually; 
one Member State cannot apply for support on behalf of another Member State. Second, 
if  different  Member  States  submit  requests  to  tackle  a  similar  issue,  even  though  the 
relevant  countries  might  be  geographically  far  apart,  the  Commission  can  identify 
synergies  between  these  individual  support  requests.  In  cases  deemed  feasible,  the 
Commission can decide to implement requests from different Member States which are 
related  to  one  another,  through  one  project,  with  a  view  to  achieve  efficiency  gains  in 
terms of contracting. Instead of an individual technical support project for each request, 
one  project  would  cover  the  requests  of  several  Member  States.  Each  Member  State 
would in any case still receive individualised support, applicable to their local situation. 

The findings from the evaluation demonstrate that the extent to which individual SRSP 
projects tackle cross-border challenges in multiple countries is low94. Only one out of 
all  the  projects  evaluated  (0.3%)  combine  independently  requests  submitted  from 
different Member States into one multi-country project95. The reason for the limited joint 

94  None  of  the  22  interviewed  beneficiary  authorities  mentioned  any  past  or  future  need  for  technical 
support for cross-border cooperation.  
95  1  out  of  the  303  projects  evaluated:  ‘Strengthening  capital  markets  in  Estonia,  Latvia,  Lithuania  and 
Poland’.  The  technical  support  provided  for  the  creation  and  strengthening  of  a  legal  and  regulatory 
framework  with  respect  to  structured  products  and  hedging  instruments  aimed  to  contribute  to  the 
development of the pan-Baltic capital market. The expertise  was ensured by the  same provider in all the 
countries.  

53 

 
 
 
 
 
 
                                                           
technical support actions in different Member States is that the requests for support are 
country-specific,  related  to  national  priorities  in  line  with  EU  goals.  The  needs  of 
Member States often have their own unique characteristics, as challenges are embedded 
in  the  specific  national  context.  The  exchange  of  information  among  beneficiary 
authorities  from  different  Member  States  during  the  design  of  the  requests  is  also  not 
advanced. This could be further enhanced in those cases where it is deemed feasible and 
of added value. The findings suggests that the Commission needs to optimise further the 
identification of synergies between individual requests from different Member States, to 
ensure  that  technical  support  efforts  are  complementary  and  closely  related  where 
relevant and practically achievable. Lastly, the sharing of solutions and knowledge after 
the project’s implementation among Member States seems to be limited. Although there 
are  individual  cases  in  which  knowledge  sharing  took  place  after  the  closure  of  the 
project, this is not yet a common practice96. Nevertheless, these individual cases suggest 
that the programme can provide technical support for cross-border cooperation. A large 
share of the respondents of the public consultation positively assessed the  programme’s 
added value in terms of addressing cross-border or Union-wide challenges97.  

In  conclusion,  the  technical  support  provided  under  the  SRSP  is  intended  to  target 
precisely  those  reform  challenges  that  Member  States  cannot  address  with  their  own 
means  and  for  which  they  seek  expertise  beyond  the  national  level.  The  increasing 
number  of  Member  States  that  requested  support  under  the  programme,  as  well  as  the 
increase in requests submitted, implies a high demand for technical support at EU level. 
Furthermore, many projects include activities that allow for the exchange of knowledge 
and good practices among Member States.The evidence suggests that the Commission is 
not  yet  sufficiently  facilitating  information  exchange  between  beneficiary  authorities  in 
different  Member  States  during  the  design  phase  of  the  project.  Considering  that  the 
programme’s internal processes and procedures have by now taken their final shape, the 
exchange  of  information  among  beneficiary  authorities  should  be  reinforced.  The 
Commission  could  also  build  on  experiences  from  successful  cases  of  cross-border 
cooperation,  including  by  optimising  the  identification  of  synergies  between  individual 
SRSP  requests.  At  the  same  time,  Member  States  could  seek  cooperation  for  requests 
under SRSP that cope with common challenges. 

96  9%  of  the  beneficiary  authorities  interviewed  (2  out  of  22)  and  12%  of  the  coordinating  authorities 
interviewed (1 out of 8) could indicate the existence of cross-border impacts; Individual cases: Slovakia’s 
experience  with  the  implementation  of  the  spending  review  process  prompted  other  countries  to  seek 
information from Slovakia about how the reform was addressed under the SRSP. Similarly, Croatia shared 
its experience of undertaking the functional integration of hospitals with SRSP support at an international 
workshop organised by the World Bank. At the same time, Italy was the first European country to have a 
national aid register, and it shared the instructions on how to draft this register with Greece as well. 
97 5 out of 10 repondents indicated that the SRSP adds value to a high extent in terms of addressing cross-

border or Union-wide challenges.  

54 

 
 
 
                                                           
6. CONCLUSIONS 

The Structural Reform Support programme was adopted in 2017 to help Member States 
in  the  process  of  designing  and  implementing  institutional,  administrative  and  growth-
sustaining reforms, by providing technical support to national authorities. This mid-term 
evaluation  assessed the  SRSP  2017 and 2018  budgetary cycles. The overall goal  of the 
evaluation  was  to  examine  how  the  SRSP  is  performing  against  the  evaluation  criteria 
(relevance, effectiveness, efficiency, coherence and EU added value) and to draw lessons 
that can serve possible future improvements.  

Overall,  the  quality  and  representativeness  of  the  collected  evidence  is  considered 
satisfactory.  The  evaluation  covered  over  300  technical  support  projects  and  included 
different  evidence  collecting  techniques.  The  most  considerable  limitation  to  the 
robustness of the findings was the fact that the programme had only very recently been 
adopted  at  the  time  of  the  evaluation.  Many  projects  were  still  in  their  initial  phase, 
posing constraints on measuring the SRSP’s results, as well as impacts. At the same time, 
some  internal  procedures  and  processes,  and  the  information  system  for  monitoring 
projects,  were  only  gradually  being  implemented.  This  limits  the  extent to  which  fixed 
conclusions  on  the  SRSP’s  success  can  be  drawn  in  this  mid-term  evaluation. 
Nevertheless, the Commission’ staff considers that the findings have an adequate level of 
robustness to provide preliminary conclusions and lessons learned on the SRSP.  

Regarding the relevance of the programme, the findings of the evaluation show that the 
design  of  the  SRSP  is  aligned  with  Member  States’s  actual  needs.  The  evidence 
demonstrates  that  the  Member  States’  needs,  as  identified  in  their  request  for  support, 
have  all  been  addressed  to  a  large  extent  by  the  programme.  Stakeholders  indicate  that 
the  SRSP  is  a  suitable  instrument  for  providing  technical  support  to  Member  States  to 
design  and  implement  growth  enhancing  reforms.  Additionally,  Member  States 
especially appreciated the high degree of flexibility, the simplicity of the contracting and 
implementing procedures managed by the Commission and the absence of co-financing 
requirements.  The  findings  suggest  that  the  SRSP  remains  a  relevant  instrument  for 
Member States. Technical support continues to be needed, given that reforms are linked 
to  long-term and wide needs.  The continuous need for support is  also  connected to  the 
experience and capacity  that the beneficiary  authorities gain with the instrument, as the 
beneficiary authority might identify other needs and potential areas for support under the 
the 
programme,  after  finishing  one  project.  These  outcomes  correspond  with 
Commission’s expectations.  

The analysis of the programme’s effectiveness shows that the projects funded under the 
SRSP are generally well designed. The way in which the Commission takes into account 
the specific needs of Member States, as well as the relevant socio-economic impacts, is 
considered a good and common practice according to the stakeholders. This makes it also 

55 

 
 
 
possible  to  translate  the  needs  into  feasible  actions  with  realistic  timelines;  only  a  few 
projects  were  delayed.  The  fact  that  technical  support  is  generally  provided  in  a 
comprehensive manner, with satisfactory consideration of the situation of the individual 
beneficiary authority, is a good practice and meets the expectations of delivering tailor-
made  technical  support.  Furthermore,  the  findings  demonstrate  that  the  SRSP  is 
progressing towards achieving its objectives, taking into account that many projects were 
still  ongoing  at  the  time  of  the  evaluation.  Most  of  the  technical  support  projects  are 
likely to achieve their results, or have already done so. However, the exchange of  good 
practices in terms of project management among Commission project managers could be 
reinforced in  order to  increase further the number of successful projects.  Evidence also 
shows that changes in the political structure of the beneficiary authority, as well as a lack 
of  collaboration  among  stakeholders,  are  important  obstacles  to  the  delivery  of  the 
projects’  outputs.  These  and  other  negative  factors  should  be  avoided  and  mitigated  to 
the extent possible. As for the SRSP’s impact, it is premature to draw solid conclusions. 
Although  the  stakeholders’  perception  of  the  impact  is  considered  proportionate  to  the 
programme’s level of completion, a comprehensive evaluation of the long-term impacts 
of the supported interventions should be carried out at a later stage, as part of the ex-post 
evaluation.  

As regards the programme’s efficiency, the evaluation findings indicate that governance 
of  the  SRSP  is  efficient.  Overall,  the  programme’s  cycles,  including  the  design  and 
implementation  of  the  technical  support,  are  aligned  with  the  beneficiary  authorities’ 
needs.  The  analysis  also  showed  that  the  current  Commission’s  internal  IT  system  is 
largely  adequate  as  a  monitoring  arrangement,  although  it  still  needed  further 
improvements at the time of evaluation. Additionally, all departments indicated that their 
cooperation  with  DG  REFORM  was  efficient.  Strong  points  highlighted  by  the 
stakeholders interviewed were: the efficiency of the governance system, the SRSP cycle 
and  cooperation  with  the  other  Commission  departments,  the  good  quality  of  the 
technical support requests, as well as the increasing number of Commission departments 
taking  part  in  the  procedures  for  the  analysis  and  selection  of  the  technical  support 
requests.  In  terms  of  cost-effectiveness,  the  analysis  indicates  that  the  achieved  results 
are commensurate with the budget allocated. Given that the SRSP was established only 
very recently and therefore only results and not benefits could be monitored at this point, 
the SRSP-funded actions can be considered, so far, cost-effective. Finally, the evidence 
demonstrates that the programme’s absorption rate is aligned with the two-year duration 
of the SRSP project cycle. Therefore, it can be concluded that the SRSP is time-efficient 
in terms of its budget execution. When it comes to potential improvements, the evidence 
suggest  that  common  standards  as  regards  the  monitoring  practices  at  the  level  of  the 
Member States could be considered in the future. Further attention could be paid to some 
minor  risks  of  delays  in  the  contracting  procedure  that  were  noted  in  the  evaluated 
period.  

56 

 
 
 
The  analysis  of  the  programme’s  coherence  focused  on  both  internal  and  external 
coherence.  The  findings  demonstrate  that  the  different  components  of  the  intervention 
logic  of  the  SRSP  are  logically  linked  to  one  another.  Furthermore,  the  SRSP  is  well 
integrated in the EU economic governance process. The annual cycles of the SRSP and 
the  European  Semester  are  adequately  feeding  into  each  other,  which  enables  a 
comprehensive and  coordinated approach towards  structural  reforms  in  Member States. 
The evidence  also  shows  that  DG REFORM  has put  in  place an  adequate coordination 
mechanism that allows for efficient coordination of technical support. The confirmation 
of the absence of double funding and the identification of potential synergies are carried 
out both by the Commission, through the consultation between DG REFORM and other 
departments,  and  by  the  Member  States,  through  the  support  request  template  and  the 
cooperation and support plans, and the regulatory provision of no double funding. At the 
time  of  the  evaluation,  the  verification  of  double  funding  by  the  Member  State  was 
carried  out  in  a  relatively  informal  manner.  Subsequently,  these  procedures  have  been 
further  developed  and  formalised.  The  findings  from  scoping 
interviews  with 
the  SRSP  and  other 
departments  confirm 
programmes  are  ensured  to  a  large  extent.  At  the  same  time,  the  Commission 
departments  stated  that  the  fact  that  the  SRSP  support  is  not  aligned  to  their 
programming  period  imposed  challenges  on  other  EU  programmes  with  similar 
objectives. Lastly, the findings suggest that Member States would need to increase their 
capacity to perform more in-depth checks for double funding. It could be considered to 
look into this, in order to further optimise the safeguarding of synergies and the absence 
of funding overlaps.  

the  complementarities  between 

that 

Lastly, the evaluation assessed the programme’s EU added value. The technical support 
provided  under  the  SRSP  is  intended  to  target  those  reform  challenges  that  Member 
States cannot address at national level and for which authorities seek support at EU level  
due  to  capacity  or  knowledge  constraints.  Moreover,  the  individual  projects  very  often 
provide  a  platform  for  exchanging  good  practices  and  lessons  learned  among  Member 
States. These are very essential features that the SRSP was expected to deliver in terms 
of EU added value and that have materialised. At the same time, the findings demonstrate 
that  the  extent  to  which  SRSP  actions  address  cross-border  challenges  in  multiple 
countries  remains  low.  The  exchange  of  information  between  beneficiary  authorities  in 
different  Member  States  could  also  be  further  facilitated,  where  relevant.  In  addition, 
Member  States  could  be  invited  to  cooperate  when  they  formulate  requests  for  support 
that are related to shared challenges.  

57 

 
 
 
 
 
 
 
Lessons learned 

Following the mid-term evaluation, the main lessons learned include: 

  The  high  degree  of  flexibility,  the  simple  contracting  and  implementing 
procedures,  and  the  absence  of  co-financing  requirements,  make  the  SRSP  an 
instrument relevant to Member States’ reform needs. 

  A high degree of commitment from all stakeholders, a high level of expertise of 
the  provider  and  favourable  political  circumstances  are  considered  factors  that 
positively influence the delivery of the projects’ outputs. A lack of collaboration 
among  stakeholders  and  changing  or  uncertain  political  circumstances  are 
considered  factors  that  negatively  influence  the  successful  implementation  of 
technical support projects.  

  DG  REFORM  could  provide  further  support  to  the  coordinating  authorities  in 
order  to  understand  their  monitoring  prerogatives  and  help  them  adapt  their 
coordination and monitoring capacity.  

 

Improvements of the Commission internal IT system are needed, in particular on 
the level  of completeness of information  related  to  expected activities, expected 
results, state of play of activities and results. 

  The  good  collaboration  and  the  continuous  support  of  the  DG  REFORM  staff 
with  beneficiary  authorities,  coordinating  authorities,  providers  and  other 
Commission  departments  were  strong  determinants  of  the  quality  of  the 
monitoring system and the efficiency and coherence of the programme.  

  The  alignment  of  the  annual  cycles  of  the  SRSP  and  the  European  Semester 
contributed to a coordinated approach towards support for the implementation of 
structural  reforms.  The  high  level  of  collaboration  and  communication  between 
DG  REFORM  and  other  Commission  departments  were  essential  in  ensuring 
complementarities between the SRSP and other programmes. 

  Sharing good practices among Member States is of significant added value, which 
comes with managing technical support at EU level. DG REFORM could further 
facilitate the exchange of information between beneficiary authorities of different 
countries, where relevant, in order to optimise synergies and solution sharing.  

58 

 
 
 
 
 
 
 
 
 
Annex 1: Procedural information 

1.  LEAD DG, DeCIDE PLANNING/CWP REFERENCES 

Directorate-General for Structural Reform Support.  

Decide planning reference: PLAN/2018/3005. 

2.  ORGANISATION AND TIMING 

The initiation of the mid-term evaluation of the SRSP was approved by the SRSS senior 
management98  and  the  Cabinet  in  May  2018.  This  was  followed  by  the  appointment  of  
the  members  of  the  Inter-service  Steering  Group  (ISSG).  The  evaluation  roadmap  was 
published  on  20  June  2018,  marking  the  evaluation’s  official  starting  date.  The 
procurement procedure for the external evaluation was initiated on 9 July 2018, and the 
contract  with  the  external  contractor  was  subsequently  signed  on  21  November  2018, 
with a kick-off meeting on 6 December 2018. The external evaluation lasted 12 months. 
The public consultation ran on  the Commission’s website  from  March  until June 2019. 
After  the  contractor  submitted  the  final  evaluation  report,  the  Commission  started 
drafting the staff working document, based on the external evaluation and the analysis of 
its own reporting and procedural documents.   

DG  REFORM  chaired  the  ISSG,  which    comprised  representatives  of  the  Secretary 
General,  REGIO,  DIGIT,  CNECT,  EAC,  HOME,  ENV,  GROW,  SANTE,  MOVE, 
AGRI, EMPL, TAXUD, JUST, FISMA, NEAR, ECFIN, ENER, COMP, ESTAT, RTD, 
HR, JRC and OLAF. The ISSG held three meetings.  

3.  EXCEPTIONS TO THE BETTER REGULATION GUIDELINES 

N/A 

4.  CONSULTATION OF THE RSB (IF APPLICABLE) 

N/A 

5.  EVIDENCE, SOURCES AND QUALITY 

An external evaluation report served as a basis for the mid-term evaluation of the SRSP.  

The contractor used a variety of research methods and sources, including: desk research, 
data  extracts  from  the  Commission’s  internal  IT  system,  scoping  interviews,  targeted 
interviews,  targeted  online  consultation  and  case  studies.  In  line  with  the  Better 
Regulation guidelines, the evaluation also included a public consultation exercise.  

98 At that time, DG REFORM was still SRSS.  

59 

 
 
                                                           
The  most  important  constraint  to  the  robustness  of  the  findings  was  the  fact  that  the 
SRSP had only very recently been established at the time of the mid-term evaluation. The 
fact  that  the  mid-term  evaluation  took  place  at  a  very  early  stage  in  the  programme’s 
implementation  limited  the  extent  to  which  fixed  conclusions  on  the  SRSP’s  success 
could be drawn.  

Furthermore, at the time, the Commission’s new internal IT system was not in place for 
SRSP 2017, being implemented only afterwards. The system was evolving significantly 
and  many  questions  /  categories  of  data  were  added  only  at  later  stages  and  not 
necessarily  completed  retroactively.  Therefore,  information  was  to  some  extent 
incomplete.  This  shortcoming  was  overcome  by  triangulating  the  information  based  on 
other  data  sources,  including  the  interviews,  online  consultation  and  case  studies. 
Furthermore,  the  response  rate  to  the  targeted  online  consultation  was  sufficient  and 
ensured a balanced representation of stakeholders.  

Additionally, the breadth and diversity of the programme’s activities posed an important 
methodological  challenge,  in  that  it  would  not  have  been  possible  within  the  available 
resources  and  timeframe  to  cover  all  activities  in  the  detail  needed  to  draw  robust 
conclusions.  Moreover,  much  of  the  programme’s  support  plays  a  contributing  role 
alongside other factors, such as the action of national administrations. This contributing 
role is difficult to assess without in-depth qualitative research.  

Lastly,  given the nature  of a mid-term evaluation,  many  of the projects  evaluated  were 
still  ongoing  at  the  time  of  the  exercise.  Therefore,  results  and  impact  can  only  be 
measured to  a limited extent.  A certain  share of the projects’ expected results  were not 
yet delivered. In those cases, the likelihood of achieving the results was assessed only to 
the extent that information was available. The case studies confirmed that it is too early 
to expect considerable long-term impacts at this stage of the programme.  

Despite  the  limitations  mentioned  above,  the  quality  and  representativeness  of  the 
evidence collected is considered satisfactory. The contractor was able to collect extensive 
and meaningful data that allowed conclusions to be drawn.  

60 

 
Annex 2: Stakeholder consultation 

As  part  of  the  evaluation  performed  by  the  external  contractor99,  four  types  of 
stakeholders were consulted through three data collection methods: 

  DG  REFORM  staff  through  scoping  interviews  and  the  targeted  interviews, 

specifically: 

o  Heads of unit (HoU) / heads of cluster and deputy heads of unit 
o  DG REFORM advisers 
o  Country coordinators 
o  Project managers  

  Coordinating  authorities  through  the  targeted  online  consultation  and  the 

interviews for the case studies; 

  Beneficiary  authorities  through  the  targeted  online  consultation  and  the 

interviews for the case studies; 

  Technical  support  providers  through  the  targeted  online  consultation  and  the 

interviews for the case studies. 

The  consultation  activities  were  conducted  between  January  and  December  2019.  The 
consultation activities are elaborated in the main text. 

Public consultation 

In addition to consulting stakeholders as part of the external evaluation, the Commission 
carried  out  a  public  consultation  on  the  Commission’s  official  website  between  March 
and June 2019 . The goal of the public consultation was to collect information from the 
general  public,  businesses,  trade  associations  and  interest  groups.  The  outcomes  of  the 
public consultation are outlined below. 

Total respondents: 11 (5 public authorities, 4 members of the general public, 1 trade 
union and 1 ‘other’). 

Geographical distribution: 5 from Latvia; 2 from Belgium; 2 from Spain; 1 from 
France; 1 from Italy. 

99 For more information see:  Ernst&Young, 2020 “Mid-term Evaluation of the Structural Reform Support 
ISBN  978-92-76-18436-2; 

(SRSP)  2017-2020”,  Final 

report,  May  2020  – 

Programme 
https://op.europa.eu 

61 

 
 
 
 
                                                           
(1) not at all; (2) to a limited extent; (3) to some extent; (4) to a high extent; and (5) to the fullest 

extent. 

62 

 
 
 
Scale : (1) not at all; (2) to a limited extent; (3) to some extent;  (4) to a high extent and (5) to the 
fullest extent 

63 

 
 
 
 
 
Scale : (1) not at all; (2) to a limited extent; (3) to some extent;  (4) to a high extent and (5) to the 
fullest extent 

Scale : (1) not at all; (2) to a limited extent; (3) to some extent;  (4) to a high extent and (5) to the 
fullest extent 

64 

 
 
 
 
Scale : (1) not at all; (2) to a limited extent; (3) to some extent;  (4) to a high extent and (5) to the 
fullest extent

65 

 
 
 
Annex 3: Evaluation framework100 

Number of 
evaluated projects 
in the sample 

303 

Evaluation 
criteria 

Evaluation question (EQ) 

Details 

Relevance 

Extent to which the SRSP 
projects, as they are designed, are 
expected to meet the identified 
Member State’s needs 

Extent to which Member States 
still need technical support to 
perform structural reforms and the 
extent to which SRSP support 
remains relevant 

EQ.1. 

To what 
extent are the 
SRSP actions 
appropriate for 
supporting 
Member States in 
strengthening their 
administrative and 
institutional 
capacity, 
supporting the 
design of growth-
sustaining 
structural reforms 
and the application 
/ implementation 
of EU law? 

Effectiveness - 
results 

EQ.2.  How 

effectively have 
SRSP actions and 

Extent to which projects have 
been implemented in accordance 
with the planned schedule 

303 

100 Drafted by the external contractor.   

66 

Indicators 

Source of information 

Information available in 
the Commission’s 
internal IT system 
updated in accordance 
with the results of the 
targeted interviews with 
the project managers, the 
results of the targeted 
online consultation 
The results of the 
targeted online 
consultation 

 

Percentage of projects whose 
expected results are related 
to: 

 

 

 

 

strengthening the 
administrative capacity 
strengthening the design 
of growth-sustaining 
structural reforms 
strengthening the 
application / 
implementation of EU 
law 

Perceived degree of support 
still needed 

 

 

 

Percentage of projects whose 
implemented activities are 
aligned with the estimated 

 

Information available in 
the Commission’s 
internal IT system 

 
 
 
                                                           
Evaluation 
criteria 

Evaluation question (EQ) 

Details 

Number of 
evaluated projects 
in the sample 

Indicators 

Source of information 

activities 
translated the 
Member States 
priorities / needs 
(conveyed in their 
requests for 
support) into 
feasible actions / 
activities? 

EQ.3. 

To what 
extent have SRSP 
actions and 
activities 
contributed to the 
design and 
implementation of 
institutional, 
administrative and 
structural reforms 
in the Member 
States? 

Extent to which the SRSP actions 
and activities were feasible in 
terms of content thus ensuring that 
SRSP reaches its objectives 
effectively 

If and how contextual factors had 
any (positive or negative) effect 
on the implementation of actions 
and activities and the achievement 
of deliverables and results 

Extent to which the provided 
technical support and its expected 
results are related to the design 
and implementation of 
institutional, administrative and 
structural reforms in the Member 
States 

The extent to which the expected 
results (of the projects related to 
the design and implementation of 
institutional, administrative and 
structural reforms in the Member 
States) have been achieved / are 
likely to be achieved 

updated in accordance 
with the results of the 
targeted interviews with 
the project managers 
The results of the 
targeted online 
consultation were used 
to confirm the results of 
the computation 
The results from the 
targeted online 
consultation 
Information available in 
the Commission’s 
internal IT system 
updated in accordance 
with the results of the 
targeted interviews with 
the project managers 
The results of the 
targeted online 
consultation were used 
to confirm the results of 
the computation 
The results from the 
targeted online 
consultation 

 

 

 

 

 

 

 

 

 

 

 

start / end dates 
Percentage of projects whose 
expected results have been 
produced 
Percentage of projects whose 
expected results have a high 
likelihood of being achieved 
been achieved 
Perceived impacts of 
contextual factors on the 
projects’ implementation 

Percentage of projects whose 
expected results are related to 
the design and 
implementation of 
institutional, administrative 
and structural reforms in the 
Member States 
Percentage of projects whose 
expected results have been 
produced (related to the 
design and implementation of 
institutional, administrative 
and structural reforms in the 
Member States) 
Percentage of projects whose 
expected results (related to 
the design and 
implementation of 
institutional, administrative 
and structural reforms in the 

226 

Number of projects 
whose expected 
results are related 
to the design and 
implementation of 
institutional, 
administrative and 
structural reforms 
in the Member 
States 

67 

Effectiveness - 
results 

 
Evaluation 
criteria 

Evaluation question (EQ) 

Details 

Effectiveness - 
results 

Effectiveness- 
results 

EQ.4. 

To what 
extent have SRSP 
actions contributed 
to supporting 
national 
authorities in the 
design / 
implementation of 
their reforms 
according to their 
priorities? 

EQ.5. 

To what 
extent SRSP 
actions were 
designed and 
implemented by 
taking into 
account initial 
conditions and 
expected socio-
economic 
impacts? 

Extent to which SRSP projects are 
addressing Member States’ own 
reform priorities 

Extent to which SRSP projects 
addressing Member States’ own 
reform priorities have achieved / 
are likely to achieve their expected 
results 

Extent to which the designed and / 
or implemented SRSP projects 
considered the challenges 
beneficiary authorities were facing 
when requesting technical support 

Extent to which the expected 
socio-economic impacts of the 
support were considered when 
defining the content of the 
technical support requests 

Number of 
evaluated projects 
in the sample 

138 

Number of projects 
within SRSP which 
are addressing 
Member States’ 
own reform 
priorities 

Indicators 

Source of information 

 

 

 

Member States) have a high 
likelihood of being achieved 
Perceived impacts of 
contextual factors on the 
projects’ implementation 
Percentage of projects 
addressing Member States’ 
own reform priorities 
Percentage of projects 
addressing Member States’ 
own reform priorities whose 
expected results have been 
achieved / have a high 
likelihood of being achieved 

 

Information available in 
the Commission’s 
internal IT system 
updated in accordance 
with the results of the 
targeted interviews with 
the project managers, 
Requests for technical 
support 

303 

  Distribution of projects by 

 

 

 

specific objective 
Perception of the 
stakeholders of the extent to 
which the projects were 
designed and implemented by 
taking into account the 
challenges beneficiary 
authorities were facing when 
requesting technical support 
Perception of the 
stakeholders regarding the 
extent to which the projects 
were designed by taking into 
account the expected socio-
economic impacts 

 

 

Information available in 
the Commission’s 
internal IT system 
updated in accordance 
with the results of the 
targeted interviews with 
the project managers 
The results from the 
targeted online 
consultation 
The results from the 
targeted online 
consultation 

68 

 
 
 
Evaluation 
criteria 

Evaluation question (EQ) 

Details 

Number of 
evaluated projects 
in the sample 

Indicators 

Source of information 

Effectiveness - 
results 

Effectiveness - 
results 

EQ.6.  Did the 

intervention pay 
due attention to 
accompanying 
measures and / or 
appropriate 
sequencing of 
reform measures? 

EQ.7. 

To what 
extent have SRSP 
actions contributed 
to supporting the 
national 
authorities in 
enhancing their 
capacity to 
formulate, develop 
and implement 
reform policies 
and strategies? 

Extent to which, within each 
Member State, SRSP has 
supported complementary projects 
on the same topic 

303 

Extent to which, within each 
Member State, SRSP has 
supported successive projects 
addressing different aspects / 
phases of a structural reform 

Extent to which the provided 
technical support and its expected 
results were related to the 
formulation, development and 
implementation of reform policies 
and strategies 

Extent to which SRSP projects 
related to enhancing Member 
States' capacity to formulate, 
develop and implement reform 
policies and strategies have 
achieved / are likely to achieve 
their expected results 

64 

Number of projects 
that are related to 
enhancing Member 
States' capacity to 
formulate, develop 
and implement 
reform policies and 
strategies 

Effectiveness - 
results 

EQ.8. 

To what 
extent have SRSP 
actions contributed 
to supporting the 

Extent to which the technical 
support provided under SRSP and 
its expected results are related to 
the definition of new processes 

110 

Number of projects 
related to the 

69 

 

 

Perceived alignment of the 
projects with the initial 
conditions and expected 
socio-economic impacts of 
the support 
Percentage of complementary 
projects (related to the same 
topic at Member State level) 

 

Percentage of projects whose 
expected results are related to 
enhancing Member States' 
capacity to formulate, 
develop and implement 
reform policies and strategies 
  Degree of achievement of the 

 

 

expected results 
Perceived impact of 
contextual factors on the 
projects’ implementation 

Percentage of projects whose 
expected results are related to 
new processes and 
methodologies 

 

 

 

 

Information available in 
the Commission’s 
internal IT system 
updated in accordance 
with the results of the 
targeted interviews with 
the project managers 

Information available in 
the Commission’s 
internal IT system 
updated in accordance 
with the results of the 
targeted interviews with 
the project managers 
The results of the 
targeted online 
consultation 

Information available in 
the Commission’s 
internal IT system 
updated in accordance 

 
Number of 
evaluated projects 
in the sample 
definition of new 
processes and 
methodologies 
while taking into 
account successful 
experiences from 
other EU Member 
States 

Evaluation 
criteria 

Evaluation question (EQ) 

Details 

efforts of national 
authorities to 
define and 
implement 
appropriate 
processes and 
methodologies by 
taking into 
account good 
practices and 
lessons learned by 
other countries in 
addressing similar 
situations? 

and methodologies 

Extent to which the definition 
(within SRSP) of the new 
processes and methodologies has 
taken into account successful 
experiences from others EU 
Member States 

Extent to which the expected 
results (of the projects related to 
the definition of new processes 
and methodologies, while taking 
into account successful 
experiences from other EU 
Member States) have been 
achieved / have a high likelihood 
of being achieved 

Effectiveness - 
results 

EQ.9. 

To what 
extent have SRSP 
actions contributed 
to assisting the 
national 
authorities in 

Extent to which the technical 
support provided under SRSP and 
its expected results are related to 
the enhancement of the efficiency 
and effectiveness of human-
resource management 

40 

Number of projects 
that are related to 
the enhancement of 
the efficiency and 
effectiveness of 

70 

Indicators 

Source of information 

 

 

 

 

 

Percentage of projects whose 
expected results are related to 
new processes and 
methodologies defined while 
taking into account 
successful experiences from 
other EU Member States 
Percentage of projects whose 
expected results (related to 
new processes and 
methodologies while taking 
into account successful 
experiences from other EU 
Member States) have been 
achieved 
Percentage of projects whose 
expected results have a high 
likelihood of being achieved 
been achieved 
Perceived impact of 
contextual factors on the 
projects ‘implementation 
Percentage of projects whose 
expected results are related to 
the enhancement of the 
efficiency and effectiveness 
of human-resource 
management 

 

 

 

with the results of the 
targeted interviews with 
the project managers 
Information available in 
the Commission’s 
internal IT system 
updated in accordance 
with the results of the 
targeted interviews with 
the project managers  
The results of the 
targeted online 
consultation 

Information available in 
the Commission’s 
internal IT system 
updated in accordance 
with the results of the 
targeted interviews with 

 
Number of 
evaluated projects 
in the sample 
human-resource 
management 

Evaluation 
criteria 

Evaluation question (EQ) 

Details 

enhancing the 
efficiency and 
effectiveness of 
human-resource 
management? 

Extent to which the expected 
results (of the projects related to 
the enhancement of the efficiency 
and effectiveness of human-
resource management) have been 
achieved / have a high likelihood 
of being achieved 

Extent to which technical support 
provided under SRSP is 
contributing to implementation of 
relevant reforms 

Effectiveness - 
impacts 

EQ.10.  To what 
extent has the 
SRSP provided 
support to 
Member States in 
implementing the 
relevant reforms in 
the policy areas as 
planned in the 
CSP and enhance 
their impacts on 
the ground? 

303 

Extent to which technical support 
provided under SRSP can have 
impacts on the ground under 
specific conditions 

Indicators 

Source of information 

 

 

 

 

 

 

Percentage of projects whose 
expected results (related to 
the enhancement of the 
efficiency and effectiveness 
of human-resource 
management) have been 
achieved 
Perceived impact of 
contextual factors on the 
projects’ implementation 
Percentage of relevant 
reforms implemented in the 
policy areas as planned in the 
cooperation and support 
plans 
Percentage of projects whose 
expected results have been 
produced 
The perception of 
stakeholders on the specific 
conditions under which 
reforms designed under 
SRSP will have impacts on 
the ground 
Types of technical support 
provided under SRSP 
contributing to 
implementation of relevant 
reforms for which impacts 
have been observed 

 

 

the project managers 
The results from the 
targeted online 
consultation 

Information available in 
the Commission’s 
internal IT system 
updated in accordance 
with the results of the 
targeted interviews with 
the project managers 

  Desk research (type of 

 

 

documents: EU legal 
acts, other Commission 
resources) 
Information available in 
the Commission’s 
internal IT system 
updated in accordance 
with the results of the 
targeted interviews with 
the project managers 
Information collected 
through the targeted 
interviews and the 
interviews conducted in 

71 

 
Evaluation 
criteria 

Evaluation question (EQ) 

Details 

Number of 
evaluated projects 
in the sample 

Indicators 

Source of information 

Effectiveness - 
impacts 

Effectiveness - 
impacts 

EQ.11.  To what 
extent can 
observed impacts 
be causally linked 
to the 
implementation of 
the reforms and to 
the support 
provided by the 
programme? 

Extent to which observed impacts 
are casually linked to the 
implementation of the reforms and 
to the support provided by the 
programme 

303 

EQ.12.  To what 

extent have the 
expected results 
set in the 
cooperation and 
support plans been 
achieved so far? 

The extent to which the SRSP 
projects achieved the expected 
results 

Factors that influence on the 
observed impacts of the technical 
support 

303 

72 

 

 

 

 

 

The perception of the 
stakeholders regarding the 
impacts of the technical 
support 
Types of impacts produced of 
the technical support 
provided under SRSP 

Percentage of projects whose 
expected results have been 
produced 
Percentage of projects whose 
expected results have a high 
likelihood of being achieved 
The perception of the 
stakeholders regarding the 
factors that have an influence 
on the observed changes 
linked to the technical 
support 

the context of the case 
studies 

 

 

 

The open results of the 
targeted online 
consultation 
Information available in 
the Commission’s 
internal IT system 
updated in accordance 
with the results of the 
targeted interviews with 
the project managers; 
Information collected 
through the interviews 
conducted in the context 
of the case studies 

  Desk research 
 

Information available in 
the Commission’s 
internal IT system 
updated in accordance 
with the results of the 
targeted interviews with 
the project managers 
The open results of the 
targeted online 
consultation 

 

 
 
Evaluation 
criteria 

Evaluation question (EQ) 

Details 

Efficiency 

EQ.13.  To what 
extent is the 
governance of the 
SRSP efficient and 
how efficient is 
the cooperation 
with other 
Commission 
Services? 

Allocation of responsibilities 
among all stakeholders involved 
ensures the achievement of the 
SRSP objectives together with an 
optimal use of the financial and 
human resources. This implies the 
absence of bottlenecks or needs 
for simplifications / streamlining 
of processes / procedures 

Implementation of the programme, 
including its monitoring, is 
aligned with the needs of the 
target groups (i.e. beneficiary 
authorities) 

All relevant stakeholders are 
involved in an effective manner in 
order to ensure that all right types 
of expertise are available in all 
programme phases (i.e. effective 
cooperation) 

Efficiency 

EQ.14.  To what 

extent have the 
actions been cost-
effective so far? 

Achieved results are 
commensurate to the budget 

Number of 
evaluated projects 
in the sample 

126 

The analysis related 
to this component 
considers only 
SRSP 2018, which 
was the first year 
that allowed for the 
planned timeline to 
be followed 
according to the 
SRSP Regulation 
(i.e. due to the late 
adoption of the 
SRSP Regulation, 
SRSP 2017 had to 
follow a shortened 
timeline). The 
assessment does 
not consider PA, 
SRSP 2017 and 
Article 11 given 
that, due to their 
specific 
characteristic as 
well as contextual 
factors, they 
followed non-
standard timelines 

201 

The cost-
effectiveness 
analysis was 

73 

Indicators 

Source of information 

 

 

 

 

Existence of bottlenecks and 
needs for simplification / 
streamlining of processes / 
procedures and of roles and 
responsibilities 
Extent to which the SRSP 
monitoring procedures and 
processes, including tools, 
are adequate and efficiently 
undertaken / applied 
Perception of the 
stakeholders on the 
coordination between the 
SRSS and Commission 
departments 
The extent to which the 
analysis and the selection 
procedures allow the 
selection of good quality 
technical support requests 

 

The extent to which the 
achieved results are 
correlated to the approved 
financial resources 

 

Information available in 
the Commission’s 
internal IT system 
updated in accordance 
with the results of the 
targeted interviews with 
the project managers  
  Desk research (type of 

 

 

 

documents: EU legal 
acts, other Commission 
resources)  
The results from the 
targeted online 
consultation 
Information collected 
through the interviews 
conducted in the context 
of the case studies 
Targeted interviews with 
Directorates-General 
(those interviewed were: 
DG COMP, DG EMPL, 
DG JUST, DG REGIO, 
DG SANTE, DG 
TAXUD) 

  Data provided by the 

 

Commission in February 
2019 

Information available in 
the Commission’s 
internal IT system 
updated in accordance 

 
Evaluation 
criteria 

Evaluation question (EQ) 

Details 

Efficiency 

EQ.15.  How efficient 
is the support 
provided in terms 
of design, means 
of delivery and 
time taken from 
the reception of 
requests from 
Member States to 
the start of the 
action's 
implementation? 

Design of the technical support 
was undertaken in the needed 
timeline 

Quality of the technical support 
requests increased over time 

Means of delivery included were 
the most appropriate to meet the 
objectives of the project(s) and to 
cover the needs of the beneficiary 
authorities 

Indicators 

Source of information 

with the results of the 
targeted interviews with 
the project managers 

  Average duration of the 

  Desk research (type of 

design of the technical 
support requests 
  Average duration of the 

 

 

design of the technical 
support projects 
Perception of the 
stakeholders on the timelines 
of the design of the technical 
support 
The extent to which the 
quality of the technical 
support requests increased 
from one SRSP annual cycle 
to another 

documents: EU legal 
acts, EU preparatory 
documents, other 
European Commission 
resources, online 
resources) 
Information available in 
the Commission’s 
internal IT system 
updated in accordance 
with the results of the 
targeted interviews with 
the project managers 
Desk research (type of 
documents: online 
resources) 
The results from the 
targeted online 

 

 

Number of 
evaluated projects 
in the sample 
possible only for 
those projects (201 
out of the 303 in 
the sample of 
projects evaluated) 
for which 
information is 
available regarding 
both approved 
financial resources 
and progress 
achieved with the 
expected results 

126 

The analysis related 
to this component 
considers only 
SRSP 2018, which 
was the first year 
that allowed for the 
planned timeline to 
be followed 
according to the 
SRSP Regulation. 
The assessment 
does not consider 
the Preparatory 
Action, SRSP 2017 
and Article 11 
given that, due to 
their specific 
characteristic as 

74 

 
Evaluation 
criteria 

Evaluation question (EQ) 

Details 

Efficiency 

EQ.16.  How time-

efficient is budget 
execution from 
commitments to 
payments? 

Financial resources have been 
contracted in a time-efficient 
manner 

Coherence 

EQ.17.  To what 

extent is the SRSP 
coherent both 

Extent to which the logic of 
intervention is internally coherent 

Number of 
evaluated projects 
in the sample 
well as contextual 
factors, they 
followed non-
standard timelines 

280 

The analysis covers 
projects under 
SRSP 2017, SRSP 
2018 and Article 11 
(2018). Projects 
and means of 
delivery concerning 
the Preparatory 
Action (2016) are 
not addressed under 
this EQ due to 
limited availability 
of information. 

303 

75 

Indicators 

Source of information 

 

consultation 
Information collected 
through the interviews 
conducted in the context 
of the case studies 
  Dataset with the scores 
of the technical support 
requests for SRSP 2017 
and SRSP 2018, 
consulted on 14 
February 2019 
Information available in 
the Commission’s 
internal IT system 
updated in accordance 
with the results of the 
targeted interviews with 
the project managers 

 

  Alignment of the absorption 
capacity to the status of the 
projects 
The extent to which the 
means of delivery are marked 
by delays 

 

 

The extent to which the 
SRSP LoI is internally 
coherent 

  Desk research (EU 
regulations, reports, 
decisions from European 

 
Evaluation 
criteria 

Evaluation question (EQ) 

Details 

Number of 
evaluated projects 
in the sample 

Indicators 

Source of information 

internally with its 
own set of 
objectives, and 
externally with 
other Union 
programmes, 
instruments and 
policies at, Union 
and international 
level? 

The perception of the stakeholders 
with respect to overlaps, 
complementarities and synergies. 

The extent to which there are other 
policies and programmes 
implemented with European funds 
that have similar scope and 
objectives with SRSP. 

The extent to which the SRSS 
general / specific objectives and 
general / specific objectives of 
similar programmes and policies 
at regional, national, Union and 
international level are coherent. 

Number of other EU instruments 
that have similar objectives like 
SRSP 

303 

EQ.18.  To what 
extent some 
actions supported 
by the SRSP could 
have also been 
supported by other 
EU Instruments? 

Reasons for choosing SRSP over 
alternative EU instruments with 
similar objectives 

Coherence 

76 

 

 

 

The perception of the 
stakeholders with respect to 
overlaps, complementarities 
and synergies 
The extent to which there are 
other policies and 
programmes implemented 
with European funds that 
have similar scope and 
objectives with SRSP 
The extent to which the 
SRSS general / specific 
objectives and general / 
specific objectives of similar 
programmes and policies at 
regional, national, Union and 
international level are 
coherent 

  Number of other EU 

instruments that have similar 
objectives like SRSP 
  Reasons for choosing SRSP 

over alternative EU 
instruments with similar 
objectives 

 

 

 

 

 

 

Commission) 
The results from the 
targeted online 
consultation 
The results from scoping 
interviews with HoU / 
Hoc, deputy HoU and 
advisers, country 
coordinators 
Targeted interviews with 
Directorates-General 
(those interviewed were: 
DG COMP, DG EMPL, 
DG JUST, DG REGIO, 
DG SANTE, DG 
TAXUD) 

The results from targeted 
online consultation 
The results from scoping 
interviews HoU / Hoc, 
deputy HoU and 
advisers, country 
coordinators 
Targeted interviews with 
Directorates-General 
(those interviewed were: 
DG COMP, DG EMPL, 
DG JUST, DG REGIO, 
DG SANTE, DG 
TAXUD) 

 
Evaluation 
criteria 

Evaluation question (EQ) 

Details 

Percentage of SRSP projects 
answering to recommendations 
identified / issued under the 
European Semester review 
mechanisms 

Percentage of projects related to 
recommendations issued under the 
European Semester reviews 
mechanism whose expected 
results have been achieved / have 
a high likelihood of being 
achieved 

Variation in the extent to which 
support under SRSP is requested 
by Member States. 

Perception of the beneficiary 
authorities of the added value 
provided by SRSP compared to 
alternative instruments 

EQ.19.  To what 

extent is the SRSP 
integrated in the 
Economic 
governance 
processes of the 
EU, such as the 
European 
Semester? 

EQ.20.  To what 

extent does the 
SRSP add value 
compared to 
interventions led 
at national level by 
Member State (or 
other key donors 
and partners)? 

EQ.21.  To what 
extent has the 
programme 
produced cross 
border cooperation 
in the areas of 

The extent to which the 
beneficiary authorities requested 
technical support to address cross-
border needs 

The extent to which projects from 
different Member States have been 

Coherence 

EU added value 

EU added value 

Indicators 

Source of information 

 

 

Percentage of SRSP projects 
answering to 
recommendations identified / 
issued under the European 
Semester review mechanisms 
Percentage of projects related 
to recommendations issued 
under the European Semester 
reviews mechanism whose 
expected results have been 
achieved / have a high 
likelihood of being achieved 

  Number of new Member 
States that submitted 
technical support requests for 
the first time under each 
budgetary cycle (PA (2016) 

 

 

Perception of beneficiary 
authorities and coordinating 
authorities of the extent to 
which the needs have had a 
cross-border component 
Percentage of projects 

The results of the case 
study interviews 

Information available in 
the Commission’s 
internal IT system 
updated in accordance 
with the results of the 
targeted interviews with 
the project managers, 
Requests for technical 
support 

 

 

 

  Data provided by the EC 

in February 2019 
regarding the technical 
support requests; 
Information available in 
the Commission’s 
internal IT system 

Information collected 
through the interviews 
conducted in the context 
of the case studies 
Information available in 
the Commission’s 

 

 

 

Number of 
evaluated projects 
in the sample 

257 

The 46 projects 
implemented in 
Greece were 
excluded from this 
analysis, as Greece 
was not part of the 
European Semester 
until mid-2018, 
when it exited 
economic 
adjustment 
programmes. 

303 

303 

77 

 
 
Evaluation 
criteria 

Evaluation question (EQ) 

Details 

Number of 
evaluated projects 
in the sample 

Indicators 

Source of information 

intervention? 

implemented to address common 
regional challenges 

designed based on the 
technical support requests of 
at least two neighbouring 
Member States 

EU added value 

The extent to which the technical 
support produced cross-border 
impacts 

The extent to which technical 
support was delivered by using 
good practices and exchange of 
information and experiences 

303  

The extent to which the technical 
support increased cooperation 
among institutions in different 
Member States 

181 means of 
delivery 

The assessment 
was performed at 
means of delivery 
level because 
information about 
the type of provider 
was only available 
at this level.  

181 is the number 
of means of 
delivery that used 
good practices and 
lessons learned 

78 

EQ.22.  To what 
extent is the 
programme 
contributing to 
sharing good 
practices among 
Member States, 
also with a view to 
increasing the 
visibility of the 
reform 
programmes, 
better identifying 
the need for 
possible 
accompanying 
measures and / or 
sequencing of 
reforms, and to 

 

 

 

  Number of projects expected 
to bring added-value through 
an impact on cross-border or 
EU-wide challenges 
Perception of beneficiary 
authorities and coordinating 
authorities of the cross-
border impacts 
Percentage of projects that 
used good practices and 
lessons learned 
Percentage of means of 
delivery that used good 
practices and lessons learned 
Percentage of means of 
delivery that include short-
term expert mission(s) 
Percentage of means of 
delivery that include study 
visit (of officials of the 
beneficiary Member State) 
Percentage of means of 
delivery that include 
workshops/conferences/semi
nars 
Types of outputs developed 
using good practices 

 

 

 

 

internal IT system 
updated in accordance 
with the results of the 
targeted interviews with 
the project managers 

Information available in 
the Commission’s 
internal IT system 
updated in accordance 
with the results of the 
targeted interviews with 
the project managers 
The results of the 
targeted online 
consultation 

 

 

 

 
 
Evaluation 
criteria 

Evaluation question (EQ) 

Details 

Number of 
evaluated projects 
in the sample 

Indicators 

Source of information 

building a Union-
wide platform and 
network of 
expertise? 

The extent to which the exchange 
of good practices among Member 
States contributed to the visibility 
of the reform programmes 

303 

 

 

 

 

Percentage of means of 
delivery that have been 
implemented by government 
bodies, public servant expert, 
public law bodies and bodies 
governed by private law with 
a public service mission 
Percentage of respondents to 
the targeted online 
consultation who believe that 
the technical support 
increased the sharing of 
knowledge and good 
practices among countries 
The extent to which the 
monitoring database of SRSS 
collects information on the 
results/impacts of the projects 
Perception of the 
coordinating authorities of 
the extent to which some 
actions have been taken to 
ensure visibility of SRSP 
technical support projects 

79